The ability of artificial intelligence systems to engage in natural conversations is a remarkable feat. However, despite this progress, the internal processes that lead to such results remain largely unknown. A recent study published in the Journal of Statistical Mechanics: Theory and Experiment (JSTAT) has shed light on this mystery. The research reveals that when small amounts of data are used for training, neural networks initially rely on the position of words in a sentence. However, as the system is exposed to enough data, it transitions to a new strategy based on the meaning of the words.

This transition occurs abruptly, once a critical data threshold is crossed – much like a phase transition in physical systems. The findings offer valuable insights into understanding the workings of these models. Just as a child learning to read starts by understanding sentences based on the positions of words, a neural network begins its journey by relying on word positions. However, as it continues to learn and train, the network “keeps going to school” and develops a deeper understanding of word meanings.

This shift is a critical discovery in the field of artificial intelligence. The researchers used a simplified model of self-attention mechanism – a core building block of transformer language models. These models are designed to process sequences of data, such as text, and form the backbone of many modern language systems.

The study’s lead author, Hugo Cui, explains that the network can use two strategies: one based on word positions and another on word meanings. Initially, the network relies on word positions, but once a certain threshold is crossed, it abruptly shifts to relying on meaning-based strategies. This transition is likened to a phase transition in physical systems, where the system undergoes a sudden, drastic change.

Understanding this phenomenon from a theoretical viewpoint is essential. The researchers emphasize that their findings can provide valuable insights into making neural networks more efficient and safer to use. The study’s results are published in JSTAT as part of the Machine Learning 2025 special issue and included in the proceedings of the NeurIPS 2024 conference.

The research by Cui, Behrens, Krzakala, and Zdeborová, titled “A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention,” offers new knowledge that can be used to improve the performance and safety of artificial intelligence systems. The study’s findings have significant implications for the development of more efficient and effective language models, ultimately leading to advancements in natural language processing and understanding.

The Niels Bohr Institute at the University of Copenhagen has made a groundbreaking discovery that could revolutionize the way we transmit information. Researchers, in collaboration with the University of Konstanz and ETH Zurich, have successfully sent vibrations through an ultra-thin drumhead, measuring only 10 mm wide, with astonishingly low loss – just one phonon out of a million. This achievement is even more impressive than electronic circuit signal handling.

The drumhead, perforated with many triangular holes, utilizes the concept of phonons to transmit signals. Phonons are essentially sound waves that travel through solid materials by vibrating atoms and pushing each other. This phenomenon is not unlike encoding a message and sending it through a material, where signal loss can occur due to various factors like heat or incorrect vibrations.

The researchers’ success lies in achieving almost lossless transmission of signals through the membrane. The reliability of this platform for sending information is incredibly high, making it a promising candidate for future applications. To measure the loss, researchers directed the signal through the material and around the holes, observing that the amplitude decreased by only about one phonon out of a million.

This achievement has significant implications for quantum research. Building a quantum computer requires super-precise transfer of signals between its different parts. The development of sensors capable of measuring the smallest biological fluctuations in our own body also relies heavily on signal transfer. As Assistant Professor Xiang Xi and Professor Albert Schliesser explain, their current focus is on exploring further possibilities with this method.

“We want to experiment with more complex structures and see how phonons move around them or collide like cars at an intersection,” says Albert Schliesser. “This will give us a better understanding of what’s ultimately possible and what new applications there are.” The pursuit of basic research is about producing new knowledge, and this discovery is a testament to the power of scientific inquiry.

In conclusion, the quantum drumhead revolution has brought us one step closer to achieving near-perfect signal transmission. As researchers continue to explore the possibilities of this method, we can expect exciting breakthroughs in various fields, ultimately leading to innovative applications that will transform our understanding of the world.

The Massachusetts General Brigham researchers have developed an innovative artificial intelligence (AI) tool called AI-CAC to analyze previously collected CT scans and identify individuals with high coronary artery calcium (CAC) levels, indicating a greater risk for cardiovascular events. Their research, published in NEJM AI, demonstrated the high accuracy and predictive value of AI-CAC for future heart attacks and 10-year mortality.

Millions of chest CT scans are taken each year, often in healthy people, to screen for lung cancer or other conditions. However, this study reveals that these scans can also provide valuable information about cardiovascular risk, which has been going unnoticed. The researchers found that AI-CAC had a high accuracy rate (89.4%) at determining whether a scan contained CAC or not.

The gold standard for quantifying CAC uses “gated” CT scans, synchronized to the heartbeat to reduce motion during the scan. However, most chest CT scans obtained for routine clinical purposes are “nongated.” The researchers developed AI-CAC, a deep learning algorithm, to probe through these nongated scans and quantify CAC.

The AI-CAC model was 87.3% accurate at determining whether the score was higher or lower than 100, indicating a moderate cardiovascular risk. Importantly, AI-CAC was also predictive of 10-year all-cause mortality, with those having a CAC score over 400 having a 3.49 times higher risk of death over a 10-year period.

The researchers hope to conduct future studies in the general population and test whether the tool can assess the impact of lipid-lowering medications on CAC scores. This could lead to the implementation of AI-CAC in clinical practice, enabling physicians to engage with patients earlier, before their heart disease advances to a cardiac event.

As Dr. Raffi Hagopian, first author and cardiologist at the VA Long Beach Healthcare System, emphasized, “Using AI for tasks like CAC detection can help shift medicine from a reactive approach to the proactive prevention of disease, reducing long-term morbidity, mortality, and healthcare costs.”