Connect with us

Computer Graphics

3D Streaming Gets Leaner: Predicting Visible Content for Immersive Experiences

A new approach to streaming technology may significantly improve how users experience virtual reality and augmented reality environments, according to a new study. The research describes a method for directly predicting visible content in immersive 3D environments, potentially reducing bandwidth requirements by up to 7-fold while maintaining visual quality.

Avatar photo

Published

on

A groundbreaking new approach to 3D streaming has emerged, poised to revolutionize how users experience virtual reality (VR) and augmented reality (AR) environments. Researchers at NYU Tandon School of Engineering have developed a method for directly predicting visible content in immersive 3D environments, potentially reducing bandwidth requirements by up to 7-fold while maintaining visual quality.

This innovative technology addresses the fundamental challenge of streaming immersive content: the massive amount of data required to render high-quality 3D experiences. Traditional video streaming sends everything within a frame, but this new approach is more like having your eyes follow you around a room – it only processes what you’re actually looking at.

The system works by dividing 3D space into “cells” and treating each cell as a node in a graph network. It uses transformer-based graph neural networks to capture spatial relationships between neighboring cells, and recurrent neural networks to analyze how visibility patterns evolve over time. This approach reduces error accumulation and improves prediction accuracy, allowing the system to predict what will be visible for a user 2-5 seconds ahead – a significant improvement over previous systems that could only accurately predict a user’s field of view (FoV) a fraction of a second ahead.

The research team’s approach has been applied in an ongoing project to bring point cloud video to dance education, making 3D dance instruction streamable on standard devices with lower bandwidth requirements. This technology has the potential to transform the way people experience immersive content, enabling more responsive AR/VR experiences with reduced data usage and allowing developers to create more complex environments without requiring ultra-fast internet connections.

“We’re seeing a transition where AR/VR is moving from specialized applications to consumer entertainment and everyday productivity tools,” said Yong Liu, professor in the Electrical and Computer Engineering Department at NYU Tandon. “Bandwidth has been a constraint. This research helps address that limitation.”

Brain Injury

Unlocking the Secrets of the Brain with Digital Twins

In a new study, researchers created an AI model of the mouse visual cortex that predicts neuronal responses to visual images.

Avatar photo

Published

on

The researchers used a combination of AI and neuroscience techniques to create the digital twin. They first recorded the brain activity of real mice as they watched movies made for people, which provided a realistic representation of what the mice might see in natural settings. The films were action-packed and had a lot of movement, which strongly activated the visual system of the mice.

The researchers then used this data to train a core model, which could be customized into a digital twin of any individual mouse with additional training. These digital twins were able to closely simulate the neural activity of their biological counterparts in response to a variety of new visual stimuli, including videos and static images.

The large quantity of aggregated training data was key to the success of the digital twins, allowing them to make accurate predictions about the brain’s response to new situations. The researchers verified these predictions against high-resolution imaging of the mouse visual cortex, which provided unprecedented detail.

This technology has significant implications for the field of neuroscience. By creating a digital twin of the mouse brain, scientists can perform experiments on a realistic simulation of the brain, allowing them to gain insights into how the brain processes information and the principles of intelligence.

The researchers plan to extend their modeling into other brain areas and to animals, including primates, with more advanced cognitive capabilities. This could ultimately lead to the creation of digital twins of at least parts of the human brain, which would be a major breakthrough in the field of neuroscience.

Content:

Unlocking the Secrets of the Brain with Digital Twins

A group of researchers have created a digital twin of the mouse brain, which can predict the response of tens of thousands of neurons to new videos and images. This AI model has been trained on large datasets of brain activity collected from real mice watching movie clips.

The digital twin is an example of a foundation model, capable of learning from large datasets and applying that knowledge to new tasks and new types of data. This technology has the potential to revolutionize the field of neuroscience, allowing scientists to perform experiments on a realistic simulation of the mouse brain and gaining insights into how the brain processes information.

The researchers used a combination of AI and neuroscience techniques to create the digital twin. They first recorded the brain activity of real mice as they watched movies made for people, which provided a realistic representation of what the mice might see in natural settings. The films were action-packed and had a lot of movement, which strongly activated the visual system of the mice.

The researchers then used this data to train a core model, which could be customized into a digital twin of any individual mouse with additional training. These digital twins were able to closely simulate the neural activity of their biological counterparts in response to a variety of new visual stimuli, including videos and static images.

The large quantity of aggregated training data was key to the success of the digital twins, allowing them to make accurate predictions about the brain’s response to new situations. The researchers verified these predictions against high-resolution imaging of the mouse visual cortex, which provided unprecedented detail.

This technology has significant implications for the field of neuroscience. By creating a digital twin of the mouse brain, scientists can perform experiments on a realistic simulation of the brain, allowing them to gain insights into how the brain processes information and the principles of intelligence.

The researchers plan to extend their modeling into other brain areas and to animals, including primates, with more advanced cognitive capabilities. This could ultimately lead to the creation of digital twins of at least parts of the human brain, which would be a major breakthrough in the field of neuroscience.

Funding:

The study received funding from the Intelligence Advanced Research Projects Activity, a National Science Foundation NeuroNex grant, the National Institute of Mental Health, the National Institute of Neurological Disorders and Stroke (grant U19MH114830), the National Eye Institute (grant R01 EY026927 and Core Grant for Vision Research T32-EY-002520-37), the European Research Council and the Deutsche Forschungsgemeinschaft.

Continue Reading

Artificial Intelligence

Breaking Barriers: Scientists Discover Copper-Free High-Temperature Superconducting Oxide

Engineers have designed and synthesized a groundbreaking new material — a copper-free superconducting oxide — capable of superconducting at approximately 40 Kelvin, or about minus 233 degrees Celsius, under ambient pressure.

Avatar photo

Published

on

Breaking Barriers: Scientists Discover Copper-Free High-Temperature Superconducting Oxide

A groundbreaking new material has been designed and synthesised by Professor Ariando and Dr Stephen Lin Er Chow from the National University of Singapore (NUS) Department of Physics. This copper-free superconducting oxide is capable of operating at approximately 40 Kelvin, or about minus 233 degrees Celsius, under ambient pressure.

This discovery marks a significant advancement in high-temperature superconductivity research, which has been a topic of interest for nearly four decades. The team’s breakthrough is the first since the Nobel-winning discovery of copper oxide superconductivity to find a copper-free material functioning under ambient pressure.

The promise of superconductors lies in their ability to eliminate energy loss due to electrical resistance, making them ideal for modern electronic applications. However, the vast majority of discovered superconducting materials function only at extremely low temperatures near absolute zero, making them impractical for widespread use.

Professor Ariando and Dr Chow’s research has identified a direct correlation between interlayer interactions in layered systems and superconducting temperatures. Building on this insight, they developed a phenomenological model that predicted several compounds capable of high-temperature superconductivity, similar to copper oxides, but without copper.

The team successfully synthesised (Sm-Eu-Ca)NiO₂ nickel oxide, one of the predicted materials, and confirmed zero electrical resistance (superconductivity) well above 30 K in this compound. This discovery has profound implications for both theoretical understanding and experimental realisation of a broader scope of superconducting materials with practical applications in modern electronics.

The research breakthrough was published in the scientific journal Nature on 20 March 2025, and it represents a major step toward the development of next-generation superconducting materials, with practical applications in modern electronics and energy-efficient technologies.

Continue Reading

Computer Graphics

The Power of Perception: How Pain and Fear Can Weaken Our Sense of Body Ownership

A study found that when people were told to imagine their virtual bodies in pain, their brains resisted the illusion of ownership. Their findings could provide insights into why some people may struggle with feeling connected to their own bodies, particularly in contexts involving depersonalization or negative physical states.

Avatar photo

Published

on

The human brain has an incredible ability to perceive and interpret its surroundings, but how we perceive our own bodies can be just as complex. A recent study from Hiroshima University explored the connection between pain, fear, and the sense of body ownership – a crucial aspect of our self-perception that allows us to distinguish ourselves from objects and respond to threats.

Researchers used virtual reality (VR) headsets to create an illusion where participants felt a virtual body as their own. They then instructed participants to imagine this virtual body in pain while being stroked on the back, creating a realistic scenario that simulated both tactile and visual sensations. The goal was to understand how our brain processes this information and whether top-down factors – previous knowledge, memories, and beliefs – can influence our perception of body ownership.

The study found that when participants were told to imagine their virtual bodies in pain, their brains resisted the illusion of ownership. This inhibition of the full-body illusion (FBI) was more pronounced in individuals with depersonalization tendencies – a condition where people struggle to feel their body as their own. The researchers suggested that this could be due to multiple factors, including the manipulation of top-down factors or participants’ difficulty perceiving negative physical symptoms.

The findings of this study have significant implications for clinical intervention and understanding disturbed body ownership in individuals with conditions like depersonalization-derealization disorder. By improving our knowledge of how pain and fear can influence our sense of body ownership, we can develop more effective treatments that target the root causes of these disorders, ultimately improving lives and enhancing sensory and perception purposes.

Continue Reading

Trending