The sinking of the Titanic 113 years ago was a tragic reminder of the importance of accurate navigation. The ship’s encounter with an iceberg led to one of the most infamous maritime disasters in history, and human error likely played a significant role. Today, autonomous systems powered by artificial intelligence (AI) are being developed to help ships avoid such accidents. However, for these systems to be widely adopted, it is crucial that they can provide transparent explanations for their actions.

Researchers from Osaka Metropolitan University’s Graduate School of Engineering have made a significant breakthrough in this area. They have created an explainable AI model specifically designed for ship navigation, which quantifies the collision risk for all vessels in a given area. This feature is particularly important as key sea-lanes have become increasingly congested, making it more challenging to ensure safe passage.

The researchers, Graduate Student Hitoshi Yoshioka and Professor Hirotada Hashimoto, aimed to develop an AI model that not only makes informed decisions but also provides clear explanations for its actions. By using numerical values to express the collision risk, the system can communicate its reasoning to the captain, enabling them to make more informed decisions.

According to Professor Hashimoto, “By being able to explain the basis for the judgments and behavioral intentions of AI-based autonomous ship navigation, I think we can earn the trust of maritime workers. I also believe that this research can contribute to the realization of unmanned ships.”

The findings of this study have been published in Applied Ocean Research, highlighting the potential for explainable AI to improve ship safety and reduce human error. As the maritime industry continues to evolve, the development of transparent and trustworthy autonomous systems will be essential for ensuring safe and efficient navigation.

A groundbreaking new approach to 3D streaming has emerged, poised to revolutionize how users experience virtual reality (VR) and augmented reality (AR) environments. Researchers at NYU Tandon School of Engineering have developed a method for directly predicting visible content in immersive 3D environments, potentially reducing bandwidth requirements by up to 7-fold while maintaining visual quality.

This innovative technology addresses the fundamental challenge of streaming immersive content: the massive amount of data required to render high-quality 3D experiences. Traditional video streaming sends everything within a frame, but this new approach is more like having your eyes follow you around a room – it only processes what you’re actually looking at.

The system works by dividing 3D space into “cells” and treating each cell as a node in a graph network. It uses transformer-based graph neural networks to capture spatial relationships between neighboring cells, and recurrent neural networks to analyze how visibility patterns evolve over time. This approach reduces error accumulation and improves prediction accuracy, allowing the system to predict what will be visible for a user 2-5 seconds ahead – a significant improvement over previous systems that could only accurately predict a user’s field of view (FoV) a fraction of a second ahead.

The research team’s approach has been applied in an ongoing project to bring point cloud video to dance education, making 3D dance instruction streamable on standard devices with lower bandwidth requirements. This technology has the potential to transform the way people experience immersive content, enabling more responsive AR/VR experiences with reduced data usage and allowing developers to create more complex environments without requiring ultra-fast internet connections.

“We’re seeing a transition where AR/VR is moving from specialized applications to consumer entertainment and everyday productivity tools,” said Yong Liu, professor in the Electrical and Computer Engineering Department at NYU Tandon. “Bandwidth has been a constraint. This research helps address that limitation.”

The researchers used a combination of AI and neuroscience techniques to create the digital twin. They first recorded the brain activity of real mice as they watched movies made for people, which provided a realistic representation of what the mice might see in natural settings. The films were action-packed and had a lot of movement, which strongly activated the visual system of the mice.

The researchers then used this data to train a core model, which could be customized into a digital twin of any individual mouse with additional training. These digital twins were able to closely simulate the neural activity of their biological counterparts in response to a variety of new visual stimuli, including videos and static images.

The large quantity of aggregated training data was key to the success of the digital twins, allowing them to make accurate predictions about the brain’s response to new situations. The researchers verified these predictions against high-resolution imaging of the mouse visual cortex, which provided unprecedented detail.

This technology has significant implications for the field of neuroscience. By creating a digital twin of the mouse brain, scientists can perform experiments on a realistic simulation of the brain, allowing them to gain insights into how the brain processes information and the principles of intelligence.

The researchers plan to extend their modeling into other brain areas and to animals, including primates, with more advanced cognitive capabilities. This could ultimately lead to the creation of digital twins of at least parts of the human brain, which would be a major breakthrough in the field of neuroscience.

Content:

Unlocking the Secrets of the Brain with Digital Twins

A group of researchers have created a digital twin of the mouse brain, which can predict the response of tens of thousands of neurons to new videos and images. This AI model has been trained on large datasets of brain activity collected from real mice watching movie clips.

The digital twin is an example of a foundation model, capable of learning from large datasets and applying that knowledge to new tasks and new types of data. This technology has the potential to revolutionize the field of neuroscience, allowing scientists to perform experiments on a realistic simulation of the mouse brain and gaining insights into how the brain processes information.

The researchers used a combination of AI and neuroscience techniques to create the digital twin. They first recorded the brain activity of real mice as they watched movies made for people, which provided a realistic representation of what the mice might see in natural settings. The films were action-packed and had a lot of movement, which strongly activated the visual system of the mice.

The researchers then used this data to train a core model, which could be customized into a digital twin of any individual mouse with additional training. These digital twins were able to closely simulate the neural activity of their biological counterparts in response to a variety of new visual stimuli, including videos and static images.

The large quantity of aggregated training data was key to the success of the digital twins, allowing them to make accurate predictions about the brain’s response to new situations. The researchers verified these predictions against high-resolution imaging of the mouse visual cortex, which provided unprecedented detail.

This technology has significant implications for the field of neuroscience. By creating a digital twin of the mouse brain, scientists can perform experiments on a realistic simulation of the brain, allowing them to gain insights into how the brain processes information and the principles of intelligence.

The researchers plan to extend their modeling into other brain areas and to animals, including primates, with more advanced cognitive capabilities. This could ultimately lead to the creation of digital twins of at least parts of the human brain, which would be a major breakthrough in the field of neuroscience.

Funding:

The study received funding from the Intelligence Advanced Research Projects Activity, a National Science Foundation NeuroNex grant, the National Institute of Mental Health, the National Institute of Neurological Disorders and Stroke (grant U19MH114830), the National Eye Institute (grant R01 EY026927 and Core Grant for Vision Research T32-EY-002520-37), the European Research Council and the Deutsche Forschungsgemeinschaft.