Sonic Sculpting

Robotics has rapidly advanced, with mobile robots now used in many applications such as for inspection, service, and construction monitoring. These applications require reliable navigation in complex and dynamic environments. Current systems often rely on vision-based sensors such as cameras and LiDAR, which work well in many cases but struggle in low-light, dusty, or cluttered settings where visibility and data quality are compromised. Our project proposes a sound-based solution, using room impulse responses (RIRs) and generative AI to reconstruct 3D geometry directly from acoustic cues. Previous research has advanced sound event localization research through identifying both the type of sound and its spatial origin. Our project extends this foundation by adding the ability to translate sound-based spatial information into actionable navigation, positioning it as a natural enhancement to Robotics’ navigation, which focuses on path planning in complex and dynamic environments such as construction sites. The project will develop a multimodal reasoning framework that treats sound as the primary channel for reconstructing 3D geometry. By combining diffusion and flow-based generative AI models with neural fields, the system will translate raw acoustic cues into spatial meshes, even where cameras cannot operate. This capability supports autonomous-systems navigation and real-time construction monitoring, offering a scalable pathway to enhance robotic navigation in other dynamic environments.