Scalable spatial sensing and dynamic reconstruction of complex sound fields

Objective

SONIFY aims at developing the first ever unified theoretical and experimental framework to measure and characterize sound fields over extended three-dimensional domains, providing scalable audio sensing and delivering models that can synthesize virtual sound fields from experimental observations of the space. Classical measurement methods lead to an intractable problem: the number of measurements to characterize a sound field grows drastically with the size of the domain, and the sound field varies if the source moves, changes, or additional sources appear. In SONIFY, I conceive a ground-breaking approach to fully characterize an acoustic space from experimental observations, to predict or estimate any possible sound field in it. The key novelty lies in transforming how we characterize the propagation of sound in complex three-dimensional media: instead of modelling the observed sound pressure field directly, SONIFY focuses on modelling the propagation and underlying physical structure of the sound field. The research constitutes a novel framework that draws from acoustic holography, elementary wave models, deep learning and novel sensing approaches. The framework is scalable and enables to characterize sound fields over large domains (on the order of hundreds of cubic meters or more) and reduces the experimental costs by four orders of magnitude. SONIFY is set to deliver unprecedented sound field analysis and manipulation ability, leading to a seamless integration between real and synthesised sound. Ultimately, it brings about a new area that integrates experimental, virtual and computational methods in acoustics and audio. The framework also serves to tackle withstanding challenges in the field, and to address open scientific problems in disciplines concerned with the analysis and manipulation of wave-fields.