AUdio models for RespiratOry and cardiac diagnostics and clinical tRAining

Objective

Human sounds, both bodily sounds (e.g., heart sounds) or voice, have been used by doctors for centuries as diagnostic signals for health and disease progression. However, while stethoscopes and microphones are very affordable, diagnosing through sound is a challenging task for medical professionals and requires years of training. The clinical training is generally based on shadowing experienced clinicians and repeating auscultation: this has scalability limits and imposes a considerable burden onto the health system. Machine learning for human sounds has beHuman sounds, being them bodily sounds (e.g., heart sounds) or voice, have been used by doctors for centuries as diagnostics signals for health and disease progression. However, while stethoscopes and microphones are very affordable, diagnosing through sound is a challenging task for medical professionals and requires years of training. The clinical training is generally based on shadowing experienced clinicians and repeating auscultation: this has scalability limits and imposes a considerable burden onto the health system. Machine learning for human sounds has been explored by the research community and is showing promise. In project ERC EAR we have advanced the state of the art in this respect for cardiac and respiratory health tasks. We showed that models could complement the skills of clinicians when diagnosing and constructed powerful and pioneering open pretrained models able to generalize to unseen cardiac and respiratory tasks with very good performance and limited labelled data for those tasks. In project AURORA, we will leverage the research results and develop a software framework that aids clinical training on respiratory and cardiac sounds and acts as a support tool for diagnostics. This will unlock the potential of human sound based diagnostics, by helping overcoming its inherent complexity and allow to leverage its inherent affordability and therefore contributing to democratization of health.