Beyond Maximum Entropy: A new paradigm for Modeling, Inference, and Learning Efficiency

Objective

Machine learning is revolutionizing science and society, enabling transformative breakthroughs such as predicting protein structures, simulating many body quantum systems, and redefining entire fields like healthcare, finance, or climate modeling. Yet, this rapid progress comes at a cost: the unprecedented scale and complexity of state-of-the-art models make them resource-intensive, inaccessible, and often opaque. These challenges limit their potential, hindering our ability to understand, optimize, and deploy them responsibly. To unlock the full power of machine learning, we must reimagine its foundations. This project challenges the 'bigger is better' paradigm in machine learning by proposing an alternative: the development of simpler, more efficient generative models that can handle the complexity of the real world while remaining analytically interpretable. Using techniques from statistical physics, disordered systems and computational physics, the project aims to uncover how these models encode patterns, address learning challenges and reduce training costs without sacrificing performance or expressiveness. Crucially, these models will be designed to extract meaningful insights from general-purpose data sets, perform reliably in data-scarce scenarios and accelerate simulations in complex systems. By developing a new generation of inference algorithms and showcasing their effectiveness through three proof-of-concept applications in neuroscience, bioinformatics, and turbulence modeling, this research aims to significantly expand the toolbox for data-driven discovery. Beyond tackling practical challenges, it seeks to advance our fundamental understanding of unsupervised learning mechanisms while prioritizing sustainability, transparency, and inclusivity. This project reimagines machine learning as a responsible and accessible tool to drive scientific and societal progress.