ODD-ML: Out-of-Distribution Deployable Machine Learning

Objective

ODD-ML addresses the open secret of machine learning (ML), which is that model deployment often fails. The problem arises because deployment contexts may differ from the data used to train ML models in unexpected ways. In an increasingly data-driven era, this severely impedes progress in ML-powered R&D and our ability to tackle societal grand challenges with existing ML tools. To solve this pervasive issue, I propose a radical alternative to current ML approaches, placing human experts at the core of iterative design-build-test-learn (DBTL) loops. My approach comprises the interlinked steps of re-conceptualizing the deployment issue as a need for active learning from domain experts and other indirect sources and, to succeed here, recognizing the imperfect and often tacit knowledge and limited time of human experts, designing ML systems that can rapidly reverse-engineer expert knowledge. I will achieve this with a combination of ideas transformative for human-AI collaboration: human inductive biases will be inferred from computational-rationality-based cognitive models, amortized on pre-computed solutions for speed, allowing interactive online use. I envision widespread impact in ML, on complex decision-making, and broadly across R&D domains.