Events

Making machine learning predictably reliable

Despite ML models’ impressive performance, training and deploying them is currently a somewhat messy endeavor. But does it have to be? In this talk, Andrew Ilyas overviews his work on making ML “predictably reliable” — enabling developers to know when their models will work, when they will fail, and why.

To begin, Ilyas uses a case study of adversarial inputs to show that human intuition can be a poor predictor of how ML models operate. Motivated by this, Ilyas presents a line of work that aims to develop a precise understanding of the ML pipeline, combining statistical tools with large-scale experiments to characterize the role of each individual design choice: from how to collect data, to what dataset to train on, to what learning algorithm to use.