CSE DSI Machine Learning Seminar - David I. Inouye (ECE, Purdue University)

Machine learning models often rely on correlations that hold during training but fail in new environments. How can we teach a model which variation should not affect its predictions? Invariant pairs—such as images of the same bird against different backgrounds—provide one way to communicate this information. Yet a limited collection may miss relevant variation, and noise can cause invariance constraints to suppress useful predictive information. 

 

This talk introduces Geometric Robustness Invariant Training (GRIT), which estimates a subspace from invariant pair differences and projects it out of pretrained features before fitting a predictor. I will discuss how pair count, noise, geometric coverage, and the number of removed directions affect out-of-distribution robustness, and how this approach compares with prediction-consistency regularization. Experiments illustrate benefits across several learning objectives and the value of combining pairs that capture different sources of variation. I will close with directions for extending the learned geometry and acquiring invariant pairs, motivated by the broader question of how data can specify what a model should ignore.