Computer Science Colloquium: On Leaky Models and Unintended Inferences

David Evans (University of Virginia)

Machine learning offers the promise to train models that perform surprisingly well on a wide range of tasks, merely by using massive computing power and generic training algorithms on available data sets. It is an open question, however, what else those models might learn about their training data, and how an adversary with some access to the model may be able to reveal it.

In this talk, I will discuss a variety of inference risks associated with machine-trained models, with a particular focus on surprising (and potentially harmful) things a model may reveal not just about individual training records but about the overall distribution of its training data. This includes attacks an adversary may use to learn statistical properties about the training distribution and about whether certain kinds of data are or are not included, and the potential for an adversary to use a model to make sensitive inferences about individuals, even for attributes not directly related to the task and regardless of whether those individuals are included the training data.

I’ll conclude with some thoughts on why defending against these types of attacks is hard, and what we might learn about how we should be training and exposing models.

About David Evans

David Evans is a Professor of Computer Science at the University of Virginia where he leads research on security and privacy with a recent focus on adversarial machine learning and inference risks in machine learning, and teaches courses on a wide variety of topics including biology, ethics, economics, and theory of computing. He is the author of an open computer science textbook and a children's book on combinatorics and computability and co-author of a book on secure computation.

He won the Outstanding Faculty Award from the State Council of Higher Education for Virginia and is Program Co-Chair for the 2022 and 2023 IEEE European Symposia on Security and Privacy. He was Program Co-Chair for the 24th ACM Conference on Computer and Communications Security (CCS 2017) and the 30th (2009) and 31st (2010) IEEE Symposia on Security and Privacy, where he initiated the Systematization of Knowledge (SoK) papers.

He has SB, SM and PhD degrees in Computer Science from MIT and has been a faculty member at the University of Virginia since 1999.