Code Freeze 2025 speaker: Karen Haigh

Assuring Cognitive Systems

Abstract

This presentation explores validating systems that learn autonomously in the field from limited data, without cloud communication or human supervision. Traditional ML testing, designed for static models, often fails in novel conditions, leading to inaccuracies. Unlike these static models, cognitive systems use self-supervised reinforcement feedback to adapt through interactions with their environment. Consequently, the focus shifts from validating the model to validating the learning process itself. A closed-loop testing infrastructure is essential to simulate novel experiences and ensure robust validation of these dynamic systems

Biography

Dr. Karen Haigh is an expert and consultant in cognitive electronic warfare and embedded AI. Her focus is on physical systems with limited communications and limited computation resources that must perform under fast hard-real-time requirements. She was a pioneer in three fields now common across the globe: (1) closed-loop planning and machine learning for autonomous robots, (2) smart homes for elder care, and (3) cognitive RF systems. She recently wrote the book "Cognitive EW: An AI Approach" with Julia Andrusenko.  Dr. Haigh has created a variety of online content discussing embedded AI for mission-critical systems, supporting rapid real-time in-mission learning, and assuring AI in the field. She received her PhD in from Carnegie Mellon University in Computer Science with a focus on AI and Robotics, and her undergraduate from the University of Ottawa in Computer Science. She is a Fellow of IEEE and AAIA.