Programs, Test Data, and Oracles: Revisiting the Foundations of Software Testing

Fall '23 MSSE Seminar #4b by Mats Heimdahl


For decades, researchers have explored the topic of software testing. Early work in this area covered the formal foundations of program testing. By exploring the foundations of testing largely separate from any specific method of testing, these researchers provided a general discussion of the testing process, including the goals, the underlying problems, and the limitations of testing. Unfortunately, a common, rigorous foundation has not been widely adopted in empirical software testing research, making it difficult to generalize and compare empirical research. Furthermore, much of current testing research has lost track of the goal of testing (finding consequential faults) and has focused on furthering specific aspects of the testing problem without considering the impact on the testing process as a whole.

In this talk I will revisit this foundational work and discuss the crucial (and often overlooked) interrelationship between the program under test, the test data used in its testing, and the test oracle determining whether or not a test passes.  I will, with examples from our empirical work, discuss new coverage criteria (Observable Coverage) attempting to leverage the interrelationship between programs, test data, and test oracles.  


Mats Heimdahl is the Department Head and a Professor of Computer Science and Engineering at the University of Minnesota. He earned an M.S. in Computer Science and Engineering from the Royal Institute of Technology (KTH) in Stockholm, Sweden and a Ph.D. in Information and Computer Science from the University of California at Irvine.

His research interests are in software engineering, safety critical systems, software safety, testing, requirements engineering, formal specification languages, and automated analysis of specifications. 

Start date
Saturday, Dec. 2, 2023, 2 p.m.
End date
Saturday, Dec. 2, 2023, 3:30 p.m.

Keller Hall 3-180, 200 Union Street SE, Minneapolis, MN 55455 

RSVP  is appreciated!