U of M part of Toyota Research Institute’s new initiative

CS&E's Hyun Soo Park will lead artificial intelligence research to better understand drivers' behavior to increase safe driving

The University of Minnesota has been selected by the Toyota Research Institute (TRI) as one of 13 additional academic institutions that will participate in the next five-year phase of a collaborative research program. These universities join MIT, Stanford and the University of Michigan, which have worked with TRI over the last five years to expand the body of research into artificial intelligence (AI) with the goal of amplifying the human experience.

Computer Science & Engineering assistant professor Hyun Soo Park will lead the TRI-sponsored research at the University of Minnesota. Professor Park specializes in image processing and computer vision, and will bring this expertise to the collaborative. He and his colleague, Professor Jianbo Shi of the University of Pennsylvania, will be developing datasets and researching high-fidelity models for in-cabin behavior understanding. This work will focus on enabling the measurement, analysis, and prediction of the behaviors of drivers and passengers to increase safe driving.

“The behavioral state of drivers and passengers in cars, for example, sleepiness and social interactions, play a major role of safe driving,” said Professor Park.

Park’s team will create a data acquisition system for the collection of large-scale in-cabin video datasets, using a multiview camera system that can allow the capture of realistic on-site human behavior at multiple resolutions including:

  • close up facial expression, head pose and eye gaze,
  • hand and arm movements and gestures, and
  • upper body movements and interaction.

With this dataset, the researchers will develop robust and generalizable algorithms for the 4D space-time representation of behaviors.

TRI will be investing more than $75 million in the academic institutions over the next five years.

“Our first five-year program pushed the boundaries of exploratory research across multiple fields, generating 69 patent applications and nearly 650 papers,” said Eric Krotkov, TRI Chief Science Officer who leads the university research program. “Our next five years are about pushing even further and doing so with a broader, more diverse set of stakeholders. To get to the best ideas, collaboration is critical. Our aim is to build a pipeline of new ideas from different perspectives and underrepresented voices that share our vision of using AI for human amplification and societal good.” 

The full list of universities that will participate in the next phase of TRI’s collaborative research program include: 

  • Carnegie Mellon University
  • Columbia University
  • Florida A&M University-Florida State University College of Engineering
  • Georgia Institute of Technology (Georgia Tech)
  • Indiana University
  • Massachusetts Institute Technology (MIT)
  • Princeton University
  • Smith College
  • Stanford University
  • Toyota Technological Institute at Chicago (TTIC)
  • University of California, Berkeley
  • University of Illinois
  • University of Michigan
  • University of Minnesota
  • University of Pennsylvania
  • UCLA

Through this program, TRI will lead 35 joint research projects focused on achieving breakthroughs around difficult technological challenges in TRI’s research areas: Automated Driving, Robotics and Machine Assisted Cognition (MAC). 

Learn more about the Toyota Research Institute and the new phase of collaborative research.