Zhi-Li Zhang Earns $4.25 Million NSF Grant for Autonomous Vehicle Research
Researchers and transportation companies around the world have been developing self-driving cars for a number of years. AVs use artificial intelligence (AI) and machine learning to identify common environments and situations in order to make driving decisions. Despite the advances over the years, there are still situations that AVs do not know how to handle. Zhang’s project leverages the advances in computing systems, like fifth-generation (5G) networks, to support “tele-operations” with a human operating AVs remotely when a complex situation arises.
“When more complex problem solving is required, the AV would need to send a large amount of rapidly-changing data over the wireless network for the tele-operator to step in,” said Zhang. “Our work will explore ways to make that data transfer more reliable, secure and safe. We are also working on improving the generative AI, so it can better predict when these complex situations may arise - like during cold weather or rain.”
AVs rely on the “vision” of their sensors in order to make driving decisions, making certain weather events problematic for the technology. With the majority of AV trials and research being conducted in California and the South where weather is more consistent, this project will aim to improve AV vision with the help of Midwest weather patterns.
Building off his work on shared autonomous vehicles (SAVs), Zhang hopes that augmenting AI with human intelligence is the missing piece that will make AVs safer and more reliable. These tele-operators won’t be controlling the car at all times, but they need to be ready to act quickly when the situation arises. One of the key challenges with this project is building systems that can keep tele-operators engaged at the right times, avoiding what Zhang calls “out of the loop syndrome”.
“This is interdisciplinary work,” said Zhang. “We need to understand the human side of things to make sure that the AVs and alert systems can communicate effectively with tele-operators. Our AI can be even more effective if we incorporate human intelligence in decision making.
“Another key aspect is cyber security. These systems need to be extremely secure to ensure that only designated individuals can remotely operate vehicles. There are a number of systems within the AVs and traffic networks that are constantly sharing information, so security is a complex issue.”
CS&E Associate Professors Kangjie Lu and Catherine Zhao will lead work on network security and computer vision, respectively. CS&E Lecturer and Research Assistant Eman Ramadan, as well as Professors Rajesh Rajamani and Nicole Morris from Mechanical Engineering, are also key contributors.
“The ultimate goal is to build a platform that can facilitate tele-operated AVs in any scenario,” said Zhang. “The technology would be open source so anyone could use it and adapt it. In the future, we hope AVs can be used to address societal problems and improve the accessibility of transportation.”
This interdisciplinary project brings together researchers from the University of Minnesota’s Department of Mechanical Engineering and the Center for Transportation Studies, as well as the Minnesota Department of Transportation (MnDoT). Zhang will work closely with collaborators at the University of Michigan, including Professors Morley Mao and Henry Liu, as well a number of industry partners, such as Ericsson, InterDigital, Lear Corporation, ATT Labs, Microsoft, and Nokia.