Collaborative Research: NRI: INT: Dense 3D Reconstruction of Dynamic Actors in Natural Environments using Multiple Flying Cameras [research project]
Funding agency
National Science Foundation (October 1, 2020 - September 30, 2023)
Investigators
Hyun Soo Park (assistant professor), Volkan Isler (professor)
Abstract
Graph theory helps scientists and engineers model various types of relations between entities in a set, whether members of a social network, or molecules in a chemical compound for example. Not surprisingly, with the advent of data-based methodologies that work by unraveling and exploiting relations between data items, graph theory tools are finding their way in a very broad range of applications. The primary goal of this project is to examine a class of methods that manipulate graphs, specifically by developing effective multilevel algorithms that take advantage of divide and conquer approaches. In multilevel techniques, smaller and smaller graphs are extracted from some original graph with the goal of keeping as much of its intrinsic information as possible. These smaller graphs are then employed instead of the original ones, resulting in significant gains in performance. This project addresses issues that are of great relevance to many current data-based methodologies and will be applicable across various disciplines. As such it will help promote interest in problems related to the current shift toward such methodologies because its research theme blends mathematical methods, innovations in algorithms, and applications. On the educational side, special courses and tutorials will be offered to graduate students from other disciplinary fields who wish to explore research in data sciences. This project will support one graduate student per year for each of the three years.
While large-scale multi-camera domes have been developed for data collection in controlled laboratory settings it is not possible to achieve a similar level of measurement quality outdoors where there is much potential benefit to such data collection. For example, use of such measurements include the body dynamics of a running cheetah, or people, or analyzing herding behaviors of animals or birds. This leads to scientists relying on extremely inefficient and dangerous data collection methods. For example, biologists studying the behaviors of wild animals try to predict where the animals will be and place some cameras which only give some limited data at specific locations. This project addresses such challenges by exploring the research of methods and development of a large-scale data collection tool for high-resolution and multi-viewpoint visual recording and motion analysis of natural group behaviors (e.g., herds of animals or groups of people) in-the-wild over very large environments (e.g., desert plains or mountain sides) using a team of flying robots.
This project develops computational models that integrate the fundamentals of computer vision and multi-agent control to measure the group of actors in 3D. Through the development of this system, this project will make major advances in technology at the intersection of perception and control that include: (1) a new study of methods for precise, rapid, and robust target motion forecasting and relative state estimation that estimates the 3D motion of the robots and actors quickly with strong uncertainty estimates; (2) a new decomposition of the perception-aware multi-objective multi-UAV safe motion planning problem, that allows long-term planning based on consistent actor forecasting uncertainty models and coverage objectives; (3) a new guaranteed safe but adaptive paradigm for reactive flight control that is able to generate safety maneuvers even under large disturbances and vehicle dynamics changes, and that can leverage prior flight experience for real-time adaptation; (4) new theory of 3D reconstruction for dynamic scenes captured by UAVs that will enable high-resolution mesh and skeletal reconstruction of the groups of actors. The research outcome will be disseminated through multiple educational activities.
More information on agency website
Keywords
computer vision