Robotic Mitigation of Aquatic Invasive Species

Aquatic invasive species (AIS) are non-native plants, animals, or other organisms that have been introduced into a new aquatic habitat where they cause harm or have the potential to cause harm. These intruders can be found in any type of water body, from freshwater lakes and rivers to marine ecosystems like oceans and estuaries. The ill effects of AIS are wide-ranging and can devastate the environment and human well-being. For example, the zebra mussel has caused billions of dollars in damage to infrastructure in North America, clogging pipes and disrupting water supplies, some types of invasive snails can host the parasite that causes swimmer's itch, and the Asian carp is displacing native fish species in the Great Lakes, impacting the fishing industry and ecosystem. Specifically, invasive aquatic vegetation often grows rapidly and efficiently, outcompeting native plants for resources like sunlight, nutrients, and space. This can lead to the decline or even extinction of native species, disrupting the entire food chain and altering the delicate balance of the ecosystem. Current methods of AIS mitigation include mechanical removal to harvest or cut the vegetation out, or chemically or biologically treating with herbicides or biocontrol agents. However, if done indiscriminately, all of these methods have the potential of harming the native species and ecosystem considerably. Also, these are all labor-intensive, expensive, and often dangerous work, as human (i.e. scuba diver) intervention is necessary. Thus, it is of the highest importance that such species are detected, classified, and their locations mapped, for eventual safe removal. 

 A current National Science Foundation National Robotics Initiative project, with MnRI and Computer Science & Engineering associate professor Junaed Sattar as the principal investigator (PI) and CFANS Professor Raymond Newman as co-PI, is looking into the use of aquatic robots to address the aforementioned challenges in mitigating the ill-effects of AIS. This project has its roots in a MnRI Seed Grant (``Robotic localization and conservation of rare aquatic species'') awarded to Professor Sattar and Mark Hove, Research Biologist at CFANS, during the 2019-2020 academic year. Initial data collection and algorithmic prototyping was made possible by the MnRI seed grant, which eventually led to this larger NSF NRI grant being awarded. In the NRI project, Professors Sattar and Newman are looking at using multimodal data (visual and acoustic) to detect and classify different kinds of AIS, and doing so using deep machine learned methods in spite of having a fraction of data available compared to other, terrestrial applications. They are also looking at mapping the growth areas of AIS using a combination of bathymetric and acoustic localization information. These are hitherto unexplored, or at best under-explored, areas both in robotics and AIS research, and innovations have the potential to significantly impact both these fields in the near future. The PIs intend to integrate all their findings into an autonomous underwater vehicle (AUV), and demonstrate the feasibility of artificially-intelligent underwater robots for AIS management through field rials in diverse geographical locations.

The first year of the project has just been completed, and some important milestones have been achieved already. The Interactive Robotics and Vision Laboratory, led by Professor Sattar, has created a working prototype of a novel AUV called MeCO, which is an open-source, artificially intelligent, multi-sensor-driven AUV. The first water trial of MeCO was held in early December 2023, demonstrating its water-worthiness and maneuverability. Ongoing work continues to integrate visual sensors, human-interface devices, and autonomous capabilities in MeCO, preparing it for the next field trials to be held at the Bellairs Research Center of Barbados in February 2024. PI Sattar’s team also took part in the 2023 field trials at the same location, gathering valuable data, and testing initial work in visual detection algorithms in real-world conditions. 

PI Newman's team has worked on linking spatially explicit sonar data to underwater images to test the accuracy of sonar to estimate plant biovolume and distinguish plants from other structures and signals (corals, algae, turbulence). The team is also learning how to distinguish seagrass taxa and other features from photographic frames. The imagery will be useful to train PI Sattar’s deep-learned detection models for direct visual identification. The off-the-shelf sonar processing software has been producing many false positives for plant biovolume. PI Newman intends to evaluate open-source software for comparison, as these false positives must be eliminated or corrected.

In the current year and beyond, PI Sattar will continue to investigate detection-driven image enhancement for underwater invasive species detection, incorporating novel network architectures (e.g. VAE-GAN, transformer-based models, and lightweight large language models), and use multimodal (visual and acoustic) data to train and evaluate detector performance. He will work with co-PI Newman to obtain training data for these novel approaches. Sattar will also aim to develop and test a prototype of the acoustic localization system for underwater robots based on time difference of arrival calculations of sound at certain frequencies. This will help localize robots, and coupled with the detection of invasive species, will help in creating a map of their growth underwater.