Past Events

Is Data All You Need? Large Robot Action Models and Good Old Fashioned Engineering

Title: Is Data All You Need?: Large Robot Action Models and Good Old Fashioned Engineering

Bio: Ken Goldberg is William S. Floyd, Distinguished Chair of Engineering at UC Berkeley and Chief Scientist of Ambi Robotics and Jacobi Robotics. Ken leads research in robotics and automation: grasping, manipulation, and learning for applications in warehouses, industry, homes, agriculture, and robot-assisted surgery. He is a Professor of IEOR with appointments in EECS and Art Practice.  Ken is Chair of the Berkeley AI Research (BAIR) Steering Committee (60 faculty) and is co-founder and Editor-in-Chief emeritus of the IEEE Transactions on Automation.

Science and Engineering (T-ASE).  He has published ten US patents and over 400 refereed papers and presented over 600 invited lectures to academic and corporate audiences. http://goldberg.berkeley.edu

Abstract: 

bio

Enthusiasm for humanoids has been skyrocketing based on recent advances in "end-to-end" large robot action models. Initial results are promising, and several collaborative efforts are underway to collect the needed demonstration data. But is data really all you need?

Although end-to-end Large Vision, Language, Action (VLA) Models have potential to generalize and reliably solve all problems in robotics, initial results have been mixed1.  It seems likely that the size of the VLA state space and dearth of available demonstration data, combined with challenges in getting models to generalize beyond the training distribution and the inherent challenges in interpreting and debugging large models, will make it difficult for pure end-to-end systems to provide the kind of robot performance that investors expect in the near future.

In this presentation, I share my concerns about current trends in robotics, including task definition, data collection, and experimental evaluation.  I propose that to reach expected performance levels, we will need "Good Old Fashioned Engineering (GOFE)" – modularity, algorithms, and metrics.   I'll present MANIP2, a modular systems architecture that can integrate learning with well-established procedural algorithmic primitives such as Inverse Kinematics, Kalman Filters, RANSAC outlier rejection, PID modules, etc. I’ll show how we are using MANIP to improve performance on robot manipulation tasks such as grasping, cable untangling, surgical suturing, motion planning, and bagging, and propose open directions for research.

References: 

[1] Nishanth J. Kumar.  Will Scaling Solve Robotics? The idea of solving the biggest robotics challenges by training large models is sparking debate. IEEE Spectrum. 28 May 2024.

[2] MANIP: A Modular Architecture for iNtegrating Iteractive Perception into Long-Horizon Robot Manipulation Systems.  Justin Yu*, Tara Sadjadpour*, Abby O’Neill, Mehdi Khfifi, Lawrence Yunliang Chen, Richard Cheng, Ashwin Balakrishna, Thomas Kollar, Ken Goldberg.  IEEE/RSJ International Conference on Robots and Systems (IROS), Abhu Dhabi, UAE.  Oct 2024. Paper 

Guest Speaker: Konstantinos Polyzos

Title: Active selection of informative images for efficient Gaussian splatting via black-box optimization
 

Abstract: 3D scene rendering is a fundamental computer vision task with diverse applications, including autonomous driving, robotics, and medical imaging, just to name a few. Gaussian splatting (GS) and its extensions and variants provide outstanding performance in fast 3D scene rendering while meeting reduced storage demands and computational efficiency. While the selection of 2D images capturing the scene of interest is crucial for the proper initialization and training of GS, hence markedly affecting the rendering performance, prior works rely on passive image selection, often leading to redundancy and high computational costs in dense-view settings or insufficient scene coverage in sparse-view scenarios. In the first part of the talk, we will focus on adaptive Bayesian optimization to efficiently optimize black-box and expensive-to-evaluate functions by judiciously adapting to the proper surrogate model as new input-output data are acquired online. Next, we will introduce a novel black-box optimization framework, namely `ActiveInitSplat', that actively selects training images for proper initialization and training of GS. ActiveInitSplat relies on density and occupancy criteria of the resultant 3D scene representation from the selected 2D images, to ensure that the latter are captured from diverse viewpoints, leading to better scene coverage and that the initialized Gaussian functions are well aligned with the actual 3D structure. We will conclude with numerical tests on real-world 3D scenes that showcase the merits of ActiveInitSplat compared to passive GS counterparts in both dense- and sparse-view settings.

 
Short bio: Konstantinos D. Polyzos is a Postdoctoral Fellow at Eric and Wendy Schmidt AI in Science, University of California, San Diego, working with Prof. Tara Javidi. He obtained his Ph.D. degree at the Department of Electrical and Computer Engineering of the University of Minnesota, under the supervision of Prof. Georgios B. Giannakis. Throughout his Ph.D. studies, he has been working on learning, inferring, and optimizing with just a few labeled data. Specifically, he has been developing and leveraging active- , transfer- , and self-supervised learning and Bayesian optimization methods to learn and/or optimize when only a few input-output data are available due to privacy concerns or high sampling costs, with application to healthcare, 5G networks, and robotics. He received the UMN ECE Department Fellowship in 2019, Gerondelis Foundation Scholarship in 2020, Onassis Foundation Scholarship in 2021, 'Eric and Wendy Schmidt AI in Science' Postdoctoral Fellowship, the Best Paper Award at the International CIT&DS 2019 International Conference in 2019, the Best Student Paper Award at the International IEEE SAM 2024 Workshop in 2024, the Best Paper Award (second place) at the International IEEE MLSP Workshop in 2024, and the Outstanding Reviewer Award (top 10 %) at the International Conference on Machine Learning (ICML 2022).

Guest Speaker -Sushmita Mitra

Title: Artificial Intelligence in Healthcare

Abstract:
With the inherent boom in the availability of large volumes of multimodal healthcare data over the Internet, their automated processing is becoming all the more relevant in today’s perspective. Manual delineation and processing are expensive, biased, and slow. Here lies the utility of artificial intelligence, encompassing machine learning and deep learning. The role of AI is to provide assistive intelligence to healthcare professionals in their endeavor to arrive at cost-effective, fast, and viable solutions for complex decision-making.

This talk outlines the role of AI in several aspects of healthcare, including classification, segmentation, and survival prediction. We discuss applications of multimodal imagery, including X-ray, CT, MR, and fundus images, to handle some diseases. Finally, a deformable deep net is introduced for efficient segmentation. 

 

Brief Bio: 

Sushmita Mitra is a full professor at the Machine Intelligence Unit (MIU) at the Indian Statistical Institute in Kolkata. From 1992 to 1994, she was a DAAD Fellow at RWTH Aachen University in Germany. She has served as a Visiting Professor in the Computer Science Departments at the University of Alberta in Edmonton, Canada; Meiji University in Japan; and Aalborg University in Esbjerg, Denmark. Dr. Mitra received the National Talent Search Scholarship from NCERT, India, from 1978 to 1983, the University Gold Medal in 1988, and the IEEE TNN Outstanding Paper Award in 1994 for her pioneering work in neuro-fuzzy computing. She was awarded the CIMPA-INRIA-UNESCO Fellowship in 1996 and the Fulbright-Nehru Senior Research Fellowship from 2018 to 2020. Dr. Mitra held the position of INAE Chair Professor from 2018 to 2020. In 2021, she received the prestigious J. C. Bose National Fellowship. Dr. Mitra is a Fellow of the IEEE, The World Academy of Sciences (TWAS), the Indian National Science Academy (INSA), the International Association for Pattern Recognition (IAPR), the Asia-Pacific Artificial Intelligence Association (AAIA), as well as a Fellow of the Indian Academy of Sciences (IASc), the Indian National Academy of Engineering (INAE), and the National Academy of Sciences, India (NASI). Her current research interests include data science, machine learning, soft computing, medical image processing, and bioinformatics.

Ethics in Writing Research Manuscripts

Discuss the ethical implications of writing research manuscripts that use generative artificial intelligence and other supportive tools. 

MnRI Master In Robotics Info. Session

Are you ready to take your passion for robotics to the next level? We invite you to join us for an exclusive information session about our Master in Robotics program!

Event Details:

  • Date: January 30th, 2025
  • Time: 8:30 AM – 9:30 AM (Central Time)
  • Location: Online via Zoom 

This session will provide you with an overview of our cutting-edge program, admissions process, curriculum, and opportunities for research and hands-on learning in the exciting field of robotics. You’ll also have the chance to ask questions and hear directly from faculty and current students.

Whether you're considering applying for the upcoming academic year or simply exploring your options, this is an excellent opportunity to learn more and connect with our admissions team.

We look forward to meeting you and sharing more about how our Master in Robotics program can help you build the future of technology!

Robotics Colloquium: Guest Speaker - Mark Wehde

Title: The Evolving Role of Robotics in Healthcare: Innovation, Efficiency, and Future Challenges

Abstract: In this presentation, we’ll explore the diverse and evolving applications of robotics in the healthcare sector, focusing on current implementations and future possibilities. We’ll start with a brief look at surgical robotics, exploring their role in enhancing precision in complex procedures despite the high level of complexity and resource requirements.

We’ll then dive into a range of accessible assistive robotics, such as devices that aid phlebotomists and facilitate ultrasound procedures, as well as clinical robots like pharmacy robots and automated transport systems.

We’ll also explore how robotics contribute to operational efficiency in healthcare settings, tackling essential tasks such as cleaning and sterilization, environmental services, and the movement of samples across organizations. Additionally, robots that deliver food, linens, and other supplies can help address staffing challenges and streamline routine processes.

Lastly, we’ll consider the potential for remote-controlled robotics in surgical and other clinical procedures, addressing both the exciting possibilities and the associated challenges.

Through these discussions, we aim to provide a comprehensive view of how robotics can enhance patient care, streamline healthcare operations, and pave the way for future innovations.

Short Bio: Mark Wehde is chair of Mayo Clinic Engineering, assistant professor of Biomedical Engineering in the Mayo Clinic College of Medicine and Science, and fellow in the Mayo Clinic Academy of Educational Excellence. He is also the James J. Renier Chair in Medical Device Innovation at the University of Minnesota Technology Leadership Institute. Mark is also a senior member of the IEEE.

 Mark received a Master of Science degree in Biomedical Engineering from Iowa State University, a Bachelor of Science degree in Electrical Engineering from South Dakota State University, and a Master of Business Administration through the University of Wisconsin MBA Consortium. 

MnRI Master In Robotics Info. Session

Are you ready to take your passion for robotics to the next level? We invite you to join us for an exclusive information session about our Master in Robotics program!

Event Details:

  • Date: December 5, 2024
  • Time: 8:30 AM – 9:30 AM (Central Time)
  • Location: Online via Zoom 

This session will provide you with an overview of our cutting-edge program, admissions process, curriculum, and opportunities for research and hands-on learning in the exciting field of robotics. You’ll also have the chance to ask questions and hear directly from faculty and current students.

Whether you're considering applying for the upcoming academic year or simply exploring your options, this is an excellent opportunity to learn more and connect with our admissions team.

We look forward to meeting you and sharing more about how our Master in Robotics program can help you build the future of technology!

Robotics Colloquium: Guest Speaker - Brad Holschuh

Title: Soft Robotics Using Shape Memory Materials for Wearable Technology Applications
 
Abstract: In this talk, I will discuss recent advancements, opportunities, and challenges associated with wearable soft robotic systems. In particular, we will focus on using shape memory materials as soft robotic actuators for on-body robotic applications, and discuss actuator strategies, garment integration architectures, and potential use cases, featuring ongoing work in the UMN Wearable Technology Laboratory (WTL).
 
Biography: Dr. Brad Holschuh is an Associate Professor of User Experience (UX) Design at the University of Minnesota, where he serves as Co-Director of the Wearable Technology Laboratory (WTL) and Director of Graduate Studies for the Human Factors and Ergonomics (HFE) graduate program and the Design Graduate Program. Since 2022, Dr. Holschuh has also held the title of UMN Distinguished University Teaching Professor and is a member of the UMN Academy of Distinguished Teachers. He earned his BS ('07), dual MS ('10), and PhD ('14) degrees from the Massachusetts Institute of Technology, where as a NASA Space Technology Research Fellow (NSTRF) in the Man-Vehicle Laboratory (MVL) he researched advanced materials for next generation space suits. At UMN, Dr. Holschuh’s research focuses on the use of wearable technology to improve human performance both in space and on Earth, with a specific focus on integrating active materials technology into wearable systems. His work encompasses wearable technology, soft robotics, human factors design, textile engineering, aerospace engineering/bioastronautics, and materials science. 
 
 

 

Machine Learning & Robotics at Milwaukee Tool-Tech Talk

Join us for an engaging session where we’ll dive into our exciting work in ML and Robotics. Food provided!

Milwaukee Tool (milwaukeetool.com) has been the industry leader for power tools. Now, we are making big moves to heavily invest in AI, Robotics, and Machine Learning. Join us, and you will shape the future of the industry.

 

Areas that our team is hiring for:

 

Hand Shake Event:

Robotics Colloquium: Guest Speaker - Andrew W. Grande

Title: Robotics for the treatment of stroke
 
Abstract: I will discuss the parallel evolution of stroke treatment and robotics in neurosurgery. I will discuss some past and current attempts to implement surgical robotics and some challenges with their adoption. I will then move on to a unique opportunity today where the needs of stroke treatment and current robotic technologies are an ideal marriage for robotic stroke treatment.   
 
Bio: Andrew Grande is a professor of neurosurgery specializing in vascular neurosurgery and stroke treatment.  He is the director of the Earl Grande Stroke and Stem Cell Lab and director of the Zarling Neuroanatomy, Neuroinnovation lab.  Over the last 14 years, he has had a passion for technology and has been very interested in developing robotics for treating neurosurgical diseases.  He started the Neurorobotics Research Consortium with Tim Kowalewski at the University of Minnesota.