Robotics Colloquium: Speaker Naveen Kuppuswamy

Talk title: Robust contact-rich manipulation using tactile diffusion policies

Abstract: Achieving robust manipulation in unstructured real-world environments like homes is a hard open challenge. While, a diverse array of manipulation skills may be required, contact-rich/forceful manipulation tasks have proven to be particularly difficult to execute. In this context, camera-based tactile sensors have shown great promise in enhancing a robot’s ability to perceive touch; however, finding tractable high-fidelity analytical / data-driven models has proven challenging.  In this talk, I will detail how a recently achieved breakthrough in visuomotor policy learning using generative AI techniques - the diffusion policy - might be leveraged towards overcoming these challenges. Our approach directly incorporates haptic feedback into a diffusion policy by simply conditioning on tactile signals as an additional input. This tactile-diffusion policy can be trained on arbitrary tasks by utilizing human/expert demonstration and directly incorporates raw images from both traditional vision sensors and camera-based tactile sensor fingers - the TRI Soft-bubble Punyo sensor as an example. We use this framework to realize a wide array of challenging real-world kitchen manipulation tasks using a Franka Panda robot; highlights include constrained manipulation of visually diverse and challenging objects (wine glasses, dishes, bottle caps), handling deformable objects (dough, paper), and forceful tool manipulation (spatula). Our results indicate that tactile-diffusion policies outperform vision-only diffusion policies in both robustness and generalization abilities by significant magnitudes. I will conclude with a discussion on the implications of this approach towards building a more general-purpose foundation for robot manipulation - the TRI large behavior model effort.

Brief Bio:

Naveen Kuppuswamy is a Senior Research Scientist and Tactile Perception and Control Lead at the Toyota Research Institute, Cambridge, MA, USA. He holds a Bachelor of Engineering from Anna University, Chennai, India, an MS in Electrical Engineering from the Korea Advanced Institute for Science and Technology (KAIST), Daejeon, South Korea, and a Ph.D. in Artificial Intelligence from the University of Zurich, Switzerland. Naveen has several years of academic and industry robotics research experience and has authored several publications in leading peer-reviewed journals and conferences on themes of manipulation, tactile sensing, and robot learning & control. His research has been recognized through multiple publications and grant awards. He is also keenly interested in the STEM education of under-represented communities worldwide. Naveen is deeply passionate about using robots to assist and improve the quality of life of those in need.

Bio: Roboticist – tactile perception and control lead – senior research scientist