CS Events Monthly View
Qualifying ExamSelf-Supervised Object Understanding for Robot Perception and Manipulation |
|
||
Monday, November 28, 2022, 10:30am - 12:00pm |
|||
Speaker: Shiyang Lu
Location : 1 Spring Street, Room 204
Committee:
Prof. Kostas Bekris (advisor)
Prof. Abdeslam Boularias
Prof. Kristin Dana
Prof. Sepehr Assadi
Event Type: Qualifying Exam
Abstract: : To be deployed in indoor human environments, autonomous robots should be able to reason about unknown objects so as to manipulate them safely and efficiently. It is, however, challenging because household objects can be arbitrary with various textures and geometries. And it is not feasible to manually annotate or train a model for every single object or even at the category level. We aim to let robots understand objects from their own experiences, either through passively observing objects from different viewpoints and locations, or actively manipulating them. In particular, we designed algorithms to automatically find patterns of rigid objects across different scenes even in clutter, and use the pseudo labels to train a segmentation model via contrastive learning. We also built a robotic system to manipulate objects without access to mesh models. The target object is tracked and reconstructed during the manipulation process. Its texture information and reconstructed model are saved in a memory bank to speed up future manipulation of the same object. Through a lifelong process, the robot becomes more capable over time as it observes and manipulates more objects. In the future, we aim to extend the current research to allow robots to robustly handle objects in more complex scenes.
: