Qualifying Exam
Qualifying ExamUnsupervised Learning of Structured Representation of the World |
|
||
Wednesday, December 08, 2021, 09:00am - 10:00am |
|||
Speaker: Gautam Singh
Location : Via Zoom
Committee:
Hao Wang
Karl Stratos
Mario Szegedy (assigned by the department)
Sungjin Ahn (Advisor)
Event Type: Qualifying Exam
Abstract: Intelligent agents would like to build representations of the world that are well-suited for planning actions and achieving goals in the physical world. Because the environments are diverse and always changing, providing supervision to train the representation learning systems makes them hard to scale and unviable. In this work, we explore unsupervised representation learning methods, based on variational auto-encoders, that do not simply return a monolithic compression of the input observation but rather provide a significantly richer representation in the following respects: (i) our encoder, taking only a few observations of a 3D scene as input, produces a representation that contains information about the full 3D scene. (ii) It organizes the representation as a set of object vectors and decomposes these vectors further into `what' and `where', thus opening the possibility for symbolic processing downstream. (iii) As the agent observes a dynamic environment, our representation method, using past observations and the learned knowledge of the dynamics, infers the state of invisible objects and unseen viewpoints of the scene. (iv) Lastly, our representation provides, not just a point estimate of the inferred environment state, but rather a belief state and thus embraces the randomness of the physical world. In achieving these characteristics, we develop novel models that advance the state of the art. To do this, we build on existing approaches for neural modeling of stochastic processes, recurrent state-space modeling, sequential Monte Carlo, and unsupervised object-centric representation learning.
Organization:
Rutgers University School of Arts and Sciences
Contact Sungjin Ahn