CS Events

Qualifying Exam

Unsupervised Learning of Structured Representation of the World


Download as iCal file

Wednesday, December 08, 2021, 09:00am - 10:00am


Speaker: Gautam Singh

Location : Via Zoom


Hao Wang

Karl Stratos

Mario Szegedy (assigned by the department)

Sungjin Ahn (Advisor)

Event Type: Qualifying Exam

Abstract:  Intelligent agents would like to build representations of the world that are well-suited for planning actions and achieving goals in the physical world. Because the environments are diverse and always changing, providing supervision to train the representation learning systems makes them hard to scale and unviable. In this work, we explore unsupervised representation learning methods, based on variational auto-encoders, that do not simply return a monolithic compression of the input observation but rather provide a significantly richer representation in the following respects: (i) our encoder, taking only a few observations of a 3D scene as input, produces a representation that contains information about the full 3D scene. (ii) It organizes the representation as a set of object vectors and decomposes these vectors further into `what' and `where', thus opening the possibility for symbolic processing downstream. (iii) As the agent observes a dynamic environment, our representation method, using past observations and the learned knowledge of the dynamics, infers the state of invisible objects and unseen viewpoints of the scene. (iv) Lastly, our representation provides, not just a point estimate of the inferred environment state, but rather a belief state and thus embraces the randomness of the physical world. In achieving these characteristics, we develop novel models that advance the state of the art. To do this, we build on existing approaches for neural modeling of stochastic processes, recurrent state-space modeling, sequential Monte Carlo, and unsupervised object-centric representation learning.


Rutgers University School of Arts and Sciences

Contact  Sungjin Ahn

Join Zoom Meeting


Join by SIP

This email address is being protected from spambots. You need JavaScript enabled to view it.

Meeting ID: 951 9624 8971

Password: 1282021

One tap mobile

+16465588656,,95196248971# US (New York)

+13017158592,,95196248971# US (Washington DC)

Join By Phone

+1 646 558 8656 US (New York)

+1 301 715 8592 US (Washington DC)

+1 312 626 6799 US (Chicago)

+1 669 900 9128 US (San Jose)

+1 253 215 8782 US (Tacoma)

+1 346 248 7799 US (Houston)

Meeting ID: 951 9624 8971

Find your local number: https://rutgers.zoom.us/u/ad2kzEELV9

Join by Skype for Business