Schedule

Schedule: Tuesdays, 2:50 - 5:50 pm, CoRE 305 (CoRE B)     (the time is flexible, however)
Instructor: Douglas DeCarlo (decarlo@cs)

Calendar (within Rutgers only)

Description

In graphics and vision, models can describe a variety of features of a class of objects, such as their shape, motion, appearance, behavior, or formation. These classes can range from being quite general (such as smooth surfaces, or articulated-rigid objects) to specific (such as trees, human faces, MRI scans of brains, or buildings).

In all cases, however, the model provides an abstract means to fill in the gaps given a sparse specification by a user (in graphics) or given incomplete, ambiguous or noisy data (in vision). In addition, in computer vision, a properly applied model will make the application more robust to corrupted data.

The material to be discussed in this seminar, presented using a series of research papers, falls into two categories: foundations and applications. The first provides the background material on how models can be specified and used. The second reviews recent examples of model-based techniques, as they are applied in computer graphics (in the production of games, special effects, animations and simulations) and in computer vision (in shape estimation, object tracking and recognition applications).

Note that these topics will not be covered in the order they are mentioned here. When necessary, I will give short presentations on the background material needed to understand the current set of papers. While having taken courses in computer graphics and computer vision will help, they are not required. You should, however, have a reasonably good understanding of linear algebra and basic statistics. Expected work includes participation in class discussions, leading the discussion on at least one of the topics, and one small project (individual or group).

Model-based Foundations

• Generic surface models
  Models of surface continuity and smoothness, parameterized shapes
• Models learned from examples
  Principal components analysis, Neural networks, etc...
• Model-based estimation
  Once you have a model, how do you estimate its parameters from data?
• Modeling vs. Learning
  Bayesian statistical methods require specifying a prior; this can be done quite differently in the presence of a detailed model.
• Control and model complexity
  In generation, as models become very complex, there is typically a loss of control over the result (often due to chaotic behavior). There have been some attempts to address this problem.

Applications

• Automatic generation in computer graphics
  Given a short description (provided by a user or artist), we want to generate plausible examples of objects (such as with trees, landscapes, clouds, and faces) and motions (ranging from human motion to fluid motion).
• Motion capture data processing
  Physically attaching sensors (either magnetic or optical) to people, and recording the motion of the sensors, can provide a description suitable for animation. Models of human motion are often used for getting these measurements into a usable form, as well as altering them, so they can be adapted to a differently sized character.
• Shape estimation, object tracking and recognition
  I will focus more on the applications of understanding of the activities of people (for interaction, sign language translation, security, etc...) and on medical image analysis.
• Model selection and creation
  There are many situations where one is not sure of which model applies. In this case, you might need to choose a particular model from an existing database, or build a model from scratch (on the fly).