Faculty Host: Vladimir Pavlovik

Abstract:

While stereo matching was originally formulated as the recovery of 3D shape from a pair of images, it is now generally accepted that using more than two images can dramatically improve the quality of the reconstruction. Unfortunately, as more images are added, the prevalence of semi-occluded regions (pixels visible in some but not all images) also increases. In addition, non-rigid effects in real scenes such as highlights, reflections, and translucency further complicate multiview stereo.

In this talk, I will describe how we progressively tackle the problems of occlusion, highlights, reflections, and translucency. To handle occlusion, we use a combination of shiftable windows and a dynamically selected subset of the neighboring images to do the matches. To handle highlights, we apply a color histogram differencing technique. Finally, to take into account reflections and translucency, we model the image formation as additive superposition of two layers at two different depths, and solve for them iteratively. I will show results for both synthetic and real image sequences as validation of these approaches.

Speaker Bio:

Sing Bing Kang received his Ph.D. in robotics from CMU in 1994. He is currently a researcher at Microsoft Corporation working on image-based modeling. His paper on the Complex Extended Gaussian Image had won the IEEE Computer Society Outstanding Paper award at CVPR'91. His IEEE Transactions on Robotics and Automation paper on human-to-robot hand mapping had been awarded the 1997 King-Sun Fu Memorial Best Transaction Paper award. Sing Bing has published about 20 refereed journal papers and about 45 refereed conference papers, and holds 12 US patents. He has also co-edited a book on panoramic vision, which was published by Springer in 2001.