Emerging Technologies

Visualizing Light Transport Phenomena With a Primal-Dual Coding Video Camera

This project presents a primal-dual coding video camera, an optical device that captures video in which the flow of light through a scene has been manipulated. The camera has the ability to visualize indirect transport effects (caustics, inter-reflections, volumetric scattering, etc.), measure direct-only light paths (surface reflections), capture transport that occurs within specific regions of 3D space, and perform structured light imaging in the presence of complex indirect effects. The operating principle behind this technology is to simultaneously code the light that goes into a scene with the light that comes out. Specifically, over the course of a single camera-exposure period, the system projects a sequence of light patterns onto the scene in lockstep with a second sequence of mask patterns that modulates the light incident on the camera sensor. This optical procedure creates RAW, unprocessed photos in which a scene's light-transport function appears to have changed.

  Download Video

Matthew O'Toole
University of Toronto

Kiriakos N. Kutulakos
University of Toronto