Meaningful abstraction


Doug DeCarlo
Anthony Santella

Good information design relies upon strategies for reducing the perceptual and cognitive effort required to understand an image. Our goal is to create well-designed imagery automatically. This requires better understanding of human processing of visual information, and new representations for visual order and organization.

Our first step towards this goal is in the investigation of image transformations. Starting with a photograph, we reduce its content (with user control) and render it in a particular style. The following is an example:

Photograph Transformation using eye-movement input

In this example, you'll notice that meaningful visual elements are highlighted, and extraneous detail is omitted. In this case, a user interface that monitors the user's gaze provides the information necessary to judge the importance of the various elements in the photograph, when used with a model of human visual perception. The user need only look at the image for a few seconds; an eye-tracker monitors the user's gaze, and the program does the rest. A model of visual perception connects the recorded eye movements with the contents of the image.
Our eye-tracker setup

Without the eye-tracker, an automatic method either leaves in too much, or takes out too much:

        
Too much detail (automatic) Not enough detail (automatic)

Below are results from our approach in two styles: painterly renderings and line drawings.

I: Painterly renderings

Painterly rendering refers to the computerized process of transforming ordinary photographs into stylized variations that resemble paintings (in that they are built of from a set of strokes). In the following, the brush strokes are placed in agreement with the gaze input and perceptual model. The contrast and color of the stroke are also adjusted.

In this first transformation, a photograph and its accompanying abstracted painting is displayed. In the painting, the background subjects have much less detail, making the subject of the rendering obvious. The next two images show how existing automatic techniques work on this image: with a fixed brush on the entire image, no meaningful abstraction occurs.

This is followed by two more examples of abstract painterly renderings.


A photograph
An abstract painterly rendering

Automatic, with fixed-size fine strokes
Automatic, with fixed-size coarse strokes

A photograph
An abstract painterly rendering
(photo courtesy http://philip.greenspun.com)

A photograph
An abstract painterly rendering

II: Line drawings

Richer styles require more sophisticated representations, rendering algorithms, and perceptual models. We have also developed transformation methods for a line-drawing style that employs bold edges and large regions of constant color. Once again, abstraction is only achieved when the gaze input is used to transform the image.

In the first example below, we show an image along with its abstracted line-drawing. For comparison, we also include automatic renderings at a particular level of detail (which lack meaningful abstraction). This is followed by two more example transformations.


A photograph
An abstract line-drawing
(photo courtesy http://philip.greenspun.com)

Automatic, at a fine level of detail
Automatic, at a coarse level of detail

A photograph
An abstract line-drawing
(photo courtesy http://philip.greenspun.com)

A photograph
An abstract line-drawing
(photo courtesy http://philip.greenspun.com)


Publications

An image from this work was selected as the cover art for the SIGGRAPH 2002 proceedings:

            

It was also described in the Spotlight section of the February 2002 issue of Computer Graphics World as seen here.

Data

The eye-movement recordings, original images, and final results are available from the papers. Here they are separated by publication:




Links


Back to The VILLAGE