Multi state deformable templates

Our approach to feature detection uses multi-state deformable templates for eyebrows, eyes and lips which are depicted in the following pictures:

Brow model

Lip model

Open and closed eye model

Tracking

Templates are fitted to the incoming images exploiting motion, shape and color information. The shape parameters of the templates are then output as face animation parameters. In the following images are also depicted the results of feature tracking.

Brows tracking results

Lip tracking results

Iris tracking results

Eyes tracking result

The facial features reconstruction process runs at 40 frame/s on a Pentium III 600 Mhz using as input images directly the camera frames (320*240 pixels). Using a 256×256 texture map the reconstruction runs at about 55 frame/s; in this case the total reconstruction time, which includes head pose reconstruction, texture warping and facial features reconstruction, discarding the time spent in reading and decoding the input stream, is about 18 frame/s.