This work presents a non intrusive hand tracking system which is capable to reconstruct 3D information from a monocular video sequence. The 3D position, orientation and posture of the hand are first reconstructed. From these data, a simple 3D mouse can be developed, using as 3D marker a reference point, the forefinger tip or the palm center when the hand is closed, and as commands a set of simple hand postures.
The developed system works in real time. The approach combines several computer vision algorithms in order to exploit their strengths and to minimize their weakness. The hand silhouette is first extracted from the incoming image. Then a 3D model of the hand is fitted to the segmented image in order to reconstruct the hand posture and the position of the marker. The contribution of our work is a simple and computationally fast system, which is very robust against noise and cluttered backgrounds, and has also the advantage of being computationally manageable on medium level computers.
The posture can assume three discrete values: 0, hand closed 1, hand closed with the forefinger extended, and 2, hand closed with thumb and forefinger extended. The tracker has been used to control a simple 3D environment. Pose 1 is used to move the marker position, while pose 2 mimics the click of the mouse. Dragging along X and Y axis of the image plane, that is moving the hand with pose 2 along X and Y, is used to rotate the 3D representation with a trackball paradigm. Moving the hand along the Z axis with pose 0 is used to zoom in and out into the 3D world. Pose 0 is used instead of pose 2 in order to avoid unexpected zooming effect when rotating the 3D world.
The system has been tested on a computer with a CPU Intel DualCore E6600, 1GB Ram, which can be considered as a medium cost processing unit (the complete system, including capture board and camera has a cost lower than 1000€). Processing the image stream at a frame rate of 25 frames/sec, the mean latency is 20ms, and one single CPU is used at 30% of its capacity. This demonstrates that there are resources available for other tasks, and that the system can be effectively used as an input device for other applications.
The accuracy is 2.80 mm RMS. The jitter is 0.92 mm RMS. These results show that the tracker provides a sufficient precision for many applications.
A. Bottino, M. De Simone
A fast and robust hand-driven 3D mouse
Proceedings of the International Conference of Computer Vision Theory
and Applications, VISAPP 2009, Lisboa, Portugal, 5-8 February 2009
A. Bottino, A. Laurentini
How to Make a Simple and Robust 3D Hand Tracking Device Using a
Single Camera
Proceeding of CSCC 2007. Agios Nikolaos, Greece. July 23-28, 2007
Copyright © 2007 - All Rights Reserved
CG&VC
@ Politecnico
di Torino