Virtual mouse driving apparatus and method using two-handed gestures

• US 20060209021A1
• Filed: 10/17/2005
• Published: 09/21/2006
• Est. Priority Date: 03/19/2005
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kwak, Ji Young, Kim, Heung Nam, Sul, Dong Myung, Yoo, Jang Hee, Lee, Kyung Hee
• Granted Patent: US 7,849,421 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A virtual mouse driving apparatus and method for processing a variety of gesture commands as equivalent mouse commands based on two-handed gesture information obtained by a video camera are provided. The virtual mouse driving method includes: keeping track of an input gesture input with the video camera; removing a background portion of the input gesture image and extracting a left-hand region and a right-hand region from the input gesture image whose background portion has been removed; recognizing a left-hand gesture and a right-hand gesture from the extracted left-hand region and the extracted right-hand region, respectively, and recognizing gesture commands corresponding to the recognition results; and executing the recognized gesture commands.

Claims

1. A virtual mouse driving method using a video camera comprising:

keeping track of an input gesture input with the video camera;

removing a background portion of the input gesture image and extracting a left-hand region and a right-hand region from the input gesture image whose background portion has been removed;

recognizing a left-hand gesture and a right-hand gesture from the extracted left-hand region and the extracted right-hand region, respectively, and recognizing gesture commands corresponding to the recognition results; and

executing the recognized gesture commands.

2. The virtual mouse driving method of claim 1, wherein the recognizing of the gesture commands comprises:

detecting a first-hand gesture, displaying a menu on a screen in response to the detection of the first-hand gesture, and recognizing one of a plurality of command icons of the displayed menu currently pointed at by or selected by a cursor according to the detected first-hand gesture; and

detecting a second-hand gesture and recognizing an object currently pointed at or selected by the cursor according to the detected second-hand gesture.

3. The virtual mouse driving method of claim 1, wherein the recognizing of the gesture commands comprises:

displaying a menu on a screen if a first-hand gesture is detected;

highlighting one of a plurality of command icons of the displayed menu pointed at by a cursor according to the detected first-hand gesture if the detected first-hand gesture is determined to be a pointing gesture; and

detecting the location of the cursor and executing a command corresponding to the highlighted command icon if another first-hand gesture is detected and is determined to be a clicking gesture.

4. The virtual mouse driving method of claim 3 further comprising making the menu disappear if no further hand gestures are detected.

5. The virtual mouse driving method of claim 1, wherein the recognizing of the gesture commands comprises:

if a second-hand gesture is detected, determining whether the second-hand gesture is a pointing gesture or a selection gesture;

varying the location of a cursor by keeping track of a location designated by the detected second-hand gesture if the detected second-hand gesture is determined to be a pointing gesture; and

selecting an object currently pointed at by the cursor and varying the location of the object by keeping track of the location designated by the detected second-hand gesture if the detected second-hand gesture is determined to be a selection gesture.

6. The virtual mouse driving method of claim 5, wherein the location designated by the detected second-hand gesture is within a displayed menu.

7. The virtual mouse driving method of claim 5, wherein the varying of the location of the cursor comprises calculating a displacement and trajectory between the detected second-hand gesture and a previously detected second-hand gesture and varying the location of the cursor based on the calculated displacement and trajectory.

8. A virtual mouse driving apparatus comprising:

a camera control unit which keeps track of a gesture with a video camera;

a gesture region extraction unit which removes a background portion of the input gesture image and extracts a left-hand region and a right-hand region from the input gesture image whose background portion has been removed;

a gesture recognition unit which senses left-hand gestures and right-hand gestures based on contours and features of the left and right hands extracted from the extracted left-hand and right-hand regions by the gesture region extraction unit and recognizes gesture commands represented by the sensed left-hand and right-hand gestures; and

a command processing unit which processes the gesture commands recognized by the gesture recognition unit as equivalent mouse commands.

9. The virtual mouse driving apparatus of claim 8, wherein the gesture recognition unit comprises:

a first-hand gesture processor which displays a menu on a screen if a first-hand gesture is detected, highlights one of a plurality of command icons of the displayed menu pointed at by a cursor if the detected first-hand gesture is determined to be a pointing gesture, and selects and executes a command corresponding to the highlighted command icon if another first-hand gesture is further detected and is determined to be a clicking gesture; and

a second-hand gesture processor which, if a second-hand gesture is detected, determines whether the second-hand gesture is the pointing gesture or a selection gesture, varies the location of the cursor by keeping track of a location designated by the detected second-hand gesture if the detected second-hand gesture is determined to be a pointing gesture, and selecting an object currently pointed at by the cursor and varying the location of the object if the detected second-hand gesture is determined to be a selection gesture.