APPARATUS FOR CONTROLLING VIRTUAL MOUSE BASED ON HAND MOTION AND METHOD THEREOF

• US 20150015490A1
• Filed: 07/07/2014
• Published: 01/15/2015
• Est. Priority Date: 07/15/2013
• Status: Active Grant

• Current Assignee: Korea Electronics Technology Institute (Government of South Korea)
• Original Assignee: Korea Electronics Technology Institute (Government of South Korea)
• Inventors: AHN, Yang Keun, JUNG, Kwang Mo
• Granted Patent: US 9,430,039 B2
• Time in Patent Office: Days
• US Class Current: 345/166
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Provided are an apparatus and method for controlling a virtual mouse based on a hand motion. The apparatus includes an image processing unit configured to measure a maximum horizontal width of a hand region of a user and a width of an index finger in any one of a first depth image captured in a state in which the index finger is spread out in a vertical direction with his or her fist and a second depth image captured in a state in which the index finger is folded with the fist, a hand motion recognizing unit configured to compare the maximum horizontal width and the width of the index finger to recognize whether the index finger is folded, and a function matching unit configured to match a change in a folded state of the index finger to a pre-set first function and output a control signal for the function.

Claims

1. An apparatus for controlling a virtual mouse based on a hand motion, the apparatus comprising:

an image processing unit configured to measure a maximum horizontal width of a hand region of a user and a width of an index finger in any one of a first depth image captured in a state in which the index finger is spread out in a vertical direction with his or her fist and a second depth image captured in a state in which the index finger is folded with the fist;

a hand motion recognizing unit configured to compare the maximum horizontal width and the width of the index finger to recognize whether the index finger is folded; and

a function matching unit configured to match a change in a folded state of the index finger to a pre-set first function and output a control signal for implementing the matched function.

2. The apparatus of claim 1, wherein the image processing unit detects a hand region of the user using foreground/background separation from the depth image, generates a horizontal histogram from the detected hand region image of the user, and measures a maximum horizontal width at a point having a maximum pixel number.

3. The apparatus of claim 1, wherein the image processing unit detects a hand region of the user using foreground/background separation from the depth image, generates a horizontal histogram from the detected hand region of the user, determines a region of interest by a predefined length specified from an upper end of the horizontal histogram, and determines an average width of the hand region within the region of interest, as a width of the index finger.

4. The apparatus of claim 1, wherein the hand motion recognizing unit compares the width of the index finger with a width corresponding to a predefined percentage of the maximum horizontal width, and when the width of the index finger is less than the width corresponding to the predefined percentage, the hand motion recognizing unit recognizes the index finger in a spread state, and when the width of the index finger is equal to or greater than the width corresponding to the predefined percentage, the hand motion recognizing unit recognizes the index finger in a folded state.

5. The apparatus of claim 1, wherein the image processing unit generates a distance-transformed image in units of pixels with respect to the image of the hand region of the user, and detects a pixel having the greatest value in the distant-transformed image, as a marker.

6. The apparatus of claim 5, wherein the hand motion recognizing unit recognizes a change in a position of the user hand using the marker, and the function matching unit matches the change in a position of the user hand to a pre-set second function and outputs a control signal for implementing the matched function.

7. The apparatus of claim 1, wherein the image processing unit comprises:

a foreground/background separating unit configured to separate a foreground and a background on the basis of the depth information from the depth image;

a horizontal hand width measuring unit configured to measure a maximum horizontal width of the hand region image of the user from which the background has been separated;

a horizontal index finger width measuring unit configured to measure a horizontal index finger width in the hand region image of the user; and

a marker detecting unit configured to detect a marker to be used to recognize a change in a position of the hand from the hand region image of the user.

8. A method for controlling a virtual mouse based on a hand motion, the method comprising:

measuring a maximum horizontal width of a hand region of a user and a width of an index finger in any one of a first depth image captured in a state in which the index finger is spread out in a vertical direction with his or her fist and a second depth image captured in a state in which the index finger is folded with the fist;

comparing the maximum horizontal width and the width of the index finger to recognize whether the index finger is folded; and

matching a change in a folded state of the index finger to a pre-set first function and outputting a control signal for implementing the matched function.

9. The method of claim 8, wherein the measuring of a maximum horizontal width and a width of the index finger comprises:

detecting a hand region of the user using foreground/background separation from the depth image, generating a horizontal histogram from the detected hand region image of the user, and measuring a maximum horizontal width at a point having a maximum pixel number; and

determining a region of interest by a predefined length specified from an upper end of the horizontal histogram, and determining an average width of the hand region within the region of interest, as a width of the index finger.

10. The method of claim 9, wherein the measuring of a maximum horizontal width and a width of the index finger further comprises generating a distance-transformed image in units of pixels with respect to the image of the hand region of the user, and detecting a pixel having the greatest value in the distant-transformed image, as a marker.

11. The method of claim 8, wherein the recognizing of whether the index finger is folded comprises:

comparing the width of the index finger with a width corresponding to a predefined percentage of the maximum horizontal width; and

when the width of the index finger is less than the width corresponding to the predefined percentage, recognizing the index finger in a spread state, and when the width of the index finger is equal to or greater than the width corresponding to the predefined percentage, recognizing the index finger in a folded state.