Contactless gesture-based control method and apparatus

• US 8,866,781 B2
• Filed: 05/02/2013
• Issued: 10/21/2014
• Est. Priority Date: 05/21/2012
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Li, Yingtao
• Primary Examiner(s): SHAH, PRIYANK J
• Application Number:
  US13/875,476
• Publication Number:
  US 20130307765A1
• Time in Patent Office: 537 Days
• Field of Search: 345/173, 382/181, 382185-189, 715/780, 715/863, 715/773
• US Class Current: 345/173
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A contactless gesture-based control method and apparatus are provided. The method includes: obtaining gestures/movements of a user; obtaining control instructions corresponding to the gestures/movements according to pre-stored mapping relationships; and executing the control instructions, where the gestures/movements include: an index finger draws a circle, and remaining four fingers clench into a fist; or, five fingers clench together and a palm moves in a direction where a device is; or, a thumb, the index finger, and a middle finger extend naturally, the remaining two fingers clench together, and the index finger moves freely; or, the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the thumb moves up and down; or the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the middle finger moves up and down.

Claims

1. A contactless gesture-based control method, comprising:

creating and storing pre-stored mapping relationships between gestures/movements and control instructions;

obtaining gestures/movements of a user;

obtaining the control instructions corresponding to the gestures/movements according to the pre-stored mapping relationships between the gestures/movements and the control instructions; and

executing the control instructions,

wherein the gestures/movements comprise that three fingers including a thumb, an index finger, and a middle finger extend naturally, that two fingers including a ring finger and a pinkie finger are clenched together, and that one of the three fingers moves, and

wherein the mapping relationships between the gestures/movements and the control instructions comprise;

a corresponding control instruction is “

a mouse moves along a trajectory of a moving finger”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the index finger moves freely;

a corresponding control instruction is “

left-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the thumb moves up and down; and

a corresponding control instruction is “

right-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the middle finger moves up and down.

2. The method according to claim 1, wherein the gestures/movements comprise right-hand gestures/movements and left-hand gestures/movements.

3. The method according to claim 2, wherein after obtaining the gestures/movements of the user and before obtaining the control instructions corresponding to the gestures/movements according to the pre-stored mapping relationships between the gestures/movements and the control instructions, the method further comprises detecting whether the gestures/movements comprise a right-hand gesture/movement or a left-hand gesture/movement.

4. The method according to claim 2, wherein the mapping relationships between the right-hand gestures/movements and the control instructions further comprise at least one of the following:

a corresponding control instruction is “

Yes”

or “

No”

when the gestures/movements comprise the index finger draws a circle and the remaining four fingers clench into a fist, wherein the circle is larger than a contour of the hand; and

a corresponding control instruction is “

Back”

when the gestures/movements comprise the five fingers clench together and the palm moves in a direction of a device.

5. The method according to claim 4, wherein the corresponding control instruction is “

Yes”

when the gestures/movements comprise the index finger draws the circle clockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand, and wherein the corresponding control instruction is “

No”

when the gestures/movements comprise the index finger draws the circle counterclockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand.

6. The method according to claim 1, wherein obtaining the gestures/movements of the user comprises:

using an image collector to collect gesture images of the user, and extracting the gestures/movements out of the gesture images;

or

using a reflective posture tracker, an infrared posture tracker, or an ultrasonic posture tracker to obtain the gestures/movements of the user.

12. The method according to claim 1, wherein the one of the three fingers moves comprises the thumb moving.

13. The method according to claim 1, wherein the one of the three fingers moves comprises the index finger moving.

14. The method according to claim 1, wherein the one of the three fingers moves comprises the middle finger moving.

7. A contactless gesture-based control apparatus, comprising:

a mapping relationship creating unit configured to create and store pre-stored mapping relationships between gestures/movements and control instructions;

a gesture/movement obtaining unit configured to obtain gestures/movements of a user;

a control instruction obtaining unit configured to obtain, according to the pre-stored mapping relationships between the gestures/movements and control instructions, the control instruction corresponding to the gestures/movements obtained by the gesture/movement obtaining unit; and

a control instruction executing unit configured to execute the control instruction obtained by the control instruction obtaining unit,

wherein the gestures/movements comprise that three fingers including a thumb, an index finger, and a middle finger extend naturally, that two fingers including a ring finger and a pinkie finger are clenched together, and that one of the three fingers moves, and

wherein the mapping relationships between the gestures/movements and the control instructions comprise;

a corresponding control instruction is “

a mouse moves along a trajectory of a moving finger”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the index finger moves freely;

a corresponding control instruction is “

left-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the thumb moves up and down; and

a corresponding control instruction is “

right-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the middle finger moves up and down.

8. The apparatus according to claim 7, wherein the gestures/movements comprise right-hand gestures/movements and left-hand gestures/movements, and wherein the apparatus further rises a detecting unit configured to, after the gesture/movement obtaining unit obtains the gestures/movements and before the control instruction obtaining unit obtains the control instruction corresponding to the gestures/movements, detect whether the gestures/movements comprise a right-hand gesture/movement or a left-hand gesture/movement.

9. The apparatus according to claim 8, wherein the mapping relationships between the right-hand gestures/movements and the control instructions comprise at least one of the following:

a corresponding control instruction is “

Yes”

or “

No”

when the gestures/movements comprise the index finger draws a circle and the remaining four fingers clench into a fist, wherein the circle is larger than a contour of the hand; and

a corresponding control instruction is “

Back”

when the gestures/movements comprise the five fingers clench together and the palm moves in a direction of a device.

10. The apparatus according to claim 9, wherein the corresponding control instruction is “

Yes”

when the gestures/movements comprise the index finger draws the circle clockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand, and wherein the corresponding control instruction is “

No”

when the gestures/movements comprise the index finger draws the circle counterclockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand.

11. The apparatus according to claim 7, wherein the gesture/movement obtaining unit is specifically configured to:

use an image collector to collect a gesture image of the user, and extract the gestures/movements out of the gesture image;

or

use a reflective posture tracker, an infrared posture tracker, or an ultrasonic posture tracker to obtain the gestures/movements of the user.

15. The apparatus according to claim 7, wherein the one of the three fingers moves comprises the thumb moving.

16. The apparatus according to claim 7, wherein the one of the three fingers moves comprises the index finger moving.

17. The apparatus according to claim 7, wherein the one of the three fingers moves comprises the middle finger moving.

18. A contactless gesture-based control method, comprising:

creating and storing pre-stored mapping relationships between gestures/movements and control instructions;

obtaining gestures/movements of a user;

obtaining the control instructions corresponding to the gestures/movements according to the pre-stored mapping relationships between the gestures/movements and the control instructions; and

executing the control instructions,

wherein the gestures/movements comprise that three fingers including a thumb, an index finger, and a middle finger extend naturally, that two fingers including a ring finger and a pinkie finger are clenched together, and that one of the three fingers moves, and

wherein the mapping relationships between the gestures/movements and the control instructions comprise;

a corresponding control instruction is “

a mouse moves along a trajectory of a moving finger”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the index finger moves freely;

a corresponding control instruction is “

right-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the thumb moves up and down; and

a corresponding control instruction is “

left-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the middle finger moves up and down.

19. The method according to claim 18, wherein the mapping relationships between the gestures/movements and the control instructions further comprise at least one of the following:

a corresponding control instruction is “

Yes”

or “

No”

when the gestures/movements comprise the index finger draws a circle and the remaining four fingers clench into a fist, wherein the circle is larger than a contour of a hand; and

a corresponding control instruction is “

Back”

when the gestures/movements comprise the five fingers clench together and the palm moves in a direction of a device.

20. The method according to claim 18, wherein the corresponding control instruction is “

Yes”

when the gestures/movements comprise the index finger draws the circle counterclockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand, and wherein the corresponding control instruction is “

No”

when the gestures/movements comprise the index finger draws the circle clockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand.

21. A contactless gesture-based control apparatus, comprising:

a mapping relationship creating unit configured to create and store pre-stored mapping relationships between gestures/movements and control instructions;

a gesture/movement obtaining unit configured to obtain gestures/movements of a user;

a control instruction obtaining unit configured to obtain, according to the pre-stored mapping relationships between the gestures/movements and control instructions, the control instruction corresponding to the gestures/movements obtained by the gesture/movement obtaining unit; and

a control instruction executing unit configured to execute the control instruction obtained by the control instruction obtaining unit,

wherein the mapping relationships between the gestures/movements and the control instructions comprise;

a corresponding control instruction is “

Yes”

or “

No”

when the gestures/movements comprise the index finger draws a circle and the remaining four fingers clench into a fist, wherein the circle is larger than a contour of a hand;

a corresponding control instruction is “

Back”

when the gestures/movements comprise the five fingers clench together and the palm moves in a direction of a device;

a corresponding control instruction is “

a mouse moves along a trajectory of a moving finger”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the index finger moves freely;

a corresponding control instruction is “

right-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the thumb moves up and down; and

a corresponding control instruction is “

left-clicking of a mouse”

when the gestures/movements comprise the thumb, the index finger, and the middle finger extend naturally, the remaining two fingers clench together, and the middle finger moves up and down.

22. The apparatus according to claim 21, wherein the mapping relationships between the gestures/movements and the control instructions further comprise at least one of the following:

a corresponding control instruction is “

Yes”

or “

No”

when the gestures/movements comprise the index finger draws a circle and the remaining four fingers clench into a fist, wherein the circle is larger than a contour of a hand; and

a corresponding control instruction is “

Back”

when the gestures/movements comprise the five fingers clench together and the palm moves in a direction of a device.

23. The apparatus according to claim 22, wherein the corresponding control instruction is “

Yes”

when the gestures/movements comprise the index finger draws the circle counterclockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand, and wherein the corresponding control instruction is “

No”

when the gestures/movements comprise the index finger draws the circle clockwise and the remaining four fingers clench into the fist, wherein the circle is larger than the contour of the hand.