Method and system implementing user-centric gesture control

• US 20110296353A1
• Filed: 05/28/2010
• Published: 12/01/2011
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Tracy, Colin, Rafaii, Abbas, Ahmed, Abdelrehim
• Granted Patent: US 8,693,724 B2
• Time in Patent Office: Days
• US Class Current: 715/848
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A user-centric method and system to identify user-made gestures to control a remote device images the user using a three-dimensional image system, and defines at least one user-centric three-dimensional detection zone dynamically sized appropriately for the user, who is free to move about. Images made within the detection zone are compared to a library of stored gestures, and the thus identified gesture is mapped to an appropriate control command signal coupleable to the remote device. The method and system also provides of a first user to hand off control of the remote device to a second user.

Claims

1. A method implementing user-centric gesture recognition enabling a user to remotely gesture-control a device, the method comprising:

disposing a three-dimensional imaging system whose field of view encompasses a three-dimensional space in which at least one user may be present, and recognizing a user when said user is within said field of view;

providing a gesture recognition system that defines a world coordinate system within said three-dimensional space based upon location of said user as determined by said imaging system, said world coordinate system moving with movement of said user;

said gesture recognition system defining relative to a current position of said user as determined by said imaging system a dynamically-sized and positioned three-dimensional zone of interaction within which zone said user may make at least one gesture;

said gesture recognition system storing a gesture library of at least one pre-defined gesture useable to control said device;

said gesture recognition system recognizing a gesture made by said user within said three-dimensional space and comparing said gesture to contents of said gesture library;

upon finding a match between a recognized said user-gesture and contents of said gesture library, outputting a signal useable to control said device.

2. The method of claim 1, further including defining at least one sub-region within said three-dimensional zone of interaction, wherein presence of a portion of said user can be detected.

3. The method of claim 1, wherein said gesture library stores at least one gesture selected from a group consisting of (i) a gesture signifying desire to acquire control of said device, (ii) a gesture signifying handing off remote control of said device from said user to a successor user, (iii) a gesture made with one hand, (iv) a gesture made with two hands, and (v) a gesture causing a menu to be displayed on said device.

4. The method of claim 1, further including causing said device to alert a user that a user-gesture may not have been correctly recognized by said gesture control system, and inviting said user to confirm improper recognition of a gesture.

5. The method of claim 4, wherein a user can confirm improper recognition of a gesture by making a user movement within a sub-region defined within said three-dimensional zone of interaction.

6. The method of claim 5, further including causing said device to signal said user that improper recognition of a gesture may have occurred, and causing said device to provide an option for said user to at least confirm said improper gesture recognition.

7. The method of claim 1, wherein said gesture library stores gestures that are stateless.

8. The method of claim 1, wherein said three-dimensional imaging system is a system selected from a group consisting of a (i) parallex-method imaging system, (ii) a time-of-flight imaging system, (iii) a phase-based time-of-flight imaging system, (iv) a shutter-based time-of-flight imaging system, and (v) a system using at least two spaced-apart cameras.

9. The method of claim 1, wherein said device includes at least one of (i) a display, (ii) a computer screen, (iii) a control panel, and (iv) a television receiver.

10. A system enabling user to remotely gesture-control a device using user-concentric gesture recognition, the system comprising:

a three-dimensional imaging system whose field of view encompasses a three-dimensional space in which at least one user may be present, and recognizing a user when said user is within said field of view;

a gesture recognition system, coupled to said three-dimensional imaging system, that defines a world coordinate system within said three-dimensional space based upon location of said user as determined by said imaging system, said world coordinate system moving with movement of said user;

said gesture recognition system including means for defining, relative to a current position of said user as determined by said imaging system, a dynamically-sized and positioned three-dimensional zone of interaction within which zone said user may make at least one gesture; and

a gesture library of at least one pre-defined gesture useable to control said device;

said gesture recognition system recognizing a gesture made by said user within said three-dimensional space and comparing said gesture to contents of said gesture library;

upon finding a match between a recognized said user-gesture and contents of said gesture library, said gesture recognition system outputting a signal useable to control said device.

11. The system of claim 10, wherein said gesture recognition system defines at least one sub-region within said three-dimensional zone of interaction, wherein presence of a portion of said user can be detected.

12. The system of claim 10, wherein said gesture library stores at least one gesture selected from a group consisting of ((i) a gesture signifying desire to acquire control of said device, (ii) a gesture signifying handing off remote control of said device from said user to a successor user, (iii) a gesture made with one hand, (iv) a gesture made with two hands, and (v) a gesture causing a menu to be displayed on said device.

13. The system of claim 10, further including means for causing said device to alert a user that a user-gesture may not have been correctly recognized by said gesture control system, and inviting said user to confirm improper recognition of a gesture.

14. The system of claim 13, wherein a user can confirm improper recognition of a gesture by making a user movement within a sub-region defined within said three-dimensional zone of interaction.

15. The system of claim 14, further including means for causing said device to signal said user that improper recognition of a gesture may have occurred, and causing said device to provide an option for said user to at least confirm said improper gesture recognition.

16. The system of claim 10, wherein said gesture library stores gestures that are stateless.

17. The system of claim 10, wherein said three-dimensional imaging system is a system selected from a group consisting of a (i) parallex-method imaging system, (ii) a time-of-flight imaging system, (iii) a phase-based time-of-flight imaging system, (iv) a shutter-based time-of-flight imaging system, and (v) a system using at least two spaced-apart cameras.

18. The system of claim 10, wherein said device includes at least one of (i) a display, (ii) a computer screen, (iii) a control panel, and (iv) a television receiver.