Navigation of a virtual plane using a zone of restriction for canceling noise

• US 9,652,030 B2
• Filed: 01/30/2009
• Issued: 05/16/2017
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Yeung, Brian, Andrews, Anton
• Primary Examiner(s): WIENER, ERIC A
• Application Number:
  US12/363,094
• Publication Number:
  US 20100199221A1
• Time in Patent Office: 3,028 Days
• Field of Search: 715863, 345157
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G10K 11/16 Methods or devices for protecting against, or for damping, noise or other acoustic waves in general G10K11/36 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A touchless human computer interface (HCI) provides a virtual surface in three-dimensional space and a zone of restriction for defining a level of sensitivity to movements in order to cancel noise that may be caused by natural wobble of a human appendage. The touchless HCI may receive input regarding a user movement, process the input to generate clean gesture data and analyze at least one dynamic variable to determine an interpreted action based upon a relationship of the clean gesture data with respect to the virtual surface.

Claims

1. A method for human computer interaction comprising:

defining a virtual surface in three-dimensional space;

defining a zone of restriction as an area or perimeter in X and Y dimensions that is a function of a Z coordinate in a Z dimension that extends in a forward direction from the human, the zone of restriction defining a sensitivity to movements of a human appendage, wherein the sensitivity to movements is reduced as the human appendage is moved forward relative to an origin that is a function of the location of the human;

receiving input generated from a camera regarding the human appendage movement in the three-dimensional space;

processing the input to generate processed gesture data;

determining at least one physical attribute of the processed gesture data; and

analyzing the at least one physical attribute of the processed gesture data to determine an interpreted action based upon a spatial relationship of the processed gesture data with respect to the virtual surface and the zone of restriction.

2. The method according to claim 1, wherein the virtual surface is a plane.

3. The method according to claim 1, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and then in back of the virtual surface, wherein the movement occurs within a predetermined time period and the interpreted action is a select.

4. The method according to claim 1, further comprising displaying a cursor as a function of the at least one physical attribute of the processed gesture data.

5. The method according to claim 4, wherein a shape of the cursor is dynamically modified as a function of a distance of a human appendage from the virtual surface.

6. The method according to claim 1, wherein the virtual surface is curved in a manner to compensate for ergonomics of a human body.

7. The method according to claim 1, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage in front of the virtual surface and the interpreted action is a hover.

8. The method according to claim 1, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage in back of the virtual surface and the interpreted action is a drag.

9. The method according to claim 1, wherein the processed gesture data includes clean gesture data.

10. A computer readable storage device storing instructions for human computer interaction, wherein the computer readable storage device is not a transitory signal, the instructions comprising:

defining a virtual surface in three-dimensional space;

defining a zone of restriction as an area or perimeter in X and Y dimensions that is a function of a Z coordinate in a Z dimension that extends in a forward direction from the human, the zone of restriction defining a sensitivity to movements of a human appendage, wherein the sensitivity to movements is reduced as the human appendage is moved forward relative to an origin that is a function of the location of the human;

receiving input generated from a camera regarding the human appendage movement in the three-dimensional space;

processing the input to generate processed gesture data;

determining at least one physical attribute of the processed gesture data, wherein the at least one physical attribute of the processed gesture data comprises the human appendage movement from in back of the virtual surface to in front of the virtual surface and then in back of the virtual surface; and

analyzing the at least one physical attribute of the processed gesture data to determine an interpreted action based upon a spatial relationship of the processed gesture data with respect to the virtual surface and the zone of restriction.

11. The computer readable storage device according to claim 10, wherein the virtual surface is a plane.

12. The computer readable storage device according to claim 10, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and then in back of the virtual surface, wherein the movement occurs within a predetermined time period and the interpreted action is a select.

13. The computer readable storage device according to claim 10, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage in front of the virtual surface and the interpreted action is a hover.

14. The computer readable storage device according to claim 10, wherein the at least one physical attribute of the processed gesture data is movement of a human appendage in back of the virtual surface and the interpreted action is a drag.

15. The computer readable storage device according to claim 10, wherein the processed gesture data includes clean gesture data.

16. A system for analyzing human computer interaction comprising:

a camera, the camera generating a signal as a function of a state of a three dimensional environment;

a gesture engine adapted to;

receive the camera signal; and

process the camera signal to generate a processed gesture signal; and

determine at least one physical attribute of the processed gesture data; and

an action interpretation engine adapted to;

interpret the processed gesture signal as a function of a virtual surface and a zone of restriction, the zone of restriction having an area or perimeter in X and Y dimensions that is a function of a Z coordinate in a Z dimension that extends in a forward direction from the human and defining a sensitivity to movements of a human appendage, wherein the sensitivity to movements is reduced as the human appendage is moved forward relative to an origin that is a function of the location of the human; and

analyze the at least one physical attribute of the processed gesture data to determine an interpreted action based upon a spatial relationship of the processed gesture data with respect to the virtual surface and the zone of restriction.

17. The system according to claim 16, wherein the virtual surface is flat directly in front of a user and curved around at least one side of a user.

18. The system according to claim 16, wherein the processed gesture signal includes a clean gesture signal.