NAVIGATION OF A VIRTUAL PLANE USING DEPTH

• US 20100199221A1
• Filed: 01/30/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yeung, Brian, Andrews, Anton
• Granted Patent: US 9,652,030 B2
• Time in Patent Office: Days
• US Class Current: 715/850
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G10K 11/16 Methods or devices for protecting against, or for damping, noise or other acoustic waves in general G10K11/36 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A touchless HCI provides a virtual surface in three-dimensional space. The touchless HCI may receive input regarding a user movement, process the input to generate clean gesture data and analyze at least one dynamical variable to determine an interpreted action based upon a relationship of the clean gesture data with respect to the virtual surface.

Claims

1. A method for touchless human computer interaction (“

HCI”

) comprising;

defining a virtual surface in three-dimensional space;

receiving input regarding a user movement in the three-dimensional space;

processing the input to generate clean gesture data;

determining at least one physical attribute of the clean gesture data;

analyzing the at least one physical attribute of the clean gesture data to determine an interpreted action based upon a relationship of the clean gesture data with respect to the virtual surface.

2. The method according to claim 1, wherein the virtual surface is a plane.

3. The method according to claim 1, wherein the input is generated from a camera.

4. The method according to claim 1, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage in front of the virtual surface and the interpreted action is a hover.

5. The method according to claim 1, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage in back of the virtual surface and the interpreted action is a drag.

6. The method according to claim 1, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and then in back of the virtual surface, wherein the movement occurs within a predetermined time period and the interpreted action is a select.

7. The method according to claim 1, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and the interpreted action is a grab of the virtual surface.

8. The method according to claim 1, further comprising displaying a cursor as a function of the at least one physical attribute of the clean gesture data.

9. The method according to claim 8, wherein the shape of the cursor is dynamically modified as a function of a distance of a human appendage from the virtual surface.

10. The method according to claim 1, further comprising defining a zone of restriction, the zone of restriction defining a sensitivity to movements parallel to the virtual surface and wherein the sensitivity varies as a function of a perpendicular distance to the virtual surface.

11. The method according to claim 1, wherein the virtual surface is curved in a manner to compensate for ergonomics of a human body.

12. A computer readable medium storing instructions for touchless HCI, the instructions comprising:

defining a virtual surface in three-dimensional space;

receiving input regarding a user movement in the three-dimensional space;

processing the input to generate clean gesture data;

determining at least one physical attribute of the clean gesture data;

analyzing the at least one physical attribute of the clean gesture data to determine an interpreted action based upon a relationship of the clean gesture data with respect to the virtual surface.

13. The computer readable medium according to claim 12, wherein the virtual surface is a plane.

14. The computer readable medium according to claim 12, wherein the input is generated from a camera.

15. The computer readable medium according to claim 12, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage in front of the virtual surface only and the interpreted action is a hover.

16. The computer readable medium according to claim 12, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage in back of the virtual surface only and the interpreted action is a drag.

17. The computer readable medium according to claim 12, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and then in back of the virtual surface, wherein the movement occurs within a predetermined time period and the interpreted action is a select.

18. The computer readable medium according to claim 12, wherein the at least one physical attribute of the clean gesture data is movement of a human appendage from in back of the virtual surface to in front of the virtual surface and the interpreted action is a grab of the virtual surface.

19. A system for analyzing HCI in a touchless environment comprising:

a camera, the camera generating a signal as a function of a state of a three dimensional environment;

a gesture engine adapted to;

receive the camera signal;

process the camera signal to generate a clean gesture signal;

a physics engine adapted to;

receive the clean gesture signal;

update dynamic variables associated with a human appendage, a virtual surface and a zone of restriction;

an action interpretation engine adapted to;

interpret the clean gesture signal as a function of the virtual surface and the zone of restriction as a select action, a swipe action, a drag action or a hover and highlight action.

20. The system according to claim 19, wherein the virtual surface is flat directly in front of a user and curved around at least one side of a user.