GESTURE DISAMBIGUATION USING ORIENTATION INFORMATION

• US 20150123901A1
• Filed: 11/04/2013
• Published: 05/07/2015
• Est. Priority Date: 11/04/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schwesinger, Mark, Yang, Emily, Kapur, Jay, Paolantonio, Sergio, Klein, Christian
• Application Number:
  US14/071,299
• Publication Number:
  US 20150123901A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Embodiments are disclosed that relate to controlling a computing device based upon gesture input. In one embodiment, orientation information of the human subject is received, wherein the orientation information includes information regarding an orientation of a first body part and an orientation of a second body part. A gesture performed by the first body part is identified based on the orientation information, and an orientation of the second body part is identified based on the orientation information. A mapping of the gesture to an action performed by the computing device is determined based on the orientation of the second body part.

Claims

1. A method for controlling a computing device based on motion of a human subject, the method comprising:

receiving orientation information of the human subject including information regarding an orientation of a first body part and an orientation of a second body part;

identifying a gesture performed by the first body part based on the orientation information;

identifying an orientation of the second body part based on the orientation information;

determining a mapping of the gesture to an action performed by the computing device based on the orientation of the second body part.

2. The method of claim 1, wherein the orientation information is representative of a virtual skeleton that models the human subject with a plurality of virtual joints characterized by three-dimensional positions, the virtual skeleton derived from a depth video of a depth camera imaging the human subject.

3. The method of claim 1, wherein determining the mapping further includes ignoring the gesture as a false positive based on the orientation of the second body part.

4. The method of claim 1, wherein determining the mapping includes mapping the gesture to a first action when the second body part is in a first orientation and mapping the gesture to a second action different from the first action when the second body part is in a second orientation different from the first orientation.

5. The method of claim 4, wherein the second body part includes a head of the human subject, the first orientation includes the head of the human subject looking at a display, and the second orientation includes the head of the human subject looking away from the display.

6. The method of claim 4, wherein the second body part includes legs of the human subject, the first orientation of the legs includes a standing position, and the second orientation of the legs includes a sitting position or a laying position.

7. The method of claim 4, wherein the second body part includes a hand, the first orientation of the hand includes the hand being empty, and the second orientation of the hand includes the hand holding an object.

8. The method of claim 7, further comprising:

recognizing that the object that the hand is holding is a secondary device in communication with the computing device, and the second action relating to operation of the secondary device.

9. A computing device comprising:

a logic machine;

a storage machine holding instructions that when executed by the logic machine;

receive orientation information of a first human subject including information regarding a first body part and a second body part;

identify a gesture performed by the first body part of the human subject based on the orientation information;

if the second body part is in a first orientation, determine a mapping of the gesture to a first action performed by the computing device based on the orientation of the second body part; and

if the second body part is in a second orientation, ignore the gesture as a false positive.

10. The computing device of claim 9, wherein the orientation information is representative of a virtual skeleton that models the first human subject with a plurality of virtual joints characterized by three-dimensional positions, the virtual skeleton derived from a depth video of a depth camera imaging the first human subject.

11. The computing device of claim 9, wherein the second body part includes a head of the first human subject, the first orientation includes the head of the first human subject looking at a display, and the second orientation includes the head of the first human subject looking away from the display.

12. The computing device of claim 11, wherein the storage machine holds instructions that when executed by the logic machine:

identify a second human subject based on the orientation information;

recognize that the head of the first human subject is looking away from the display and toward the second human subject; and

determine a mapping of the gesture to a second action performed by the computing device based on the head of the first human subject looking at the second human subject.

13. The computing device of claim 9, wherein the first body part includes an arm of the human subject, the second body part includes a hand, the first orientation of the hand includes the hand being empty, and the second orientation of the hand includes the hand holding an object.

14. The computing device of claim 12, wherein the storage machine holds instructions that when executed by the logic machine:

recognize that the object that the hand is holding is a secondary device in communication with the computing device, and wherein determining the mapping includes mapping the gesture to a second action different from the first action and related to operation of the secondary device.

15. The computing device of claim 9, wherein the second body part includes legs of the first human subject, the first orientation of the legs includes a standing position, and the second orientation of the legs includes a sitting position or a laying position.

16. The computing device of claim 9, wherein the first body part includes an arm of the first human subject, the second body part includes a head of the first human subject, the first orientation includes the head being positioned above the arm, and the second orientation includes the head being positioned below the arm.

17. A method for controlling a computing device based on motion of a first human subject, the method comprising:

receiving orientation information for a first human subject including information regarding an orientation of a first body part and an orientation of a second body part;

identifying a gesture performed by the first body part based on the orientation information, the gesture being of a gesture type having a plurality of gesture instances mapped to different actions;

identifying an orientation of the second body part based on the orientation information;

determining a gesture instance of the gesture type performed by the first body part based on the orientation of the second body part; and

performing an action mapped to the gesture instance that controls operation of the computing device.

18. The method of claim 17, wherein the orientation information is representative of a virtual skeleton that models the first human subject with a plurality of virtual joints characterized by three-dimensional positions, the virtual skeleton derived from a depth video of a depth camera imaging the first human subject.

19. The method of claim 17, wherein the gesture type includes a gesture instance associated with a false positive status, and if the gesture instance associated with the false positive status is determined to be performed by the first body part, then ignoring the gesture as a false positive.

20. The method of claim 17, wherein the second body part includes a head of the first human subject, a first orientation of the head including the head looking at a second human subject, a second orientation of the head including the head looking at a display, a third orientation of the head including the head looking at a controller held by the human subject, and each of the first orientation, the second orientation, and the third orientation being associated with a different gesture instance to which a different action is mapped.