Gesture processing using a domain-specific gesture language

• US 9,946,354 B2
• Filed: 08/29/2014
• Issued: 04/17/2018
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Livshits, Benjamin, Veanes, Margus, D'Antoni, Loris, Figueiredo, Lucas S., Molnar, David A.
• Primary Examiner(s): DRAGOESCU, CLAUDIA B
• Application Number:
  US14/472,942
• Publication Number:
  US 20160062468A1
• Time in Patent Office: 1,327 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

The claimed subject matter includes techniques for processing gestures. An example method includes receiving a gesture from an application. The gesture includes one or more primitives from a language that is domain-specific to gestures. The method also further includes receiving skeletal data from a motion detection system. The method also includes comparing the skeletal data with the gesture from the application in a runtime module. The method also further includes sending a gesture event to the application.

Claims

1. A system for static analysis of gestures, comprising a processor to:

receive a plurality of gestures comprising one or more primitives from a language domain-specific to gestures specifying an ordering of poses to be performed for each gesture, and translate each of the gestures into a relative constraint that represents an allowable coordinate for a specific body joint; and

statically analyze the constraint for a conflict.

4. The system of claim 1, wherein the processor is to also translate the gesture into a time constraint.

5. The system of claim 1, comprising a motion detector and an application, wherein the processor is to receive skeletal data from the motion detector, wherein the processor is to return a result based on the skeletal data to the application without sending the skeletal data to the application.

6. The system of claim 5, wherein the skeletal data comprises a position, a movement, a rotation about skeletal joints, or any combination thereof.

7. The system of claim 5, wherein the application is to a display a privacy notice that explains that skeletal data is collected by the motion detection system but that specific detected gesture events are available to the applications for application functionality.

8. The system of claim 7, wherein the privacy notice comprises an application-eye-view that displays information that the application is to access if the user agrees.

9. The system of claim 5, comprising a trust boundary to prevent the processor from sending the skeletal data to the application.

10. The system of claim 4, wherein the time constraint indicates a speed or velocity for a pose from a previous pose.

2. A method for processing gestures, comprising:

receiving, via a processor, a gesture from an application, the gesture comprising one or more primitives from a language that is domain-specific to gestures specifying an ordering of poses to be performed for each gesture;

translating, via the processor, the gesture into a relative constraint that represents an allowable coordinate for a specific body joint;

statically analyzing, via the processor, the application for a conflict based on the relative constraint.

11. The method of claim 2, comprising translating, via the processor, the gesture into a time constraint.

12. The method of claim 2, comprising receiving skeletal data, via a motion detector, and returning, via the processor, a result based on the skeletal data to an application without sending the skeletal data to the application.

13. The method of claim 12, wherein the skeletal data comprises a position, a movement, a rotation about skeletal joints, or any combination thereof.

14. The method of claim 12, comprising displaying, via the application, a privacy notice that explains that skeletal data is collected by the motion detection system but that specific detected gesture events are available to the applications for application functionality.

15. The method of claim 14, wherein the privacy notice comprises an application-eye-view that displays information that the application is to access if the user agrees.

16. The method of claim 12, comprising preventing, via a trust boundary, the processor from sending the skeletal data to the application.

17. The method of claim 11, wherein the time constraint indicates a speed or velocity for a pose from a previous pose.

3. One or more computer-readable storage device for processing gestures, comprising a plurality of instructions that, when executed by a processor, cause the processor to:

receive a gesture from an application, the gesture comprising one or more primitives from a language that is domain-specific to gestures specifying an ordering of poses to be performed for each gesture;

receive a gesture library, the gesture library comprising primitives from the language that is domain-specific to gestures;

translate the gesture into a relative constraint that represents an allowable coordinate for a specific body joint; and

statically analyze the application for at least one conflict based on the relative constraint.

18. The one or more computer-readable storage device of claim 3, comprising instructions that cause the processor to translate the gesture into a time constraint.

19. The one or more computer-readable storage device of claim 3, comprising instructions that cause the processor to receive skeletal data from a motion detect and return a result based on the skeletal data to an application without sending the skeletal data to the application.

20. The one or more computer-readable storage device of claim 19, comprising instructions that cause the processor to display a privacy notice that explains that skeletal data is collected by the motion detection system but that specific detected gesture events are available to the applications for application functionality.

21. The one or more computer-readable storage device of claim 19, wherein the privacy notice comprises an application-eye-view that displays information that the application is to access if the user agrees.

22. The one or more computer-readable storage device of claim 19, comprising instructions that cause the processor to avoid sending the skeletal data to the application.

23. The one or more computer-readable storage device of claim 18, wherein the time constraint indicates a speed or velocity for a pose from a previous pose.