SYSTEM AND METHOD FOR DETECTING GESTURES

• US 20140157209A1
• Filed: 03/12/2013
• Published: 06/05/2014
• Est. Priority Date: 12/03/2012
• Status: Abandoned Application

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Dalal, Navneet, Nariyawala, Mehul, Mohan, Ankit, Gulshan, Varun
• Application Number:
  US13/796,772
• Publication Number:
  US 20140157209A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A system and method that includes detecting an application change within a multi-application operating framework; updating an application hierarchy model for gesture-to-action responses with the detected application change; detecting a gesture; according to the hierarchy model, mapping the detected gesture to an action of an application; and triggering the action.

Claims

1. A method comprising:

detecting an application change within a multi-application operating system;

updating an application hierarchy model for gesture-to-action responses with the detected application change;

detecting a gesture;

according to the hierarchy model, mapping the detected gesture to an action of an application; and

triggering the mapped action of the application.

2. The method of claim 1, wherein detecting an application change comprises detecting a selection of a new top-level application within the operating system;

and updating an application-gesture priority queue comprises promoting the new top-level application in the hierarchy model.

3. The method of claim 2, further comprising signaling a change in the application-gesture priority queue upon updating the application-gesture priority queue.

4. The method of claim 1, wherein detecting application change comprises detecting a change of context within an active application.

5. The method of claim 4, wherein detecting a change of context comprises detecting the loading of a media object in an active application.

6. The method of claim 1, wherein mapping the detected gesture to an action of an application comprises:

if a gesture is not actionable within an initial application in the hierarchy model, progressively checking gesture-to-action responses of an application in a lower hierarchy of the hierarchy model.

7. The method of claim 1, wherein detecting a gesture comprises limiting gesture detection processing to a subset of gestures defined by gestures in the application hierarchy model.

8. The method of claim 1, further comprising detecting user settings according to facial recognition of a user;

and if an active application is set as a preferred application in the detected user settings, promoting the preferred application in the application hierarchy model.

9. The method of claim 1, wherein a gesture is actionable by at least two applications in the hierarchy model;

and wherein mapping the detected gesture to an action of an application comprises selecting the action of the application with the highest priority in the hierarchy model.

10. The method of claim 9, wherein at least one gesture in the set of gestures is defined by a thumbs up gesture heuristic that is used for at least voting, approval, and confirming for a first, second, and third application respectively.

11. The method of claim 1, wherein detecting a gesture comprises detecting a gesture presence from a set of gestures characterized in the gesture-to-action responses in the hierarchy model.

12. The method of claim 11, further comprising for at least one gesture in the set of gestures, subsequently initiating the action for at least a second time if a prolonged presence of the at least one gesture is detected.

13. The method of claim 11, wherein for at least one gesture in the set of gestures, initiating a modified form of the action if a translation of the at least one gesture is detected.

14. The method of claim 13, wherein at least one gesture in the set of gestures is defined by a pinch gesture heuristic;

wherein the gesture-to-action response is a scrolling response if the detected gesture is a pinch gesture heuristic and a translation of the pinch gesture along an axis scrolls is detected.

15. The method of claim 11, wherein at least a first gesture in the set of gestures is defined by a thumbs up gesture heuristic, at least a second gesture in the set of gestures is defined by a mute gesture heuristic, and at least a third gesture in the set of gestures is defined by an extended sideways thumb gesture heuristic.

16. The method of claim 15, wherein at least a fourth gesture in the set of gestures is defined by a palm up gesture heuristic and at least a fifth gesture in the set of gestures is defined by a palm down heuristic.

17. A method comprising:

detecting an application change within a multi-application operating framework;

updating an application hierarchy model for gesture-to-action responses with the detected application change;

detecting a gesture from a set of presence-based gestures;

mapping the detected gesture to a gesture-to-action response in the hierarchy model, wherein if a gesture-to-action response is not identified within an initial application in the hierarchy model, progressively checking gesture-to-action responses of an application in a lower hierarchy of the hierarchy model;

if the detected gesture is a first gesture, detecting translation of the first gesture;

if the detected gesture is a second gesture, detecting rotation of the first gesture;

if the detected gesture is a third gesture, detecting prolonged presence of the third gesture; and

triggering the action, wherein the action is modified according to a modified action if defined for translation, prolonged presence or rotation.

18. The method of claim 17, wherein at least one gesture in the set of presence-based gestures is defined by a thumbs up gesture heuristic, at least a one gesture in the set of gestures is defined by a mute gesture heuristic, and at least one gesture in the set of gestures is defined by an extended sideways thumb gesture heuristic.

19. The method of claim 17, wherein at least one gesture in the set of presence-based gestures is defined by a palm up gesture heuristic, and at least a one gesture in the set of gestures is defined by a palm down gesture heuristic.