GESTURE-BASED MOBILE INTERACTION

• US 20090051648A1
• Filed: 08/20/2008
• Published: 02/26/2009
• Est. Priority Date: 08/20/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: SHAMAIE, ATID, MACDOUGALL, FRANCIS
• Granted Patent: US 9,261,979 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Gesture-based mobile interaction, motion of a device is sensed using image data, and a gesture corresponding to the sensed motion of the device is recognized. Functionality of the device corresponding to the recognized gesture is determined and invoked.

Claims

1. A computer-implemented method comprising:

sensing motion of a device using image data;

recognizing a gesture corresponding to the sensed motion of the device;

determining functionality of the device corresponding to the recognized gesture; and

invoking the functionality.

2. The method of claim 1, wherein the motion comprises:

a first motion in a first trajectory; and

a second motion in a second, different trajectory.

3. The method of claim 1, wherein recognizing the gesture further comprising:

recognizing a character shape, geometric shape or pattern defined by the sensed motion of the device.

4. The method of claim 1, wherein:

sensing the motion of the device further comprises detecting a subconscious body movement of a user of the device; and

the invoked functionality is associated with the subconscious body movement.

5. The method of claim 1, further comprising:

detecting first and second user inputs at first and second times, respectively,

wherein sensing the motion of the device further comprise sensing motion of the device occurring between the first and second times.

6. The method of claim 5, wherein the first input is a shaking user input, a control selection user input, or a held-static user input.

7. The method of claim 1, wherein recognizing the gesture further comprises:

selecting the gesture based on comparing the sensed motion to a modeled vocabulary of gestures.

8. The method of claim 7, further comprising:

normalizing the sensed motion of the device; and

comparing the normalized, sensed motion to the modeled vocabulary of gestures.

9. The method of claim 7, further comprising:

determining a parameter based on comparing the sensed motion to the normalized motion.

10. The method of claim 9, wherein the functionality is determined based on the parameter.

11. The method of claim 9, further comprising inputting the parameter to the invoked functionality.

12. The method of claim 7, wherein the vocabulary of gestures are modeled using a Hidden Markov Model (HMM) or using a rule-based model.

13. The method of claim 7, further comprising:

selectively activating fewer than all available gestures in the vocabulary of gestures,

wherein the gesture is selected based on comparing the sensed motion to the activated gestures.

14. The method of claim 13, wherein the fewer than all of the available gestures are enabled based on a manufacturer setting, a user setting or an application setting.

15. The method of claim 7, further comprising:

training the device to recognize a particular gesture; and

adding the particular gesture to the vocabulary of gestures.

16. The method of claim 1, wherein invoking the functionality further comprises executing an application, manipulating an image, or inputting a character.

17. The method of claim 1, wherein sensing motion of the device further comprises:

detecting first and second user inputs and at first and second times, respectively, the first time occurring before the second time;

sensing a first motion in a first trajectory before the first time;

sensing a second motion in a second, different trajectory after the second time;

joining the first and second motions; and

outputting the joined first and second motions as the sensed motion.

18. The method of claim 1, wherein invoking the functionality further comprises navigating in a virtual environment, manipulating an image, entering a character, executing an application, or invoking media hub functionality.

19. A device comprising:

a sensor configured to sense motion using image data; and

a processor configured to;

recognize a gesture corresponding to the sensed motion sensed by the sensor;

determine functionality corresponding to the recognized gesture; and

invoke the functionality.

20. The device of claim 19, wherein:

the sensor is a camera; and

the motion is sensed using optical flow.

21. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

sensing motion of a device using image data;

recognizing a gesture corresponding to the sensed motion of the device;

determining functionality of the device corresponding to the recognized gesture; and

invoking the functionality.