Gesture-based mobile interaction

• US 9,261,979 B2
• Filed: 08/20/2008
• Issued: 02/16/2016
• Est. Priority Date: 08/20/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shamaie, Atid, MacDougall, Francis B.
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): BRAY, STEPHEN A
• Application Number:
  US12/194,772
• Publication Number:
  US 20090051648A1
• Time in Patent Office: 2,736 Days
• Field of Search: 345156-158, 715/863, 715/765
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Motion of a device is sensed using image data, and a gesture corresponding to the sensed motion of the device is recognized. Functionality of the device corresponding to the recognized gesture is determined and invoked.

Claims

1. A computer-implemented method comprising:

sensing motion of a device;

determining a normalized motion by normalizing a shape of the sensed motion of the device, wherein the normalizing comprises scaling the sensed motion in a plurality of spatial dimensions;

recognizing a gesture corresponding to the normalized motion of the device;

determining functionality of the device based on the recognized gesture and a ratio of a size of the shape of the sensed motion in a first of the plurality of spatial dimensions to a size of the shape of the sensed motion in a second of the plurality of spatial dimensions; and

invoking the determined functionality.

2. The method of claim 1, wherein the motion comprises:

a first motion in a first trajectory; and

a second motion in a second, different trajectory.

3. The method of claim 1, wherein recognizing the gesture further comprises:

recognizing a character shape, geometric shape or pattern defined by the sensed motion of the device.

4. The method of claim 1, wherein:

sensing the motion of the device further comprises detecting a subconscious body movement of a user of the device; and

the invoked functionality is associated with the subconscious body movement.

5. The method of claim 1, further comprising:

detecting first and second user inputs at first and second times, respectively,

wherein sensing the motion of the device further comprises sensing motion of the device occurring between the first and second times.

6. The method of claim 5, wherein the first input is a shaking user input, a control selection user input, or a held-static user input.

7. The method of claim 1, wherein recognizing the gesture further comprises:

selecting the gesture based on comparing the sensed motion to a modeled vocabulary of gestures.

8. The method of claim 7, wherein the vocabulary of gestures are modeled using a Hidden Markov Model (HMM) or using a rule-based model.

9. The method of claim 7, further comprising:

selectively activating fewer than all available gestures in the vocabulary of gestures,

wherein the gesture is selected based on comparing the sensed motion to the activated gestures.

10. The method of claim 9, wherein the fewer than all of the available gestures are enabled based on a manufacturer setting, a user setting or an application setting.

11. The method of claim 7, further comprising:

training the device to recognize a particular gesture; and

adding the particular gesture to the vocabulary of gestures.

12. The method of claim 1, wherein invoking the functionality further comprises executing an application, manipulating an image, or inputting a character.

13. The method of claim 1, wherein sensing motion of the device further comprises:

detecting first and second user inputs and at first and second times, respectively, the first time occurring before the second time;

sensing a first motion in a first trajectory before the first time;

sensing a second motion in a second, different trajectory after the second time;

joining the first and second motions; and

outputting the joined first and second motions as the sensed motion.

14. The method of claim 1, wherein invoking the functionality further comprises navigating in a virtual environment, manipulating an image, entering a character, executing an application, or invoking media hub functionality.

15. The method of claim 1, wherein motion of the device is sensed using image data.

16. The method of claim 1, wherein the normalizing includes adjusting a magnitude of the sensed motion along at least one dimension of the plurality of spatial dimensions.

17. The method of claim 1, wherein the sensed motion is scaled by 0% in at least one of the plurality of spatial dimensions.

18. The method of claim 1, wherein the determining the functionality is dependent on a speed at which the gesture was performed.

19. A device comprising:

a sensor configured to sense motion; and

a processor configured to;

determine a normalized motion by normalizing a shape of the motion sensed by the sensor, wherein the normalizing comprises scaling the sensed motion in a plurality of spatial dimensions;

recognize a gesture corresponding to the normalized motion sensed by the sensor;

determine functionality of the device based on the recognized gesture and a ratio of a size of the shape of the motion sensed by the sensor in a first of the plurality of spatial dimensions to a size of the shape of the motion sensed by the sensor in a second of the plurality of spatial dimensions; and

invoke the determined functionality.

20. The device of claim 19, wherein:

the sensor is a camera; and

the motion is sensed using optical flow.

21. A non-transitory computer-readable medium tangibly encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

sensing motion of a device;

determining a normalized motion by normalizing a shape of the sensed motion of the device, wherein the normalizing comprises scaling the sensed motion in a plurality of spatial dimensions;

recognizing a gesture corresponding to the normalized motion of the device;

determining functionality of the device based on the recognized gesture and a ratio of a size of the shape of the sensed motion in a first of the plurality of spatial dimensions to a size of the shape of the sensed motion in a second of the plurality of spatial dimensions; and

invoking the determined functionality.

22. An apparatus comprising:

means for sensing motion of a device;

means for determining a normalized motion by normalizing a shape of the sensed motion of the device, wherein the normalizing comprises scaling the sensed motion in a plurality of spatial dimensions;

means for recognizing a gesture corresponding to the normalized motion of the device;

means for determining functionality of the device based on the recognized gesture and a ratio of a size of the shape of the sensed motion in a first of the plurality of spatial dimensions to a size of the shape of the sensed motion in a second of the plurality of spatial dimensions; and

means for invoking the determined functionality.