SYSTEMS AND METHODS FOR PERFORMING A DEVICE ACTION BASED ON A DETECTED GESTURE

• US 20140368422A1
• Filed: 06/14/2013
• Published: 12/18/2014
• Est. Priority Date: 06/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: GUPTA, Samir K., KEATING, Virginia Walker, HILDRETH, Evan R.
• Granted Patent: US 9,020,194 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Systems and methods for performing an action based on a detected gesture are provided. The systems and methods provided herein may detect a direction of an initial touchless gesture and process subsequent touchless gestures based on the direction of the initial touchless gesture. The systems and methods may translate a coordinate system related to a user device and a gesture library based on the detected direction such that subsequent touchless gestures may be processed based on the detected direction. The systems and methods may allow a user to make a touchless gesture over a device to interact with the device independent of the orientation of the device since the direction of the initial gesture can set the coordinate system or context for subsequent gesture detection.

Claims

1. A method for performing an action, comprising:

detecting, by a first gesture detection sensor coupled to or integrated in a device, motion of a control object moving over the device;

determining, by one or more processors of the device, a direction of movement of the control object;

detecting, by a second gesture detection sensor, a touchless gesture performed over the device; and

performing, by one or more processors, an action based on the detected touchless gesture and the determined direction of movement.

2. The method of claim 1, further comprising translating a coordinate system related to the device based on the determined direction of movement.

3. The method of claim 2, wherein translating a coordinate system comprises translating a gesture library based on the determined direction of movement.

4. The method of claim 3, wherein performing an action comprises:

matching the touchless gesture to a translated gesture in the translated gesture library; and

performing an action corresponding to the matching translated gesture.

5. The method of claim 2, further comprising:

translating an expected gesture to the translated coordinate system;

matching the detected touchless gesture to the translated expected gesture; and

performing an action corresponding to the expected gesture.

6. The method of claim 1, wherein detecting movement of a control object moving over the device comprises capturing images, with the first gesture detection sensor, of the control object as it moves over the device.

7. The method of claim 6, wherein determining a direction of movement of the control object comprises:

dividing each captured image into a set of sub-regions,

calculating a region statistic for each sub-region;

determining a displacement between region statistics of a current image and one or more previous images; and

determining the direction of movement of the control object based on the determined displacement.

8. The method of claim 1, wherein the control object performs the touchless gesture.

9. The method of claim 8, wherein the control object comprises a hand of a user.

10. The method of claim 1, wherein the first gesture detection sensor or the second gesture detection sensor comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

11. The method of claim 1, wherein the first gesture detection sensor comprises the second gesture detection sensor.

12. The method of claim 1, wherein performing an action comprises displaying content substantially aligned with the determined direction on the device.

13. A method for performing an action based on a detected touchless gesture, comprising:

detecting, by a gesture detections sensor coupled to or integrated with a device, a first gesture passing over the device;

determining, by one or more processors, if the detected first gesture matches a known gesture in a gesture library or an expected gesture;

translating, by one or more processors, a coordinate system associated with the device based on a direction of the detected first gesture when the detected first gesture does not match a known gesture in a gesture library or an expected gesture; and

performing, by one or more processors, an action based on a second detected gesture when the second detected gesture is determined to match a known gesture in a gesture library or an expected gesture in the translated coordinate system.

14. The method of claim 13, wherein translating a coordinate system comprises translating the coordinate system based on the direction until a reset condition is satisfied.

15. The method of claim 14, wherein a reset condition comprises at least one of a predetermined time before the second gesture is detected and a detected motion of the device.

16. The method of claim 14, further comprising translating the translated coordinate system to the coordinate system associated with the device when a reset condition is satisfied.

17. A device for performing an action, comprising:

a gesture detection sensor configured to;

detect movement of a control object moving over the device and capture information related to the detected movement; and

detect a touchless gesture performed over the device and capture information related to the touchless gesture;

one or more processors configured to;

determine a direction of motion of the control object from the captured information related to the detected movement; and

perform an action based on the captured information related to the information related to the touchless gesture and the determined direction of motion.

18. The device of claim 17, wherein the one or more processors are further configured to translate a coordinate system related to gesture interpretation for the device based on the determined direction of motion.

19. The device of claim 18, wherein the one or more processors are further configured to translate a gesture library to the translated coordinate system.

20. The device of claim 19, wherein the one or more processors are further configured to match the captured information related to the touchless gesture to a translated gesture in the translated gesture library and perform an action corresponding to the matching translated gesture.

21. The device of claim 18, wherein the one or more processors are further configured to translate an expected gesture to the translated coordinate system and match the captured information related to the touchless gesture to the translated expected gesture and perform an action corresponding to the expected gesture.

22. The device of claim 17, wherein the gesture detection sensor is configured to capture images, wherein the one or more processors are configured to determine the direction of motion of the control object by determining a displacement of a region statistic between regions of a current image from the gesture detection sensor and one or more previous images from the gesture detection sensor.

23. The device of claim 17, wherein the control object performs the touchless gesture.

24. The device of claim 23, wherein the control object comprises a hand of a user.

25. The device of claim 17, wherein the gesture detection sensor comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

26. The device of claim 17, wherein the performed action comprises displaying content on the device, the content being substantially aligned with the determined direction.

27. A system for performing an action, comprising:

means for detecting movement of a control object moving over a device;

means for determining a direction of movement of the control object;

means for detecting a touchless gesture performed over the device; and

means for performing an action based on the detected touchless gesture and the determined direction of movement.

28. The system of claim 27, further comprising means for translating a coordinate system related to the device based on the determined direction of movement.

29. The system of claim 28, wherein the means for translating a coordinate system is configured to translate a gesture library based on the determined direction of movement.

30. The system of claim 29, further comprising:

means for matching the touchless gesture to a translated gesture in the translated gesture library, wherein the means for performing an action is configured to perform an action corresponding to a matching translated gesture.

31. The system of claim 28, further comprising:

means for translating the touchless gesture based on the translated coordinate system;

means for matching the translated touchless gesture to an expected gesture, wherein the means for performing an action is configured to perform an action corresponding to the expected gesture.

32. The system of claim 28, wherein the translated coordinate system is oriented differently than the coordinate system was oriented prior to detecting the movement of the control object.

33. The system of claim 27, wherein the control object performs the touchless gesture, and wherein the control object comprises a hand of a user.

34. The system of claim 33, wherein the means for determining the direction of movement of the control object is configured to detect one or more features on the hand and determine the direction based on the detected one or more features.

35. The system of claim 27, wherein:

the means for detecting movement of a control object moving over the device is configured to capture images of the control object as it moves over the device; and

the means for determining a direction of movement of the control object comprises;

means for dividing each captured image into a set of sub-regions,

means for calculating a region statistic for each sub-region;

means for determining a displacement between region statistics of a current image and one or more previous images; and

means for determining the direction of movement of the control object based on the determined displacement.

36. The system of claim 27, wherein the means for detecting movement of a control object moving over the device and the means for detecting a touchless gesture performed over the device comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

37. The system of claim 27, further comprising means for causing content to be displayed in response to the detected movement of the control object, wherein the means for performing the action comprises means for manipulating the displayed content.

38. The system of claim 27, wherein the means for performing an action is configured to display content on the device such that the content is substantially aligned with the determined direction.

39. A computer-readable medium including instructions that, when executed by one or more processors of a device, cause the device to perform a method for performing an action, the method comprising:

detecting motion of a control object moving over the device;

detecting a touchless gesture performed over the device; and

performing an action based on the detected touchless gesture and a direction of the motion.