Systems and methods for performing a device action based on a detected gesture

• US 9,020,194 B2
• Filed: 06/14/2013
• Issued: 04/28/2015
• Est. Priority Date: 06/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gupta, Samir K., Keating, Virginia Walker, Hildreth, Evan R.
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US13/918,078
• Publication Number:
  US 20140368422A1
• Time in Patent Office: 683 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Systems and methods for performing an action based on a detected gesture are provided. The systems and methods provided herein may detect a direction of an initial touchless gesture and process subsequent touchless gestures based on the direction of the initial touchless gesture. The systems and methods may translate a coordinate system related to a user device and a gesture library based on the detected direction such that subsequent touchless gestures may be processed based on the detected direction. The systems and methods may allow a user to make a touchless gesture over a device to interact with the device independent of the orientation of the device since the direction of the initial gesture can set the coordinate system or context for subsequent gesture detection.

Claims

1. A method for performing an action, comprising:

detecting, by a first gesture detection sensor coupled to or integrated in a device, motion of a control object moving over the device;

determining, by one or more processors of the device, a direction of movement of the control object;

translating a coordinate system related to the device based on the determined direction of movement;

translating an expected gesture to the translated coordinate system;

detecting, by a second gesture detection sensor, a touchless gesture performed over the device;

matching the detected touchless gesture to the translated expected gesture; and

performing, by one or more processors, an action based on the matching.

2. The method of claim 1, wherein translating a coordinate system comprises translating a gesture library based on the determined direction of movement.

3. The method of claim 2, wherein performing an action comprises:

matching the touchless gesture to a translated gesture in the translated gesture library; and

performing an action corresponding to the matching translated gesture.

4. The method of claim 1, wherein detecting movement of a control object moving over the device comprises capturing images, with the first gesture detection sensor, of the control object as it moves over the device.

5. The method of claim 4, wherein determining a direction of movement of the control object comprises:

dividing each captured image into a set of sub-regions, calculating a region statistic for each sub-region;

determining a displacement between region statistics of a current image and one or more previous images; and

determining the direction of movement of the control object based on the determined displacement.

6. The method of claim 1, wherein the control object performs the touchless gesture.

7. The method of claim 6, wherein the control object comprises a hand of a user.

8. The method of claim 1, wherein the first gesture detection sensor or the second gesture detection sensor comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

9. The method of claim 1, wherein the first gesture detection sensor comprises the second gesture detection sensor.

10. The method of claim 1, wherein performing an action comprises displaying content aligned with the determined direction on the device.

11. A device for performing an action, comprising:

a gesture detection sensor configured to;

detect movement of a control object moving over the device and capture information related to the detected movement; and

detect a touchless gesture performed over the device and capture information related to the touchless gesture;

one or more processors configured to;

determine a direction of motion of the control object from the captured information related to the detected movement;

translate a coordinate system related to gesture interpretation for the device based on the determined direction of motion;

translate an expected gesture to the translated coordinate system;

match the captured information related to the touchless gesture to the translated expected gesture; and

perform an action based on the matching.

12. The device of claim 11, wherein the one or more processors are further configured to translate a gesture library to the translated coordinate system.

13. The device of claim 12, wherein the one or more processors are further configured to match the captured information related to the touchless gesture to a translated gesture in the translated gesture library and perform an action corresponding to the matching translated gesture.

14. The device of claim 11, wherein the gesture detection sensor is configured to capture images, wherein the one or more processors are configured to determine the direction of motion of the control object by determining a displacement of a region statistic between regions of a current image from the gesture detection sensor and one or more previous images from the gesture detection sensor.

15. The device of claim 11, wherein the control object performs the touchless gesture.

16. The device of claim 15, wherein the control object comprises a hand of a user.

17. The device of claim 11, wherein the gesture detection sensor comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

18. The device of claim 11, wherein the performed action comprises displaying content on the device, the content being aligned with the determined direction.

19. A system for performing an action, comprising:

means for detecting movement of a control object moving over a device;

means for determining a direction of movement of the control object;

means for translating a coordinate system related to the device based on the determined direction of movement;

means for translating the touchless gesture based on the translated coordinate system;

means for detecting a touchless gesture performed over the device;

means for matching the translated touchless gesture to an expected gesture; and

means for performing an action based on the matching.

20. The system of claim 19, wherein the means for translating a coordinate system is configured to translate a gesture library based on the determined direction of movement.

21. The system of claim 20, further comprising:

means for matching the touchless gesture to a translated gesture in the translated gesture library, wherein the means for performing an action is configured to perform an action corresponding to a matching translated gesture.

22. The system of claim 19, wherein the translated coordinate system is oriented differently than the coordinate system was oriented prior to detecting the movement of the control object.

23. The system of claim 19, wherein the control object performs the touchless gesture, and wherein the control object comprises a hand of a user.

24. The system of claim 23, wherein the means for determining the direction of movement of the control object is configured to detect one or more features on the hand and determine the direction based on the detected one or more features.

25. The system of claim 19, wherein:

the means for detecting movement of a control object moving over the device is configured to capture images of the control object as it moves over the device; and

the means for determining a direction of movement of the control object comprises;

means for dividing each captured image into a set of sub-regions,

means for calculating a region statistic for each sub-region;

means for determining a displacement between region statistics of a current image and one or more previous images; and

means for determining the direction of movement of the control object based on the determined displacement.

26. The system of claim 19, wherein the means for detecting movement of a control object moving over the device and the means for detecting a touchless gesture performed over the device comprises at least one of a camera, an ultrasound sensor, an infrared sensor, and a shadow detector.

27. The system of claim 19, further comprising means for causing content to be displayed in response to the detected movement of the control object, wherein the means for performing the action comprises means for manipulating the displayed content.

28. The system of claim 19, wherein the means for performing an action is configured to display content on the device such that the content is aligned with the determined direction.

29. A computer-readable medium including instructions that, when executed by one or more processors of a device, cause the device to perform a method for performing an action, the method comprising:

detecting motion of a control object moving over the device;

determine a direction of movement of the control object;

translate a coordinate system related to the device based on the determined direction of movement;

translate the touchless gesture based on the translated coordinate system;

detecting a touchless gesture performed over the device;

match the translated touchless gesture to an expected gesture; and

performing an action based on the detected touchless gesture and a direction of the motion.