METHODS AND APPARATUS FOR CONTACTLESS GESTURE RECOGNITION

• US 20110310005A1
• Filed: 06/16/2011
• Published: 12/22/2011
• Est. Priority Date: 06/17/2010
• Status: Active Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, An M., Razdan, Ashu, Cheng, Heng-Tze, Buller, Elliot B.
• Application Number:
  US13/161,955
• Publication Number:
  US 20110310005A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/3262 Power saving in digitizer or tablet
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04W 52/0209 in terminal devices terminal devices per se H04W88/02
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

Systems and methods are described for performing contactless gesture recognition for a computing device, such as a mobile computing device. An example technique for managing a gesture-based input mechanism for a computing device described herein includes identifying parameters of the computing device relating to accuracy of gesture classification performed by the gesture-based input mechanism and managing a power consumption level of at least an infrared (IR) light emitting diode (LED) or an IR proximity sensor of the gesture-based input mechanism based on the parameters of the computing device.

Claims

1. A mobile computing device comprising:

a sensor system configured to obtain data relating to three-dimensional user movements, the sensor system comprising an infrared (IR) light emitting diode (LED) and an IR proximity sensor; and

a sensor controller module communicatively coupled to the sensor system and configured to identify properties of the device indicative of clarity of the data relating to the three-dimensional user movements obtained by the sensor system and probability of correct input gesture identification with respect to the three-dimensional user movements and to regulate power consumption of at least one of the IR LED or the IR proximity sensor of the sensor system based on the properties of the device.

2. The device of claim 1 further comprising an ambient light sensor communicatively coupled to the sensor controller module and configured to identify an ambient light level of an area at which the device is located, wherein the sensor controller module is further configured to adjust a power level of the IR LED according to the ambient light level.

3. The device of claim 1 further comprising an activity monitor module communicatively coupled to the sensor controller module and configured to determine a level of user activity with respect to the device, wherein the sensor controller module is further configured to regulate the power consumption of the sensor system according to the level of user activity.

4. The device of claim 3 wherein the sensor controller module is further configured to place the sensor system in a slotted operating mode if the level of user activity is determined to be below a predefined threshold.

5. The device of claim 1 wherein the device comprises at least two front-facing edges, IR LEDs and IR proximity sensors of the sensor system are positioned on at least two of the front-facing edges of the device, the properties of the device comprise orientation of the device, and the sensor controller module is further configured to selectively activate IR LEDs and IR proximity sensors positioned on at least one of the front-facing edges of the device based on the orientation of the device.

6. The device of claim 1 wherein the device further comprises:

at least one front-facing edge; and

one or more apertures positioned along the at least one front-facing edge;

wherein the one or more apertures are covered with an IR transmissive material and one of an IR LED or an IR proximity sensor of the sensor system is positioned behind each of the one or more apertures.

7. The device of claim 1 wherein the sensor system further comprises risers respectively coupled to the IR LED and the IR proximity sensor such that the IR LED and the IR proximity sensor are elevated by the risers.

8. The device of claim 1 further comprising:

a framing module communicatively coupled to the sensor system and configured to partition the data obtained by the sensor system into frame intervals;

a feature extraction module communicatively coupled to the framing module and the sensor system and configured to extract features from the data obtained by the sensor system; and

a gesture recognition module communicatively coupled to the sensor system, the framing module and the feature extraction module and configured to identify input gestures corresponding to respective ones of the frame intervals based on the features extracted from the data obtained by the sensor system.

9. The device of claim 8 wherein the gesture recognition module is further configured to identify the input gestures based on at least one of cross correlation, linear regression or signal statistics.

10. The device of claim 1 wherein the sensor system is configured to obtain the data relating to the three-dimensional user movements with reference to a plurality of moving objects.

11. A method of managing a gesture-based input mechanism for a computing device, the method comprising:

identifying parameters of the computing device relating to accuracy of gesture classification performed by the gesture-based input mechanism; and

managing a power consumption level of at least an infrared (IR) light emitting diode (LED) or an IR proximity sensor of the gesture-based input mechanism based on the parameters of the computing device.

12. The method of claim 11 wherein the identifying comprises identifying an ambient light level of an area associated with the computing device and the managing comprises adjusting a power level of the IR LED according to the ambient light level.

13. The method of claim 11 wherein the identifying comprises determining a level of user interaction with the computing device via the gesture-based input mechanism and the managing comprises:

comparing the level of user interaction to a threshold; and

placing the gesture-based input mechanism in a power saving mode if the level of user interaction is below the threshold.

14. The method of claim 11 wherein the identifying comprises identifying an orientation of the computing device and the managing comprises activating or deactivating the IR LED or the IR proximity sensor based on the orientation of the computing device.

15. The method of claim 11 further comprising:

obtaining sensor data from the gesture-based input mechanism;

partitioning the sensor data in time, thereby obtaining respective frame intervals;

extracting features from the sensor data; and

classifying gestures represented in respective ones of the frame intervals based on the features extracted from the sensor data.

16. The method of claim 15 wherein the classifying comprises classifying the gestures represented in the respective ones of the frame intervals based on at least one of cross correlation, linear regression or signal statistics.

17. The method of claim 15 wherein the obtaining comprises obtaining sensor data relating to a plurality of moving objects.

18. A mobile computing device comprising:

sensor means configured to obtain infrared (IR) light-based proximity sensor data relating to user interaction with the device; and

controller means communicatively coupled to the sensor means and configured to identify properties of the device and to manage power consumption of at least part of the sensor means based on the properties of the device.

19. The device of claim 18 wherein the controller means is further configured to measure an ambient light level at an area associated with the device and to adjust the power consumption of at least part of the sensor means based on the ambient light level.

20. The device of claim 18 wherein the controller means is further configured to determine an extent of the user interaction with the device and to adjust the power consumption of at least part of the sensor means according to the extent of the user interaction with the device.

21. The device of claim 20 wherein the controller means is further configured to power off the sensor means upon determining that no user interaction with the device has been identified by the sensor means within a time interval.

22. The device of claim 20 wherein the controller means is further configured to place the sensor means in a power save operating mode if the extent of the user interaction with the device is below a threshold.

23. The device of claim 18 wherein the sensor means comprises a plurality of sensor elements, and the controller means is further configured to selectively activate one or more of the plurality of sensor elements based on an orientation of the device.

24. The device of claim 18 further comprising gesture means communicatively coupled to the sensor means and configured to classify the proximity sensor data by identifying input gestures represented in the proximity sensor data.

25. A computer program product residing on a non-transitory processor-readable medium and comprising processor-readable instructions configured to cause a processor to:

obtain three-dimensional user movement data from an infrared (IR) proximity sensor associated with a mobile device that measures reflection of light from an IR light emitting diode (LED);

identify properties of the mobile device indicative of accuracy of the three-dimensional user movement data; and

regulate power usage of at least a portion of the IR LEDs and IR proximity sensors based on the properties of the mobile device.

26. The computer program product of claim 25 wherein the parameters of the mobile device comprise an ambient light level at an area associated with the mobile device.

27. The computer program product of claim 25 wherein the parameters of the mobile device comprise a history of user interaction with the mobile device.

28. The computer program product of claim 25 wherein the parameters of the mobile device comprise an orientation of the mobile device.

29. The computer program product of claim 25 wherein the instructions configured to cause the processor to detect the one or more gestures are further configured to cause the processor to:

group the three-dimensional user movement data according to respective frame time intervals;

extract features from the three-dimensional user movement data; and

identify input gestures provided within respective ones of the frame time intervals based on the features extracted from the three-dimensional user movement data.

30. The computer program product of claim 29 wherein the instructions configured to cause the processor to identify input gestures are further configured to cause the processor to identify the input gestures based on at least one of cross correlation, linear regression or signal statistics.