ENVIRONMENTAL GESTURE RECOGNITION

• US 20100188328A1
• Filed: 01/29/2009
• Published: 07/29/2010
• Est. Priority Date: 01/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dodge, Steven, Wang, Zheng
• Granted Patent: US 8,704,767 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A data-holding subsystem. The data-holding subsystem includes instructions stored thereon that when executed by a logic subsystem in communication with the data-holding subsystem: receive one or more signals, determine a sensor type for each signal of the one or more signals, identify a sensor type specific pattern corresponding to a motion gesture in at least one of the signals, and generate a gesture message based on the motion gesture. The gesture message may be usable by an operating system of a computing device that includes the data-holding subsystem to provide a system-wide function usable by one or more application programs of the computing device to provide an application specific function.

Claims

1. A data-holding subsystem including instructions stored thereon that when executed by a logic subsystem in communication with the data-holding subsystem:

receive one or more signals;

determine a sensor type for each signal of the one or more signals;

identify a sensor type specific pattern corresponding to a motion gesture in at least one of the signals; and

generate a gesture message based on the motion gesture, the gesture message being usable by an operating system of a computing device that includes the data-holding subsystem to provide a system-wide function usable by one or more application programs of the computing device to provide an application specific function.

2. The data-holding subsystem of claim 1, wherein the sensor type includes one or more of an accelerometer, a gyroscope, a camera, a light sensor, and a touch sensor.

3. The data-holding subsystem of claim 1, wherein the computing device is a mobile computing device and the one or more signals are generated by environmental sensors integrated into the mobile computing device.

4. The data-holding subsystem of claim 3, wherein the motion gesture includes a rotation of the mobile computing device along a relative X-axis of the mobile computing device and the application specific function includes a scrolling function.

5. The data-holding subsystem of claim 3, wherein the motion gesture includes a rotation of the mobile computing device along a relative Y-axis of the mobile computing device and the application specific function includes a change page function.

6. The data-holding subsystem of claim 3, wherein the motion gesture includes a rotation of the mobile computing device along a relative Z-axis of the mobile computing device and the application specific function includes a display rotation function.

7. The data-holding subsystem of claim 3, wherein the motion gesture includes a translation of the mobile computing device along a relative X-axis of the mobile computing device and the system-wide function includes an application switching function.

8. The data-holding subsystem of claim 3, wherein the motion gesture includes a translation of the mobile computing device along a relative Y-axis of the computing device and the application specific function includes an editing function.

9. The data-holding subsystem of claim 3, wherein the motion gesture includes a translation of the mobile computing device along a relative Z-axis of the mobile computing device and the system-wide function includes a zoom function.

10. A computing device comprising:

an extensible gesture recognition engine configured to;

receive one or more signals, determine a sensor type for each signal of the one or more signals,

monitor each signal for sensor type specific patterns that are associated with the sensor type determined for that signal, the sensor type specific patterns corresponding to different gestures,

for each signal, in response to identification of a sensor type specific pattern, generate a gesture corresponding to that sensor type specific pattern,

determine a confidence level for the gesture of each signal based on heuristics applied to the one or more signals, and

generate a gesture message for the gesture responsive to the confidence level being above a confidence threshold, the gesture message being usable by one or more applications of the computing device to provide an application specific function.

11. The computing device of claim 10, wherein the extensible gesture recognition engine is configured to recognize sensor type specific patterns corresponding to customized gestures defined by a third party software application or a third party hardware device.

12. The computing device of claim 10, wherein the extensible gesture recognition engine is configured to generate a composite gesture responsive to collective recognition of two or more sensor type specific patterns corresponding to the composite gesture.

13. The computing device of claim 10, wherein the extensible gesture recognition engine is configured to check for degradation of the one or more signals responsive to the confidence level being below the confidence threshold, and upon detection of degradation of one or more signals, determine a new confidence level based on heuristics applied to non-degraded signals.

14. The computing device of claim 10, wherein at least one of the one or more signals is received from a sensor of a remote computing device in communication with the computing device.

15. The computing device of claim 10, wherein a plurality of signals is received over a duration, and the extensible gesture recognition engine is configured to determine usage patterns of the plurality of signals learned for the duration, and the confidence level is further based on the usage patterns.

16. The computing device of claim 10, wherein the gesture message is usable by an operating system of the computing device to generate a system-wide function.

17. The computing device of claim 10, wherein an application specific function is mapped to the gesture message via an application programming interface.

18. The computing device of claim 17, wherein the sensor type includes one or more of an accelerometer, a gyroscope, a camera, a light sensor, and a touch sensor.

19. A computing device comprising:

an extensible gesture recognition engine comprising;

a recognition module configured to;

receive one or more signals,

determine a sensor type for each signal of the one or more signals,

monitor each signal for sensor type specific patterns that are associated with the sensor type determined for that signal, the sensor type specific patterns corresponding to different gestures,

for each signal, in response to identification of a sensor type specific pattern, generate a gesture corresponding to that sensor type specific pattern,

determine a confidence level for the gesture of each signal based on heuristics applied to the one or more signals, and

generate a gesture message for the gesture responsive to the confidence level being above a confidence threshold, the gesture message being usable by one or more applications of the computing device to provide an application specific function; and

a customization module configured to;

add a new signal to a list of available sensor signals used for gesture recognition in response to receiving the new signal from a sensor in communication with the extensible gesture recognition engine,

define one or more customized gestures based on one or more signals from the list of available sensor signals, and

adjust a confidence threshold for a selected gesture.

20. The computing device of claim 19, wherein the customization module adds, defines, and adjusts responsive to input from a third party software application or third party hardware device.