3D gesture recognition

• US 9,552,069 B2
• Filed: 07/11/2014
• Issued: 01/24/2017
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: DeJean, Gerald Reuben II, Alanis Reyes, Eden Alejandro, Thai, Trang T., Gilad-Bachrach, Ran, Lymberopoulos, Dimitrios
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US14/329,875
• Publication Number:
  US 20160011668A1
• Time in Patent Office: 928 Days
• Field of Search: 345/173, 345/174
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/046 by electromagnetic means
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The description relates to 3D gesture recognition. One example gesture recognition system can include a gesture detection assembly. The gesture detection assembly can include a sensor cell array and a controller that can send signals at different frequencies to individual sensor cells of the sensor cell array. The example gesture recognition system can also include a gesture recognition component that can determine parameters of an object proximate the sensor cell array from responses of the individual sensor cells to the signals at the different frequencies, and can identify a gesture performed by the object using the parameters.

Claims

1. A system, comprising:

a gesture detection assembly including;

a sensor cell array,

a switching network configured to send signals independently to individual sensor cells of the sensor cell array, and

a controller configured to control the switching network to send the signals at different frequencies to the individual sensor cells of the sensor cell array; and

a gesture recognition component configured to;

determine parameters of an object proximate the sensor cell array from responses of the individual sensor cells to the signals sent at the different frequencies, and

identify a gesture performed by the object using the parameters.

2. The system of claim 1, wherein the individual sensor cells are near-field proximity sensors.

3. The system of claim 1, wherein the object is a human body part and the parameters include measurements of a position and a distance of the human body part relative to the sensor cell array over a duration of time.

4. The system of claim 1, wherein the switching network comprises switches for directing the signals from a single source to any of multiple individual sensors of the sensor cell array.

5. The system of claim 4, wherein the switching network is a multilayer switching network.

6. The system of claim 1, wherein the controller is further configured to multiplex the sensor cell array using the switching network.

7. The system of claim 6, wherein the controller is further configured to multiplex the sensor cell array by sequentially sending a first signal at a first frequency to multiple individual sensor cells before sending a second signal at a second frequency to the sensor cell array.

8. The system of claim 7, wherein the second signal at the second frequency is sent to the multiple individual sensor cells or sent to a different subset of individual sensor cells than the first signal.

9. The system of claim 1, wherein the responses to the signals are created in part due to a water content of the object.

10. A computer readable memory device or storage device storing computer readable instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform acts comprising:

driving first individual sensor cells of a sensor cell array with first signals at a first frequency from a set of available frequencies;

receiving, from the first individual sensor cells, first responses to the first signals sent at the first frequency;

driving second individual sensor cells of the sensor cell array with second signals at a second frequency from the set of available frequencies;

receiving, from the second individual sensor cells, second responses to the second signals sent at the second frequency; and

identifying a gesture from the first and second responses.

11. The computer readable memory device or storage device of claim 10, wherein the first frequency and the second frequency are selected based at least on training data from a device that includes the sensor cell array or from a different device that includes a different sensor cell array, and wherein the acts further comprise repeating the driving and receiving for additional frequencies from the set of available frequencies.

12. The computer readable memory device or storage device of claim 11, wherein the training data are collected for an object at multiple positions relative to the sensor cell array or the different sensor cell array, or wherein at least a portion of the training data is obtained from a user performing the gesture.

13. The computer readable memory device or storage device of claim 10, wherein the identifying the gesture comprises detecting an object that is performing the gesture.

14. The computer readable memory device or storage device of claim 13, wherein the detecting the object comprises determining a distance and a position of the object relative to the sensor cell array.

15. The computer readable memory device or storage device of claim 14, wherein the identifying the gesture further comprises detecting the object multiple times over a duration of time.

16. A system, comprising:

a sensor cell array;

a switching network configured to send signals independently to sensor cells of the sensor cell array; and

a controller configured to;

select a first frequency for a first signal,

identify an individual sensor cell of the sensor cell array to send the first signal at the first frequency,

control the switching network to send the first signal at the first frequency to the individual sensor cell before controlling the switching network to send a second signal at a second frequency to the individual sensor cell or another individual sensor cell of the sensor cell array, and

receive, from the individual sensor cell, a response to the first signal sent at the first frequency.

17. The system of claim 16, further comprising a gesture recognition component configured to identify a gesture using the response and additional responses received from additional sensor cells of the sensor cell array.

18. The system of claim 17, wherein the gesture recognition component is further configured to identify the gesture using the additional responses from the second signal sent at the second frequency to the individual sensor cell and the another individual sensor cell.

19. The system of claim 17, wherein the gesture recognition component is further configured to identify the gesture using a frequency response mapping table.

20. The system of claim 16, wherein the controller is further configured to select the second frequency based at least in part on the response to the first signal sent at the first frequency.