Gesture recognition

• US 8,526,767 B2
• Filed: 10/20/2008
• Issued: 09/03/2013
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: Atmel Corporation (Microchip Technology Incorporated)
• Original Assignee: Atmel Corporation (Microchip Technology Incorporated)
• Inventors: Bowens, Alan
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US12/254,043
• Publication Number:
  US 20090273571A1
• Time in Patent Office: 1,779 Days
• Field of Search: 382/312, 382/314, 382/315, 348/14.03, 715/700, 715/719, 345/157, 345173-177, 345/181
• US Class Current: 382/315
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0445 using two or more layers of sensing electrodes, e.g. using two layers of electrodes separated by a dielectric layer
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A state machine gesture recognition algorithm for interpreting streams of coordinates received from a touch sensor. The gesture recognition code can be written in a high level language such as C and then compiled and embedded in a microcontroller chip, or CPU chip as desired. The gesture recognition code can be loaded into the same chip that interprets the touch signals from the touch sensor and generates the time series data, e.g. a microcontroller, or other programmable logic device such as a field programmable gate array.

Claims

1. A touch sensor device comprising:

a sensor having a sensitive area extending in at least one-dimension and arranged to output sense signals responsive to proximity of an object to the sensitive area;

a processor operable to execute position-processing logic stored in one or more tangible media, the position-processing logic, when executed by the processor, configured to;

calculate positions of interactions with the sensitive area from an analysis of the sense signals; and

output a times series of data indicative of the interaction positions on the sensor, the interaction positions corresponding to touches; and

a processor operable to execute gesture-processing logic stored in one or more tangible media, the gesture-processing logic, when executed by the processor, configured to analyze the time series of data to distinguish one or more gesture inputs from the time series of data, the gesture-processing logic being coded with gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising;

a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;

a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture; and

a multi-touch state-machine module operable to;

receive, directly from the first one-touch state-machine module, the first output;

receive, directly from the second one-touch state-machine module, the second output; and

recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and

output the recognized multi-touch gesture.

2. The device of claim 1, each of the plurality of state-machine modules including an idle-state module and a plurality of gesture-interpretation-state modules, the idle-state module being entered at the start of operation and being able to be returned to from at least some of the gesture-interpretation-state modules.

3. The device of claim 2, each of the plurality of gesture-interpretation-state modules for each of the one-touch state-machine modules including a touch-state module and the idle-state module passing, responsive to a touch, control to the touch-state module.

4. The device of claim 3, the plurality of gesture-interpretation-state modules for each of the first and second one-touch state-machine modules including a press-state module to which control can pass from a touch-state module if a touch of a duration longer than a threshold duration is sensed in the touch-state module.

5. The device of claim 4, the press-state being a super-state comprising multiple sub-states to distinguish between different durations of press.

6. The device of claim 2, the plurality of gesture-interpretation-state modules including a plurality of state modules operable to recognize motion-related gestures derived from one or more moving touches.

7. The device of claim 2, the plurality of gesture-interpretation-state modules for the multi-touch state-machine module including at least one multi-touch state-module operable to process multiple simultaneous touches based on the first and second outputs from the first and second one-touch state-machine modules, the gesture-processing logic being operable to pass control to the multi-touch state-machine module based on the number of simultaneous touches defined by the time series of data at the time.

8. The device of claim 7, wherein:

the multi-touch state-machine module is for processing two simultaneous touches;

and the device further comprises another multi-touch state-machine module for processing three simultaneous touches.

9. The device of claim 1, the gesture-recognition code being configured to recognize gestures having up to N simultaneous touches, N being at least 2, and comprising N one-touch state-machine modules operable to recognize only one-touch gestures and N−

1 multi-touch state-machine modules each operable to recognize only n-touch gestures, wherein n=2 to N.

10. The device of claim 1, the position-processing logic and the gesture-processing logic being accommodated in, and run on, a single integrated circuit.

11. The device of claim 1, the position-processing logic being accommodated in, and running on, a first integrated circuit and the gesture-processing logic being accommodated in, and running on, one or more separate integrated circuits.

12. A method comprising:

receiving a times series of data indicative of the interaction positions on a sensor, the interaction positions corresponding to touches;

analyzing the time series of data to distinguish one or more gesture inputs from the time series of data using gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising;

a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;

a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture; and

a multi-touch state-machine module operable to;

receive, directly from the first one-touch state-machine module, the first output;

receive, directly from the second one-touch state-machine module, the second output; and

recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and

outputting the recognized multi-touch gesture.

13. A single integrated circuit comprising:

a memory element;

a processor operable to execute logic stored in one or more tangible media, the logic, when executed by the processor, operable to;

receive a times series of data indicative of the interaction positions on a sensor, the interaction positions corresponding to touches;

analyze the time series of data to distinguish one or more gesture inputs from the time series of data, the gesture-processing logic being coded with gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising;

a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;

a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture; and

a multi-touch state-machine module operable to;

receive, directly from the first one-touch state-machine module, the first output;

receive, directly from the second one-touch state-machine module, the second output; and

recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and

output the recognized multi-touch gesture.

14. A computer comprising:

a memory element;

a processor operable to execute logic stored in one or more tangible media, the logic, when executed by the processor, operable to;

receive a times series of data indicative of the interaction positions on a sensor, the interaction positions corresponding to touches;

analyze the time series of data to distinguish one or more gesture inputs from the time series of data using gesture-recognition code comprising a plurality of state-machine modules, the plurality of state-machine modules comprising;

a first one-touch state-machine module, the first one-touch state-machine module being operable to recognize at least a first one-touch gesture and generate a first output based on the first one-touch gesture;

a second one-touch state-machine module, the second one-touch state-machine module being operable to recognize at least a second one-touch gesture and generate a second output based on the second one-touch gesture; and

a multi-touch state-machine module operable to;

receive, directly from the first one-touch state-machine module, the first output;

receive, directly from the second one-touch state-machine module, the second output; and

recognize, based on at least the first and second outputs, at least one multi-touch gesture, the first one-touch state-machine module, the second one-touch state-machine module, and the multi-touch state-machine module being distinct state-machine modules; and

output the recognized multi-touch gesture.