SEQUENTIAL CLASSIFICATION RECOGNITION OF GESTURE PRIMITIVES AND WINDOW-BASED PARAMETER SMOOTHING FOR HIGH DIMENSIONAL TOUCHPAD (HDTP) USER INTERFACES

• US 20120007821A1
• Filed: 07/11/2011
• Published: 01/12/2012
• Est. Priority Date: 07/11/2010
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester F. Ludwig
• Inventors: ZALIVA, Vadim
• Granted Patent: US 8,754,862 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method for classification recognition of gestures and gesture primitives in a touch-based user interface. In an implementation the method comprises receiving tactile image data responsive to data generated from user touch of a user touch interface comprising a sensor array. The tactile image data is processed to create a plurality of numerical values responsive to data generated from the user touch interface. These numerical values are applied to a principle component analysis operation to produce a reduced-dimensionality data vector which is applied to a classifier having a plurality of classifier outputs interpretable as probabilities. The classifier outputs provide likelihoods that an execution gesture is from a collection of pre-defined gestures, and a decision test is used to produce a decision output indicating a gesture outcome useful in user interface applications. The arrangement can recognize single finger “6D” actions of roll, pitch, yaw, left-right, forward-back, and variations in applied pressure.

Claims

1. A method for classification recognition of gesture primitives in a touch-based user interface using at least one computational processor, the method comprising:

receiving tactile image data responsive to data generated from user touch of a user touch interface comprising a sensor array;

processing the tactile image data with a series of operations to produce a first processed data vector, the series of operations comprising a plurality of numerical values responsive to data generated from the user touch interface; and

further processing the numerical values with a principle component analysis operation to produce a reduced-dimensionality data vector;

applying the reduced-dimensionality data vector to a classifier configured to have a plurality of classifier outputs that can be interpreted as probabilities, the classifier further configured to associate the likelihood of user touch comprising the execution of a gesture over an interval of time with a at least one pre-defined gesture from a collection of pre-defined gestures;

applying the classifier outputs to a decision test, the decision test producing a decision output

wherein the decision output is associated with user touch as represented by the numerical values responsive to data generated from the user touch interface over the interval of time, and

wherein decision output is used to specify a gesture outcome.

2. The method of claim 1 wherein the classifier comprises an artificial neural network.

3. The method of claim 1 wherein the classifier comprises a heuristic.

4. The method of claim 1 wherein the classifier comprises a genetic algorithm.

5. The method of claim 1 wherein the decision test comprises a “

One-of-N with Confidence Thresholds”

test.

6. The method of claim 1 wherein the decision test comprises a “

Winner-Takes-All”

test.

7. The method of claim 1 wherein the decision output is used as a user interface signal.

8. The method of claim 1 wherein the collection of pre-defined gestures includes the roll angle change of a single finger on the user touch interface.

9. The method of claim 1 wherein the collection of pre-defined gestures includes the pitch angle change of a single finger on the user touch interface.

10. The method of claim 1 wherein the collection of pre-defined gestures includes the yaw angle change of a single finger on the user touch interface.

11. The method of claim 1 wherein the collection of pre-defined gestures includes the simultaneous change in the sway (“

x”

) and surge (“

y)”

of the position of a single finger on the user touch interface.

12. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise moments calculated from the tactile image data.

13. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a roll angle measurement derived from the tactile image data.

14. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a pitch angle measurement derived from the tactile image data.

15. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a yaw angle measurement derived from the tactile image data.

16. The method of claim 2 wherein the output layer of the artificial neural network comprises a logistics function element.