Sequential classification recognition of gesture primitives and window-based parameter smoothing for high dimensional touchpad (HDTP) user interfaces

• US 8,754,862 B2
• Filed: 07/11/2011
• Issued: 06/17/2014
• Est. Priority Date: 07/11/2010
• Status: Expired due to Fees

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester F. Ludwig
• Inventors: Zaliva, Vadim
• Primary Examiner(s): Wang, Quan-Zhen
• Assistant Examiner(s): Lin, Chun-Nan
• Application Number:
  US13/180,512
• Publication Number:
  US 20120007821A1
• Time in Patent Office: 1,072 Days
• Field of Search: 345156-184
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method for classification recognition of gestures and gesture primitives in a touch-based user interface. In an implementation the method comprises receiving tactile image data responsive to data generated from user touch of a user touch interface comprising a sensor array. The tactile image data is processed to create a plurality of numerical values responsive to data generated from the user touch interface. These numerical values are applied to a principle component analysis operation to produce a reduced-dimensionality data vector which is applied to a classifier having a plurality of classifier outputs interpretable as probabilities. The classifier outputs provide likelihoods that an execution gesture is from a collection of pre-defined gestures, and a decision test is used to produce a decision output indicating a gesture outcome useful in user interface applications. The arrangement can recognize single finger “6D” actions of roll, pitch, yaw, left-right, forward-back, and variations in applied pressure.

Claims

1. A method for recognition of user gesture primitives in a touch-based user interface using at least one computational processor, the method comprising:

receiving tactile image data responsive to user touch of a user touch interface comprising a tactile sensor array, the user touch comprising a user touch gesture primitives executed over an interval of time;

processing the tactile image data with at least one analysis processing operation to produce a processed data vector comprising a plurality of numerical values responsive to data generated from the tactile image data, the at least one analysis processing operation comprising calculations including deriving first and second moments from the tactile image data and producing the processed data vector based on a centroid calculation on the derived first and second moments; and

further processing the numerical values with a principle component analysis operation to produce a reduced-dimensionality data vector;

applying the reduced-dimensionality data vector to a classifier, the classifier configured to selectively provide a plurality of classifier outputs, the plurality of classifier outputs selectively provided responsive to the reduced-dimensionality data vector, each classifier output representing an associated pre-defined gesture from a set of pre-defined gestures, the classifier further configured to provide an associated probability value for each classifier output;

applying the classifier outputs and the associated probability value for each provided classifier output to a decision test, the decision test producing a decision output,

wherein the plurality of classifier outputs provided by the classifier and the associated probability value for each provided classifier output are responsive to the tactile image data;

wherein the decision output is responsive to the user touch gesture primitives, and

wherein the decision output is used to specify an identified gesture to an external system, the external system comprising software.

2. The method of claim 1 wherein the classifier comprises an artificial neural network.

3. The method of claim 1 wherein the classifier comprises a heuristic.

4. The method of claim 1 wherein the classifier comprises a genetic algorithm.

5. The method of claim 1 wherein the decision output is used as a user interface signal.

6. The method of claim 1 wherein the collection of predefined gestures includes the roll angle change of a single finger on the user touch interface.

7. The method of claim 1 wherein the collection of predefined gestures includes the pitch angle change of a single finger on the user touch interface.

8. The method of claim 1 wherein the collection of pre-defined gestures includes the yaw angle change of a single finger on the user touch interface.

9. The method of claim 1 wherein the collection of pre-defined gestures includes the simultaneous change in the sway (“

x”

) of the position of a single finger on the user touch interface.

10. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise moments calculated from the tactile image data.

11. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a roll angle measurement derived from the tactile image data.

12. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a pitch angle measurement derived from the tactile image data.

13. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a yaw angle measurement derived from the tactile image data.

14. The method of claim 2 wherein the output layer of the artificial neural network comprises a logistics function element.

15. The method of claim 1 wherein the at least one analysis processing operation produces a plurality of processed data vectors during the time used execution of the intended touch gesture.

16. The method of claim 1 wherein the collection of pre-defined gestures includes the simultaneous change in the surge (“

y)”

of the position of a single finger on the user touch interface.