General User Interface Gesture Lexicon and Grammar Frameworks for Multi-Touch, High Dimensional Touch Pad (HDTP), Free-Space Camera, and Other User Interfaces

• US 20120317521A1
• Filed: 03/07/2012
• Published: 12/13/2012
• Est. Priority Date: 03/07/2011
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Lester F. Ludwig
• Inventors: Ludwig, Lester F.
• Granted Patent: US 9,442,652 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/268 Morphological analysis

Abstract

A method for a multi-touch gesture-based user interface wherein a plurality of gestemes are defined as functions of abstract space and time and further being primitive gesture segments that can be concatenated over time and space to construct gestures. Various distinct subset of the gestemes can be concatenated in space and time to construct a distinct gestures. Real-time multi-touch gesture-based information provided by user interface is processed to at least a recognized sequence of specific gestemes and that the sequence of gestemes that the user'"'"'s execution a gesture has been completed. The specific gesture rendered by the user is recognized according to the sequence of gestemes. Many additional features are then provided from this foundation, including gesture grammars, structured-meaning gesture-lexicon, context, and the use of gesture prosody.

Claims

1. A method for a multi-touch gesture-based user interface, the method comprising:

defining a plurality of gestemes, the gestemes being defined as functions of abstract space and time, the gestemes further being primitive gesture segments that can be concatenated over time and space to construct gestures;

concatenating a first subset of the plurality of gestemes which can be concatenated in space and time to construct a first gesture;

concatenating a second subset of the plurality of gestemes which can be concatenated in time and space to construct a second gesture;

receiving real-time multi-touch gesture-based information from a multi-touch gesture-based user interface;

processing the real-time multi-touch gesture-based information to sequentially identify over intervals of real-time at least a recognized sequence of specific gestemes belonging to the plurality of gestemes;

determining from the sequence of gestemes that the user'"'"'s execution a gesture has been completed;

determining from the sequence of gestemes a specific gesture represented by the sequence of gestemes;

wherein the gesture is recognized as either the first gesture or the second gesture according to the sequence of gestemes.

2. The method of claim 1 wherein gesture grammars are provided using the sequence of gestemes.

3. The method of claim 1 wherein structured-meaning gesture-lexicon frameworks are provided using the sequence of gestemes.

4. The method of claim 1 wherein gesture context frameworks are provided using the sequence of gestemes.

5. The method of claim 1 wherein the use of gesture prosody is provided.

6. A method for controlling an application operating on a processor with a multi-touch gesture-based user interface, the method comprising:

receiving a gesture in real time from a high-definition touch pad (HDTP);

deconstructing the gesture into a sequence of gestemes as the gesture is received;

determining from the sequence of gestemes that the gesture has been completed;

recognizing the sequence of gesteme as a particular command to the application; and

transmitting the particular command to the application operating on the processor.

7. The method of claim 6 wherein gesture grammars are provided using the sequence of gestemes.

8. The method of claim 6 wherein structured-meaning gesture-lexicon frameworks are provided using the sequence of gestemes.

9. The method of claim 6 wherein gesture context frameworks are provided using the sequence of gestemes.

10. The method of claim 6 wherein the use of gesture prosody is provided.