Gesteme (gesture primitive) recognition for advanced touch user interfaces

• US 10,430,066 B2
• Filed: 12/06/2012
• Issued: 10/01/2019
• Est. Priority Date: 12/06/2011
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Inventors: Ludwig, Lester F., Zaliva, Vadim
• Primary Examiner(s): Nguyen, Chanh D
• Assistant Examiner(s): Truong, Nguyen H
• Application Number:
  US13/707,518
• Publication Number:
  US 20130141375A1
• Time in Patent Office: Days
• Field of Search: 345156-184, 178 1801- 2004
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

This invention relates to signal space architectures for generalized gesture capture. Embodiments of the invention includes a gesture-primitive approach involving families of “gesteme” from which gestures can be constructed, recognized, and modulated via prosody operations. Gestemes can be associated with signals in a signal space. Prosody operations can include temporal execution modulation, shape modulation, and modulations of other aspects of gestures and gestemes. The approaches can be used for advanced touch user interfaces such as High-Dimensional Touch Pad (HDTP) in touchpad and touchscreen forms, video camera hand-gesture user interfaces, eye-tracking user interfaces, etc.

Claims

1. A system for implementing a user interface, the system comprising:

a finger touch sensing arrangement for generating tactile sensing measurements of one or more finger-rendered non-writing touch gestures from a set of non-writing gestures, the tactile sensing measurements in response to tactile input on the user interface without graphically rendering the tactile contact path on a visual screen, each non-writing gesture comprising a sequence of at least two gestemes, each gesteme comprising a primitive element of finger-rendered tactile input comprised by at least one non-writing gesture from the set of gestures; and

a processor for receiving the tactile sensing measurements and executing instructions to;

generate at least one gesteme recognition event responsive to the detection of a gesteme from the tactile sensing measurements,

process a sequence of gesteme recognition events to generate a non-writing gesture recognition event, wherein a non-writing gesture corresponds to a particular temporally-ordered sequence of gestemes, and

generate user interface output signals responsive to the non-writing gesture recognition event.

2. The system of claim 1 wherein each gesteme is comprised of at least one primitive.

3. The system of claim 2, wherein the processor further executes instructions to generate a primitive recognition event responsive to the detection of a primitive.

4. The system of claim 3, wherein the gesteme recognition event is responsive to a sequence of primitive recognition events.

5. The system of claim 4 wherein the primitive is detected by a vector quantization operation acting on a vector of user interface parameter values, the user interface parameter values calculated from tactile sensing measurements.

6. The system of claim 4 wherein the primitive is detected by a vector quantization operation acting on a sequence of user interface parameter values, the user interface parameter values calculated from tactile sensing measurements.

7. The system of claim 4 wherein a plurality of different gestemes comprise at least one specific primitive.

8. The system of claim 1 wherein a plurality of different gestures comprise at least one specific gesteme.

9. The system of claim 1 wherein the gesteme recognition event comprises shape recognition.

10. The system of claim 1 wherein the gesteme recognition event comprises prosody recognition.

11. The system of claim 10 wherein the gesture prosody recognition comprises temporal gesture prosody.

12. The system of claim 10 wherein the gesture prosody recognition comprises gesture shape prosody.

13. The system of claim 10 wherein the gesture prosody comprises modulation of at least one user interface parameter.

14. The system of claim 1 wherein the gesteme recognition event is accompanied by gesteme prosody recognition.

15. The system of claim 14 wherein the gesture prosody recognition comprises temporal gesture prosody.

16. The system of claim 14 wherein the gesture prosody recognition comprises gesture shape prosody.

17. The system of claim 14 wherein the gesture prosody recognition modifies the interpretation of the gesteme recognition event.

18. The system of claim 14 wherein the gesture prosody comprises modulation of at least one user interface parameter.

19. The system of claim 1 wherein the gesture prosody recognition modifies the interpretation of the gesture recognition event.

20. The system of claim 1 wherein gesture prosody is used as part of gesture grammar arrangement.

21. The system of claim 1 wherein the tactile sensing of the tactile input on the user interface comprises multi-dimensional posture detection.

22. The system of claim 1 wherein a prosody of the temporally-ordered sequence of gestemes is used as part of a larger grammatical structure.

23. The system of claim 1 wherein the user interface output signals control a behavior of a software application.

24. The system of claim 1 wherein the user interface output signals control a behavior of an operating system.