CONTINUOUS RECOGNITION OF MULTI-TOUCH GESTURES

• US 20100020025A1
• Filed: 12/23/2008
• Published: 01/28/2010
• Est. Priority Date: 07/25/2008
• Status: Active Grant

• Current Assignee: IntuiLab
• Original Assignee: IntuiLab
• Inventors: LEMORT, Alexandre, VALES, Stephane
• Granted Patent: US 8,390,577 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The present invention extends to methods, systems, and computer program products for processing multi-touch inputs. It provides a method for continuous recognition of multi-touch gestures performed by one or more users on at least one multi-touch input device. The invention in one aspect allows users to start interacting with one multi-gesture and subsequently perform any other multi-touch gestures, if they want it, without needing to lift all their fingers off the multi-touch input device between each multi-touch gesture.

Claims

1. A computer implemented multi-touch (MT) gesture recognition (GR) method comprising the steps of:

Acquiring input data from a multi-touch input device;

Filtering input data and calculating meaningful features for gesture recognition;

Performing a continuous recognition of multi-touch gestures that allows users to start interacting with one multi-gesture and subsequently perform any other multi-touch gestures, if they want it, without needing to lift all their fingers off the multi-touch input device between each multi-touch gesture;

Performing at least one action during the user interaction sequence when a subset of input data is classified as a gesture.

2. The method of claim 1, wherein the said multi-touch input device can be any combination of hardware, software and telecommunications.

3. The method of claim 1, wherein input data is at least a list of contact points on said multi-touch input device.

4. The method of claim 3, wherein a contact point can be a physical object, a finger, or any human body parts.

5. The method of claim 1, wherein the said filtering input data step comprises a spatial segmentation method defined by the steps of:

Finding the underneath application for each contact point,

Associating each contact points within an application to a user interface element of this application,

Reducing the gesture recognition to a small set of gestures which are expected for this user interface element.

6. The method as recited in claim 1, wherein said action consists in controlling a change in the user interface in accordance with the recognized gesture and its related features.

7. The method as recited in claim 1, wherein the said action consists in providing a user feedback about the recognized gesture.

8. The method as recited in claim 1, wherein the said action is the conjunction of a user feedback, a change made to the user interface and an application specific action.

9. The method as recited in claim 1, wherein the said action is a backward compatibility module that interprets the recognized gesture as a command, convert the command to an equivalent sequence of mouse and/or keyboard events (mouse click, mouse move, key pressed, etc.) and sends these events to the underneath application as virtual mouse and/or keyboard events.

17. The method as recited in claim 9, wherein the said action is the conjunction of a user feedback, a change made to the user interface and an application specific action.

10. A computer implemented multi-touch (MT) gesture recognition (GR) method comprising the steps of:

Acquiring input data from at least two multi-touch input devices;

Grouping corresponding separately-generated sets of input data in chronological order so as to provide a single set of input data that can be interpreted in a time-coherent manner;

Filtering said single set of time-aligned input data and calculating meaningful features for gesture recognition;

Performing a continuous recognition of multi-touch gestures that allows users to start interacting with one multi-gesture and subsequently perform any other multi-touch gestures, if they want it, without needing to lift all their fingers off the multi-touch input device between each multi-touch gesture;

Performing at least one action during the user interaction sequence when a subset of input data is classified as a gesture.

11. The method of claim 10, wherein the said multi-touch input device can be any combination of hardware, software and telecommunications.

12. The method of claim 10, wherein input data is at least a list of contact points on said multi-touch input device.

13. The method of claim 12, wherein a contact point can be a physical object, a finger, or any human body parts.

14. The method of claim 10, wherein the said filtering input data step comprises a spatial segmentation method defined by the steps of:

Finding the underneath application for each contact point,

Associating each contact points within an application to a user interface element of this application,

Reducing the gesture recognition to a small set of gestures which are expected for this user interface element.

15. The method as recited in claim 10, wherein said action consists in controlling a change in the user interface in accordance with the recognized gesture and its related features.

16. The method as recited in claim 10, wherein the said action consists in providing a user feedback about the recognized gesture.

18. The method as recited in claim 10, wherein the said action is a backward compatibility module that interprets the recognized gesture as a command, convert the command to an equivalent sequence of mouse and/or keyboard events (mouse click, mouse move, key pressed, etc.) and sends these events to the underneath application as virtual mouse and/or keyboard events.

19. The method as recited in claim 10, wherein the said multi-touch gesture is performed using at least two multi-touch input devices.

20. The method as recited in claim 19, wherein the said multi-touch gesture is performed by at least two users.

21. The method as recited in claim 20, wherein the said multi-touch input devices are remote from each other and operatively connected using a network connection.

22. The method as recited in claim 21, wherein the said application is a remote collaboration tool such as a shared workspace.

23. The method as recited in claim 21, wherein the said application is a video game.