Multi-Touch Marking Menus and Directional Chording Gestures

• US 20110234503A1
• Filed: 03/26/2010
• Published: 09/29/2011
• Est. Priority Date: 03/26/2010
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: FITZMAURICE, George, Lepinski, Gerard Julian, Grossman, Tovi
• Granted Patent: US 9,405,404 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method for interacting with a multi-touch surface to perform an operation. The method includes detecting that at least one finger is in contact with the multi-touch surface, identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface, and performing a first function associated with the operation based on the chord. The method may also include integrating the chords into directional movement gestures and using the gestures to control system functionality.

Claims

1. A method for interacting with a multi-touch surface to perform an operation, the method comprising:

detecting that at least one finger is in contact with the multi-touch surface;

identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface; and

performing a first function associated with the operation based on the chord.

2. The method of claim 1, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.

3. The method of claim 2, wherein the one or more user interface elements comprise a radial marking menu.

4. The method of claim 3, wherein the radial marking menu is offset by a fixed distance from a location at which the at least one finger is in contact with the multi-touch surface.

5. The method of claim 1, wherein the step of identifying the chord comprises:

determining that five fingers are in contact with the multi-touch surface;

identifying, for each of the five fingers, a contact point on the multi-touch surface associated with the finger;

detecting that zero or more of the five fingers have been lifted away from the multi-touch surface; and

identifying the chord associated with the at least one finger based on associating each finger remaining in contact with the multi-touch surface with a corresponding contact point on the multi-touch surface.

6. The method of claim 1, wherein the step of identifying the chord comprises:

capturing one or more images of the at least one finger that is in contact with the multi-touch surface;

determining a bounding box within the one or more images that encloses the at least one finger that is in contact with the multi-touch surface; and

analyzing the bounding box to identify the chord associated with the at least one finger that is in contact with the multi-touch surface.

7. The method of claim 6, wherein the one or more images are captured from beneath the multi-touch surface.

8. The method of claim 1, further comprising the steps of:

detecting a chording gesture caused by the at least one finger sliding along the multi-touch surface; and

performing a second function associated with the operation based on the chording gesture.

9. The method of claim 8, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface, and wherein the step of performing the second function comprises selecting a user interface element associated with the chording gesture.

10. A computer-readable storage medium storing instructions that, when executed by a processor, cause a computing device to interact with a multi-touch surface to perform an operation, by performing the steps of:

detecting that at least one finger is in contact with the multi-touch surface;

identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface; and

performing a first function associated with the operation based on the chord.

11. The computer-readable storage medium of claim 10, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.

12. The computer-readable storage medium of claim 11, wherein the one or more user interface elements comprise a radial marking menu.

13. The computer-readable storage medium of claim 12, wherein the radial marking menu is offset by a fixed distance from a location at which the at least one finger is in contact with the multi-touch surface.

14. The computer-readable storage medium of claim 10, wherein the step of identifying the chord comprises:

determining that five fingers are in contact with the multi-touch surface;

identifying, for each of the five fingers, a contact point on the multi-touch surface associated with the finger;

detecting that zero or more of the five fingers have been lifted away from the multi-touch surface; and

identifying the chord associated with the at least one finger based on associating each finger remaining in contact with the multi-touch surface with a corresponding contact point on the multi-touch surface.

15. The computer-readable storage medium of claim 10, wherein the step of identifying the chord comprises:

capturing one or more images of the at least one finger that is in contact with the multi-touch surface;

determining a bounding box within the one or more images that encloses the at least one finger that is in contact with the multi-touch surface; and

analyzing the bounding box to identify the chord associated with the at least one finger that is in contact with the multi-touch surface.

16. The computer-readable storage medium of claim 15, wherein the one or more images are captured from beneath the multi-touch surface.

17. The computer-readable storage medium of claim 10, further comprising the steps of:

detecting a chording gesture caused by the at least one finger sliding along the multi-touch surface; and

performing a second function associated with the operation based on chording gesture.

18. The computer-readable storage medium of claim 17, wherein the step of performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface, and wherein the step of performing the second function comprises selecting a user interface element associated with the chording gesture.

19. A computer system, comprising:

a processor; and

a memory storing instructions that when executed by the processor cause the computer system to interact with a multi-touch surface to perform an operation, by performing the steps of;

detecting that at least one finger is in contact with the multi-touch surface,

identifying a chord defined by a combination of fingers associated with the at least one finger in contact with the multi-touch surface, and

performing a first function associated with the operation based on the chord.

20. The computer system of claim 19, wherein performing the first function comprises displaying one or more user interface elements associated with the chord on the multi-touch surface.