DISAMBIGUATION OF MULTITOUCH GESTURE RECOGNITION FOR 3D INTERACTION

• US 20140019917A1
• Filed: 07/15/2013
• Published: 01/16/2014
• Est. Priority Date: 01/25/1999
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Piemonte, Patrick S., Moore, Bradford A., Chen, Billy P.
• Granted Patent: US 9,529,440 B2
• Time in Patent Office: Days
• US Class Current: 715/848
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

A multitouch device can interpret and disambiguate different gestures related to manipulating a displayed image of a 3D object, scene, or region. Examples of manipulations include pan, zoom, rotation, and tilt. The device can define a number of manipulation modes, including one or more single-control modes such as a pan mode, a zoom mode, a rotate mode, and/or a tilt mode. The manipulation modes can also include one or more multi-control modes, such as a pan/zoom/rotate mode that allows multiple parameters to be modified simultaneously.

Claims

1. A method for operating an electronic device, the method comprising:

displaying an image of a 3D region;

detecting an initial gesture on a touch-sensitive surface associated with the electronic device, the initial gesture having characteristics including one or more contact areas and an initial motion of at least one of the one or more contact areas;

selecting, based on one or more of the characteristics of the initial gesture, a manipulation mode, wherein the manipulation mode is selected from a plurality of modes including at least one single-control mode and at least one multi-control mode; and

modifying the image of the 3D region based on the detected initial gesture and the selected manipulation mode.

2. The method of claim 1 further comprising:

detecting a continuation of the initial gesture; and

further modifying the image of the 3D region based on the detected continuation of the initial gesture and the selected manipulation mode.

3. The method of claim 2, wherein detecting the continuation and further modifying the image are repeated until a gesture termination event is detected.

4. The method of claim 3, wherein the gesture termination event is detected in the event that the number of contact areas detected by the touch-sensitive surface decreases to zero.

5. The method of claim 3, wherein the gesture termination event is detected in the event that all of the contact areas in contact with the touch-sensitive surface stop moving for at least a threshold period of time.

6. The method of claim 1, wherein the plurality of modes includes:

a single-control tilt mode wherein the image is modified by modifying a tilt angle of the image relative to a base plane;

a single-control pan mode wherein the image is modified by laterally shifting a field of view across a viewing plane;

a single-control zoom mode wherein the image is modified by enlarging or shrinking the image;

a single-control rotate mode wherein the image is modified by rotating the field of view in the viewing plane; and

a multi-control pan-zoom-rotate mode wherein the image is modifiable by a combination of the modifications associated with the tilt mode, the pan mode, and the zoom mode.

7. The method of claim 1, wherein selecting a manipulation mode includes applying a heuristic to the one or more characteristics of the initial gesture.

8. The method of claim 1, wherein the image of the 3D region comprises a map.

9. The method of claim 1, wherein the image of the 3D region comprises a CAD object.

10. A computer-implemented system, comprising:

one or more data processors; and

one or more non-transitory computer-readable storage media including instructions configured to cause the one or more data processors to perform operations including;

displaying an image of a 3D region;

detecting an initial gesture on a touch-sensitive surface associated with the system, the initial gesture having characteristics including one or more contact areas and an initial motion of at least one of the one or more contact areas;

selecting, based on one or more of the characteristics of the initial gesture, a manipulation mode, wherein the manipulation mode is selected from a plurality of modes including at least one single-control mode and at least one multi-control mode; and

modifying the image of the 3D region based on the detected initial gesture and the selected manipulation mode.

11. The system of claim 10, further comprising:

detecting a continuation of the initial gesture; and

further modifying the image of the 3D region based on the detected continuation of the initial gesture and the selected manipulation mode.

12. The system of claim 11, wherein detecting the continuation and further modifying the image are repeated until a gesture termination event is detected.

13. The system of claim 12, wherein the gesture termination event is detected in the event that the number of contact areas detected by the touch-sensitive surface decreases to zero.

14. The system of claim 12, wherein the gesture termination event is detected in the event that all of the contact areas in contact with the touch-sensitive surface stop moving for at least a threshold period of time.

15. The system of claim 10, wherein the plurality of modes includes:

a single-control tilt mode wherein the image is modified by modifying a tilt angle of the image relative to a base plane;

a single-control pan mode wherein the image is modified by laterally shifting a field of view across a viewing plane;

a single-control zoom mode wherein the image is modified by enlarging or shrinking the image;

a single-control rotate mode wherein the image is modified by rotating the field of view in the viewing plane; and

a multi-control pan-zoom-rotate mode wherein the image is modifiable by a combination of the modifications associated with the tilt mode, the pan mode, and the zoom mode.

16. The system of claim 10, wherein selecting a manipulation mode includes applying a heuristic to the one or more characteristics of the initial gesture.

17. A computer-program product tangibly embodied in a non-transitory machine readable storage medium, including instructions configured to cause a data processing apparatus to:

display an image of a 3D region;

detect an initial gesture on a touch-sensitive surface associated with an electronic device, the initial gesture having characteristics including one or more contact areas and an initial motion of at least one of the one or more contact areas;

select, based on one or more of the characteristics of the initial gesture, a manipulation mode, wherein the manipulation mode is selected from a plurality of modes including at least one single-control mode and at least one multi-control mode; and

modify the image of the 3D region based on the detected initial gesture and the selected manipulation mode.

18. The computer-program product of claim 17, further comprising:

detecting a continuation of the initial gesture; and

further modifying the image of the 3D region based on the detected continuation of the initial gesture and the selected manipulation mode.

19. The computer-program product of claim 18, wherein detecting the continuation and further modifying the image are repeated until a gesture termination event is detected.

20. The computer-program product of claim 19, wherein the gesture termination event is detected in the event that the number of contact areas detected by the touch-sensitive surface decreases to zero.

21. The computer-program product of claim 20, wherein the gesture termination event is detected in the event that all of the contact areas in contact with the touch-sensitive surface stop moving for at least a threshold period of time.

22. The computer-program product of claim 17, wherein the plurality of modes includes:

a single-control tilt mode wherein the image is modified by modifying a tilt angle of the image relative to a base plane;

a single-control pan mode wherein the image is modified by laterally shifting a field of view across a viewing plane;

a single-control zoom mode wherein the image is modified by enlarging or shrinking the image;

a single-control rotate mode wherein the image is modified by rotating the field of view in the viewing plane; and

a multi-control pan-zoom-rotate mode wherein the image is modifiable by a combination of the modifications associated with the tilt mode, the pan mode, and the zoom mode.

23. The computer-program product of claim 17, wherein selecting a manipulation mode includes applying a heuristic to the one or more characteristics of the initial gesture.