Disambiguation of multitouch gesture recognition for 3D interaction

• US 10,782,873 B2
• Filed: 12/20/2016
• Issued: 09/22/2020
• Est. Priority Date: 01/25/1999
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Piemonte, Patrick S., Moore, Bradford A., Chen, Billy P.
• Primary Examiner(s): Tank, Andrew L
• Application Number:
  US15/385,762
• Publication Number:
  US 20170115871A1
• Time in Patent Office: 1,372 Days
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

A multitouch device can interpret and disambiguate different gestures related to manipulating a displayed image of a 3D object, scene, or region. Examples of manipulations include pan, zoom, rotation, and tilt. The device can define a number of manipulation modes, including one or more single-control modes such as a pan mode, a zoom mode, a rotate mode, and/or a tilt mode. The manipulation modes can also include one or more multi-control modes, such as a pan/zoom/rotate mode that allows multiple parameters to be modified simultaneously.

Claims

1. A method, comprising:

at an electronic device with a touch-sensitive surface and a display;

displaying a region of a map on the display;

while displaying the region of the map on the display, detecting a touch input on the touch-sensitive surface that includes movement of one or more contacts on the touch-sensitive surface; and

,

in response to detecting the touch input, determining a response from at least three possible responses to the touch input based on evaluating the touch input against a plurality of criteria;

in accordance with a determination that the touch input meets panning criteria, wherein the panning criteria include a requirement that the touch input include at least a first number of contacts in order for the panning criteria to be met, panning the map in accordance with the movement of the one or more contacts;

in accordance with a determination that the touch input meets tilting criteria, wherein the tilting criteria include a requirement that the touch input include at least a second number of contacts, greater than the first number of contacts, and a requirement that at least the second number of contacts move, on the touch-sensitive surface, within a predefined maximum tolerance, in parallel with a predefined direction on the touch-sensitive surface, in order for the tilting criteria to be met, tilting the map relative to a ground plane of the map in accordance with the movement of the contacts, wherein the tilted map comprises a three-dimensional map and tilting the map includes changing a viewing angle of a three-dimensional object included in the displayed region of the three-dimensional map; and

,

in accordance with a determination that the touch input meets multi-control criteria, wherein the multi-control criteria include a requirement that the touch input include at least the second number of contacts in order for the multi-control criteria to be met, panning and zooming the map in accordance with the movement of the contacts.

2. The method of claim 1, wherein the tilting criteria include a requirement that at least the second number of contacts move at a speed less than a predefined upper limit.

3. The method of claim 2, wherein the tilting criteria include a requirement that at least the second number of contacts are detected in an arrangement along an axis that is different from the predefined direction on the touch-sensitive surface in order for the tilting criteria to be met.

4. The method of claim 1, wherein the multi-control criteria include a requirement that at least the second number of contacts move toward or away from each other in order for the multi-control criteria to be met.

5. The method of claim 1, including:

in accordance with a determination that the touch input meets the panning criteria, panning the map in accordance with the movement of the one or more contacts until liftoff of the one or more contacts is detected;

in accordance with a determination that the touch input meets the tilting criteria, tilting the map in accordance with the movement of the contacts until liftoff of the contacts is detected; and

,

in accordance with a determination that the touch input meets the multi-control criteria, panning and zooming the map in accordance with the movement of the contacts until liftoff of the contacts is detected.

6. The method of claim 1, wherein tilting the map is performed without zooming the map.

7. The method of claim 1, wherein panning the map is performed without tilting the map.

8. The method of claim 1, wherein panning and zooming the map is performed without tilting the map.

9. The method of claim 1, wherein the panning criteria include a requirement that the touch input include at least the first number of contacts at one or more initial map locations in order for the panning criteria to be met;

and

in accordance with a determination that the touch input meets the panning criteria, panning the map in accordance with the movement of the one or more contacts and maintaining the first number of contacts at the one or more initial map locations.

10. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and

configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a region of a map on the display;

while displaying the region of the map on the display, detecting a touch input on the touch-sensitive surface that includes movement of one or more contacts on the touch-sensitive surface; and

,

in response to detecting the touch input, determining a response from at least three possible responses to the touch input based on evaluating the touch input against a plurality of criteria;

in accordance with a determination that the touch input meets panning criteria, wherein the panning criteria include a requirement that the touch input include at least a first number of contacts in order for the panning criteria to be met, panning the map in accordance with the movement of the one or more contacts;

in accordance with a determination that the touch input meets tilting criteria, wherein the tilting criteria include a requirement that the touch input include at least a second number of contacts, greater than the first number of contacts, and a requirement that at least the second number of contacts move, on the touch-sensitive surface, within a predefined maximum tolerance, in parallel with a predefined direction on the touch-sensitive surface, in order for the tilting criteria to be met, tilting the map relative to a ground plane of the map in accordance with the movement of the contacts, wherein the tilted map comprises a three-dimensional map and tilting the map includes changing a viewing angle of a three-dimensional object included in the displayed region of the three-dimensional map; and

,

in accordance with a determination that the touch input meets multi-control criteria, wherein the multi-control criteria include a requirement that the touch input include at least the second number of contacts in order for the multi-control criteria to be met, panning and zooming the map in accordance with the movement of the contacts.

11. The device of claim 10, wherein the tilting criteria include a requirement that at least the second number of contacts move at a speed less than a predefined upper limit.

12. The device of claim 11, wherein the tilting criteria include a requirement that at least the second number of contacts are detected in an arrangement along an axis that is different from the predefined direction on the touch-sensitive surface in order for the tilting criteria to be met.

13. The device of claim 10, wherein the multi-control criteria include a requirement that at least the second number of contacts move toward or away from each other in order for the multi-control criteria to be met.

14. The device of claim 10, including instructions for:

in accordance with a determination that the touch input meets the panning criteria, panning the map in accordance with the movement of the one or more contacts until liftoff of the one or more contacts is detected;

in accordance with a determination that the touch input meets the tilting criteria, tilting the map in accordance with the movement of the contacts until liftoff of the contacts is detected; and

,

in accordance with a determination that the touch input meets the multi-control criteria, panning and zooming the map in accordance with the movement of the contacts until liftoff of the contacts is detected.

15. The device of claim 10, wherein tilting the map is performed without zooming the map.

16. The device of claim 10, wherein panning the map is performed without tilting the map.

17. The device of claim 10, wherein panning and zooming the map is performed without tilting the map.

18. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the electronic device to:

display a region of a map on the display;

while displaying the region of the map on the display, detect a touch input on the touch-sensitive surface that includes movement of one or more contacts on the touch-sensitive surface; and

,

in response to detecting the touch input;

in accordance with a determination that the touch input meets panning criteria, wherein the panning criteria include a requirement that the touch input include at least a first number of contacts in order for the panning criteria to be met, pan the map in accordance with the movement of the one or more contacts;

in accordance with a determination that the touch input meets tilting criteria, wherein the tilting criteria include a requirement that the touch input include at least a second number of contacts, greater than the first number of contacts, and a requirement that at least the second number of contacts move, on the touch-sensitive surface, within a predefined maximum tolerance, in parallel with a predefined direction on the touch-sensitive surface, in order for the tilting criteria to be met, tilt the map relative to a ground plane of the map in accordance with the movement of the contacts, wherein the tilted map comprises a three-dimensional map and tilting the map includes changing a viewing angle of a three-dimensional object included in the displayed region of the three-dimensional map; and

,

in accordance with a determination that the touch input meets multi-control criteria, wherein the multi-control criteria include a requirement that the touch input include at least the second number of contacts in order for the multi-control criteria to be met, pan and zoom the map in accordance with the movement of the contacts.

19. The storage medium of claim 18, wherein the tilting criteria include a requirement that at least the second number of contacts move at a speed less than a predefined upper limit.

20. The storage medium of claim 19, wherein the tilting criteria include a requirement that at least the second number of contacts are detected in an arrangement along an axis that is different from the predefined direction on the touch-sensitive surface in order for the tilting criteria to be met.

21. The storage medium of claim 18, wherein the multi-control criteria include a requirement that at least the second number of contacts move toward or away from each other in order for the multi-control criteria to be met.

22. The storage medium of claim 18, including instructions, which when executed by the electronic device, cause the electronic device to:

in accordance with a determination that the touch input meets the panning criteria, pan the map in accordance with the movement of the one or more contacts until liftoff of the one or more contacts is detected;

in accordance with a determination that the touch input meets the tilting criteria, tilt the map in accordance with the movement of the contacts until liftoff of the contacts is detected; and

,

in accordance with a determination that the touch input meets the multi-control criteria, pan and zoom the map in accordance with the movement of the contacts until liftoff of the contacts is detected.

23. The storage medium of claim 18, wherein tilting the map is performed without zooming the map.

24. The storage medium of claim 18, wherein panning the map is performed without tilting the map.

25. The storage medium of claim 18, wherein panning and zooming the map is performed without tilting the map.