Devices, methods, and graphical user interfaces for providing control of a touch-based user interface absent physical touch capabilities

• US 9,513,799 B2
• Filed: 08/25/2011
• Issued: 12/06/2016
• Est. Priority Date: 06/05/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Fleizach, Christopher Brian, Seymour, Eric Taylor, Craig, James P.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): HUERTAS TORRES, AYESHA E
• Application Number:
  US13/217,654
• Publication Number:
  US 20120306748A1
• Time in Patent Office: 1,930 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An electronic device with a display and a touch-sensitive surface displays, on the display, a first visual indicator that corresponds to a virtual touch. The device receives a first input from an adaptive input device. In response to receiving the first input from the adaptive input device, the device displays a first menu on the display. The first menu includes a virtual touches selection icon. In response to detecting selection of the virtual touches selection icon, a menu of virtual multitouch contacts is displayed.

Claims

1. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, on the display, a user interface;

while displaying the user interface, receiving a request to display options for interacting with the user interface using virtual multitouch contacts;

in response to receiving the request to display the options for interacting with the user interface using virtual multitouch contacts, displaying, on the display a menu of a plurality of virtual multitouch contacts;

after displaying the menu of the plurality of virtual multitouch contacts on the display, receiving a sequence of one or more inputs that includes selection of a respective virtual multitouch contact of the plurality of virtual multitouch contacts; and

,

in response to receiving the sequence of one or more inputs, performing an operation in the user interface that corresponds to the respective virtual multitouch contact.

2. The device of claim 1, wherein the one or more programs include instructions for:

in response to detecting selection of the respective virtual multitouch contact of the plurality of virtual multitouch contacts, displaying one or more visual indicators that correspond to the respective virtual multitouch contact.

3. The device of claim 2, wherein the one or more programs include instructions for:

in response to receiving the sequence of one or more inputs;

moving the one or more visual indicators.

4. The device of claim 1, wherein the one or more programs include instructions for:

displaying a virtual gestures icon;

detecting selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, displaying a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detecting selection of a respective virtual gesture icon in the menu of virtual gestures; and

,

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, performing an action that corresponds to the respective virtual gesture.

5. The device of claim 1, wherein the one or more programs include instructions for:

displaying a virtual gestures icon;

detecting selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, displaying a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detecting selection of a respective virtual gesture icon in the menu of virtual gestures;

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, displaying a number of visual indicators that correspond to the respective virtual gesture icon;

receiving a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, performing an operation in accordance with the respective virtual gesture.

6. The device of claim 1, wherein the one or more programs include instructions for:

displaying a pinch/depinch gesture icon;

detecting selection of the virtual pinch/depinch gesture icon;

in response to detecting selection of the virtual pinch/depinch gesture icon, displaying two visual indicators that correspond to contacts in the virtual pinch/depinch gesture;

receiving a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, performing an operation in accordance with the virtual pinch/depinch gesture.

7. The device of claim 1, wherein the one or more programs include instructions for:

displaying a virtual device icon;

detecting selection of the virtual device icon;

in response to detecting selection of the virtual device icon, displaying a menu of virtual device operations, the menu of virtual device operations including a plurality of icons representing types of virtual device operations;

detecting selection of a respective virtual device operation icon in the menu of virtual device operations; and

,

in response to detecting selection of the respective virtual device operation icon in the menu of virtual device operations, performing an action that corresponds to the respective virtual device operation.

8. The device of claim 1, wherein the one or more programs include instructions for:

displaying a virtual device rotation icon;

detecting selection of the virtual device rotation icon;

in response to detecting selection of the virtual device rotation icon, displaying a menu of virtual device orientations, the menu of virtual device orientations including a plurality of icons representing types of virtual device orientations;

detecting selection of a respective virtual device orientation icon in the menu of virtual device orientations; and

,

in response to detecting selection of the respective virtual device orientation icon in the menu of virtual device orientations, orienting the display in accordance with the respective virtual device orientation.

9. The device of claim 1, wherein the device includes a home button, and the one or more programs include instructions for:

displaying a home button icon;

detecting selection of the home button icon; and

,

in response to detecting selection of the home button icon, performing an action that corresponds to activation of the home button.

10. The device of claim 1, wherein the operation does not correspond to a multitouch contact actually detected on the touch-sensitive surface.

11. The device of claim 1, wherein the sequence of one or more inputs does not include a corresponding multitouch contact actually detected on the touch-sensitive surface.

12. A method, comprising:

at an electronic device with a display and a touch-sensitive surface;

displaying, on the display, a user interface;

while displaying the user interface, receiving a request to display options for interacting with the user interface using virtual multitouch contacts;

in response to receiving the request to display the options for interacting with the user interface using virtual multitouch contacts,

displaying, on the display a menu of a plurality of virtual multitouch contacts;

after displaying the menu of the plurality of virtual multitouch contacts on the display, receiving a sequence of one or more inputs that includes selection of a respective virtual multitouch contact of the plurality of virtual multitouch contacts; and

,

in response to receiving the sequence of one or more inputs, performing an operation in the user interface that corresponds to the respective virtual multitouch contact.

13. The method of claim 12, including:

in response to detecting selection of the respective virtual multitouch contact of the plurality of virtual multitouch contacts, displaying one or more visual indicators that correspond to the respective virtual multitouch contact.

14. The method of claim 13, including:

in response to receiving the sequence of one or more inputs;

moving the one or more visual indicators.

15. The method of claim 12, including:

displaying a virtual gestures icon;

detecting selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, displaying a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detecting selection of a respective virtual gesture icon in the menu of virtual gestures; and

,

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, performing an action that corresponds to the respective virtual gesture.

16. The method of claim 12, including:

detecting selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, displaying a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detecting selection of a respective virtual gesture icon in the menu of virtual gestures;

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, displaying a number of visual indicators that correspond to the respective virtual gesture icon;

receiving a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, performing an operation in accordance with the respective virtual gesture.

17. The method of claim 12, including:

displaying a pinch/depinch gesture icon;

detecting selection of the virtual pinch/depinch gesture icon;

in response to detecting selection of the virtual pinch/depinch gesture icon, displaying two visual indicators that correspond to contacts in the virtual pinch/depinch gesture;

receiving a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, performing an operation in accordance with the virtual pinch/depinch gesture.

18. The method of claim 12, including:

displaying a virtual device icon;

detecting selection of the virtual device icon;

in response to detecting selection of the virtual device icon, displaying a menu of virtual device operations, the menu of virtual device operations including a plurality of icons representing types of virtual device operations;

detecting selection of a respective virtual device operation icon in the menu of virtual device operations; and

,

in response to detecting selection of the respective virtual device operation icon in the menu of virtual device operations, performing an action that corresponds to the respective virtual device operation.

19. The method of claim 12, including:

displaying a virtual device rotation icon;

detecting selection of the virtual device rotation icon;

in response to detecting selection of the virtual device rotation icon, displaying a menu of virtual device orientations, the menu of virtual device orientations including a plurality of icons representing types of virtual device orientations;

detecting selection of a respective virtual device orientation icon in the menu of virtual device orientations; and

,

in response to detecting selection of the respective virtual device orientation icon in the menu of virtual device orientations, orienting the display in accordance with the respective virtual device orientation.

20. The method of claim 12, wherein the device includes a home button, the method including:

displaying a home button icon;

detecting selection of the home button icon; and

,

in response to detecting selection of the home button icon, performing an action that corresponds to activation of the home button.

21. The method of claim 12, wherein the operation does not correspond to a multitouch contact actually detected on the touch-sensitive surface.

22. The method of claim 12, wherein the sequence of one or more inputs does not include a corresponding multitouch contact actually detected on the touch-sensitive surface.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

display, on the display, a user interface;

while displaying the user interface, receive a request to display options for interacting with the user interface using virtual multitouch contacts;

in response to receiving the request to display the options for interacting with the user interface using virtual multitouch contacts, display, on the display, a menu of a plurality of virtual multitouch contacts;

after displaying the menu of the plurality of virtual multitouch contacts on the display, receive a sequence of one or more inputs that includes selection of a respective virtual multitouch contact of the plurality of virtual multitouch contacts; and

,

in response to receiving the sequence of one or more inputs, perform an operation in the user interface that corresponds to the respective virtual multitouch contact.

24. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

in response to detecting selection of the respective virtual multitouch contact of the plurality of virtual multitouch contacts, display one or more visual indicators that correspond to the respective virtual multitouch contact.

25. The computer readable storage medium of claim 24, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

in response to receiving the sequence of one or more inputs;

move the one or more visual indicators.

26. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to for:

display a virtual gestures icon;

detect selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, display a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detect selection of a respective virtual gesture icon in the menu of virtual gestures; and

,

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, perform an action that corresponds to the respective virtual gesture.

27. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

display a virtual gestures icon;

detect selection of the virtual gestures icon;

in response to detecting selection of the virtual gestures icon, display a menu of virtual gestures, the menu of virtual gestures including a plurality of icons representing types of virtual gestures;

detect selection of a respective virtual gesture icon in the menu of virtual gestures;

in response to detecting selection of the respective virtual gesture icon in the menu of virtual gestures, display a number of visual indicators that correspond to the respective virtual gesture icon;

receive a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, perform an operation in accordance with the respective virtual gesture.

28. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

display a pinch/depinch gesture icon;

detect selection of the virtual pinch/depinch gesture icon;

in response to detecting selection of the virtual pinch/depinch gesture icon, display two visual indicators that correspond to contacts in the virtual pinch/depinch gesture;

receive a second input from the adaptive input device; and

,

in response to receiving the second input from the adaptive input device, perform an operation in accordance with the virtual pinch/depinch gesture.

29. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

display a virtual device icon;

detect selection of the virtual device icon;

in response to detecting selection of the virtual device icon, display a menu of virtual device operations, the menu of virtual device operations including a plurality of icons representing types of virtual device operations;

detect selection of a respective virtual device operation icon in the menu of virtual device operations; and

,

in response to detecting selection of the respective virtual device operation icon in the menu of virtual device operations, perform an action that corresponds to the respective virtual device operation.

30. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

display a virtual device rotation icon;

detect selection of the virtual device rotation icon;

in response to detecting selection of the virtual device rotation icon, display a menu of virtual device orientations, the menu of virtual device orientations including a plurality of icons representing types of virtual device orientations;

detect selection of a respective virtual device orientation icon in the menu of virtual device orientations; and

,

in response to detecting selection of the respective virtual device orientation icon in the menu of virtual device orientations, orient the display in accordance with the respective virtual device orientation.

31. The computer readable storage medium of claim 23, including instructions, which when executed by the electronic device with the display and the touch-sensitive surface, cause the device to:

display a home button icon;

detect selection of the home button icon; and

,

in response to detecting selection of the home button icon, perform an action that corresponds to activation of the home button.

32. The computer readable storage medium of claim 23, wherein the operation does not correspond to a multitouch contact actually detected on the touch-sensitive surface.

33. The computer readable storage medium of claim 23, wherein the sequence of one or more inputs does not include a corresponding multitouch contact actually detected on the touch-sensitive surface.