Devices, methods, and graphical user interfaces for providing accessibility using a touch-sensitive surface

• US 8,751,971 B2
• Filed: 08/30/2011
• Issued: 06/10/2014
• Est. Priority Date: 06/05/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Fleizach, Christopher Brian
• Primary Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US13/221,833
• Publication Number:
  US 20120311508A1
• Time in Patent Office: 1,015 Days
• Field of Search: 715/863, 715/810, 715/817
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G10H 1/38 Chord
• CPC Class Codes: G10H 2220/096 using a touch screen touch screen note input, e.g. using a displayed keyboard G10H2220/241; personal digital assistant [PDA] G10H2230/015

Abstract

An electronic device presents a first user interface element of a first type and a second user interface element of a second type. In a sighted mode, the device detects a first interaction with the first user interface element, and performs an operation in accordance with sighted-mode gesture responses for the first user interface element. The device detects a second interaction with the second user interface element, and performs an operation in accordance with sighted-mode gesture responses for the second user interface element. In an accessible mode, the device detects a third interaction with the first user interface element, and performs an operation in accordance with accessible-mode gesture responses for the first user interface element. The device detects a series of interactions with the second user interface element; and, for each interaction, performs an operation in accordance with the sighted-mode gesture responses for the second user interface element.

Claims

1. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

while the electronic device is operating in a sighted mode;

detecting a first interaction with a user interface element of the first accessible-response type;

in response to detecting the first interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the first accessible-response type;

detecting a second interaction with a user interface element of the second accessible-response type; and

,

in response to detecting the second interaction with the user interface element of the second accessible-response type, performing an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type; and

,

while the electronic device is operating in a first accessible mode;

detecting a third interaction with the user interface element of the first accessible-response type;

in response to detecting the third interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more first-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detecting a series of interactions with the user interface element of the second accessible-response type; and

,

in response to detecting each interaction in the series of interactions with the user interface element of the second accessible-response type, performing an operation in accordance with the set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type.

2. The device of claim 1, including instructions for:

prior to detecting the series of interactions with the user interface element of the second accessible-response type, detecting an initial interaction while in the first accessible mode with the user interface element of the second accessible-response type; and

in response to detecting the initial interaction while in the first accessible mode with the user interface element of the second accessible-response type, outputting audible accessibility information associated with the user interface element of the second accessible-response type.

3. The device of claim 1, including instructions for:

while the electronic device is operating in a second accessible mode distinct from the first accessible mode;

detecting a fourth interaction with the user interface element of the first accessible-response type;

in response to detecting the fourth interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detecting a fifth interaction with the user interface element of the second accessible-response type; and

in response to detecting the fifth interaction with the user interface element of the second accessible-response type, performing an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the second accessible-response type.

4. The device of claim 1, including instructions for:

while the electronic device is operating in the first accessible mode, detecting an input; and

,

in response to detecting the input, changing from the first accessible mode to a second accessible mode distinct from the first accessible mode.

5. A method, comprising:

at an electronic device with a display and a touch-sensitive surface;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

while the electronic device is operating in a sighted mode;

detecting a first interaction with a user interface element of the first accessible-response type;

in response to detecting the first interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the first accessible-response type;

detecting a second interaction with a user interface element of the second accessible-response type; and

,

in response to detecting the second interaction with the user interface element of the second accessible-response type, performing an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type; and

,

while the electronic device is operating in a first accessible mode;

detecting a third interaction with the user interface element of the first accessible-response type;

in response to detecting the third interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more first-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detecting a series of interactions with the user interface element of the second accessible-response type; and

,

in response to detecting each interaction in the series of interactions with the user interface element of the second accessible-response type, performing an operation in accordance with the set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type.

6. The method of claim 5, further comprising:

prior to detecting the series of interactions with the user interface element of the second accessible-response type, detecting an initial interaction while in the first accessible mode with the user interface element of the second accessible-response type; and

,

in response to detecting the initial interaction while in the first accessible mode with the user interface element of the second accessible-response type, outputting audible accessibility information associated with the user interface element of the second accessible-response type.

7. The method of claim 5, further comprising:

while the electronic device is operating in a second accessible mode distinct from the first accessible mode;

detecting a fourth interaction with the user interface element of the first accessible-response type;

in response to detecting the fourth interaction with the user interface element of the first accessible-response type, performing an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detecting a fifth interaction with the user interface element of the second accessible-response type; and

,

in response to detecting the fifth interaction with the user interface element of the second accessible-response type, performing an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the second accessible-response type.

8. The method of claim 5, further comprising:

while the electronic device is operating in the first accessible mode, detecting an input; and

,

in response to detecting the input, changing from the first accessible mode to a second accessible mode distinct from the first accessible mode.

9. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

present in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

while the electronic device is operating in a sighted mode;

detect a first interaction with a user interface element of the first accessible-response type;

in response to detecting the first interaction with the user interface element of the first accessible-response type, perform an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the first accessible-response type;

detect a second interaction with a user interface element of the second accessible-response type; and

,

in response to detecting the second interaction with the user interface element of the second accessible-response type, perform an operation in accordance with a set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type; and

,

while the electronic device is operating in a first accessible mode;

detect a third interaction with the user interface element of the first accessible-response type;

in response to detecting the third interaction with the user interface element of the first accessible-response type, perform an operation in accordance with a set of one or more first-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detect a series of interactions with the user interface element of the second accessible-response type; and

,

in response to detecting each interaction in the series of interactions with the user interface element of the second accessible-response type, perform an operation in accordance with the set of one or more sighted-mode gesture responses for the user interface element of the second accessible-response type.

10. The computer readable storage medium of claim 9, wherein the instructions further cause the device to:

prior to detecting the series of interactions with the user interface element of the second accessible-response type, detect an initial interaction while in the first accessible mode with the user interface element of the second accessible-response type; and

,

in response to detecting the initial interaction while in the first accessible mode with the user interface element of the second accessible-response type, output audible accessibility information associated with the user interface element of the second accessible-response type.

11. The computer readable storage medium of claim 9, wherein the instructions further cause the device to:

while the electronic device is operating in a second accessible mode distinct from the first accessible mode;

detect a fourth interaction with the user interface element of the first accessible-response type;

in response to detecting the fourth interaction with the user interface element of the first accessible-response type, perform an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the first accessible-response type;

detect a fifth interaction with the user interface element of the second accessible-response type; and

,

in response to detecting the fifth interaction with the user interface element of the second accessible-response type, perform an operation in accordance with a set of one or more second-accessible-mode gesture responses for the user interface element of the second accessible-response type.

12. The computer readable storage medium of claim 9, wherein the instructions further cause the device to:

while the electronic device is operating in the first accessible mode, detect an input; and

,

in response to detecting the input, change from the first accessible mode to a second accessible mode distinct from the first accessible mode.

13. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, wherein;

for a respective user interface element of the first accessible-response type;

when the electronic device is operating in a sighted mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the first accessible-response type, and

when the electronic device is operating in a first accessible mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for a respective user interface element of the second accessible-response type;

when the electronic device is operating in the sighted mode, responding to detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type, and

when the electronic device is operating in the first accessible mode, responding to each interaction in a series of detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with the respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

14. The device of claim 13, wherein the presenting includes:

presenting audible accessibility information in accordance with the respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

presenting standard audible information in accordance with at least one of the one or more respective sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

15. The device of claim 13, including instructions for:

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the first accessible mode, prior to detecting the series of interactions with the respective user interface element of the second accessible-response type, responding to detecting an initial interaction while in the first accessible mode with the respective user interface element of the second accessible-response type by outputting audible accessibility information associated with the respective user interface element of the second accessible-response type.

16. The device of claim 13, including instructions for:

for the respective user interface element of the first accessible-response type;

when the electronic device is operating in a second accessible mode distinct from the first accessible mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the second accessible mode, responding to detected interactions with the respective user interface element of the second accessible-response type in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the second accessible-response type.

17. The device of claim 13, including instructions for:

while the electronic device is operating in the first accessible mode, detecting an input; and

,

in response to detecting the input, changing from the first accessible mode to a second accessible mode distinct from the first accessible mode.

18. A method, comprising:

at an electronic device with a display and a touch-sensitive surface;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, wherein;

for a respective user interface element of the first accessible-response type;

when the electronic device is operating in a sighted mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the first accessible-response type, and

when the electronic device is operating in a first accessible mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for a respective user interface element of the second accessible-response type;

when the electronic device is operating in the sighted mode, responding to detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type, and

when the electronic device is operating in the first accessible mode, responding to each interaction in a series of detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with the respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

19. The method of claim 18, wherein the presenting includes:

presenting audible accessibility information in accordance with the respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

presenting standard audible information in accordance with at least one of the one or more respective sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

20. The method of claim 18, further comprising:

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the first accessible mode, prior to detecting the series of interactions with the respective user interface element of the second accessible-response type, responding to detecting an initial interaction while in the first accessible mode with the respective user interface element of the second accessible-response type by outputting audible accessibility information associated with the respective user interface element of the second accessible-response type.

21. The method of claim 18, further comprising:

for the respective user interface element of the first accessible-response type;

when the electronic device is operating in a second accessible mode distinct from the first accessible mode, responding to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the second accessible mode, responding to detected interactions with the respective user interface element of the second accessible-response type in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the second accessible-response type.

22. The method of claim 18, further comprising:

while the electronic device is operating in the first accessible mode, detecting an input; and

,

in response to detecting the input, changing from the first accessible mode to a second accessible mode distinct from the first accessible mode.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

present in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, wherein;

for a respective user interface element of the first accessible-response type;

when the electronic device is operating in a sighted mode, the device responds to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the first accessible-response type, and

when the electronic device is operating in a first accessible mode, the device responds to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for a respective user interface element of the second accessible-response type;

when the electronic device is operating in the sighted mode, the device responds to detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with a respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type, and

when the electronic device is operating in the first accessible mode, the device responds to each interaction in a series of detected interactions with the respective user interface element of the second accessible-response type by performing operations in accordance with the respective set of one or more sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

24. The computer readable storage medium of claim 23, wherein the instructions further cause the device to:

present audible accessibility information in accordance with the respective set of one or more first-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

present standard audible information in accordance with at least one of the one or more respective sighted-mode gesture responses for the respective user interface element of the second accessible-response type.

25. The computer readable storage medium of claim 23, wherein the instructions further cause the device to:

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the first accessible mode, prior to detecting the series of interactions with the respective user interface element of the second accessible-response type, respond to detecting an initial interaction while in the first accessible mode with the respective user interface element of the second accessible-response type by outputting audible accessibility information associated with the respective user interface element of the second accessible-response type.

26. The computer readable storage medium of claim 23, wherein the instructions further cause the device to:

for the respective user interface element of the first accessible-response type;

when the electronic device is operating in a second accessible mode distinct from the first accessible mode, respond to detected interactions with the respective user interface element of the first accessible-response type by performing operations in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the first accessible-response type; and

,

for the respective user interface element of the second accessible-response type;

when the electronic device is operating in the second accessible mode, respond to detected interactions with the respective user interface element of the second accessible-response type in accordance with a respective set of one or more second-accessible-mode gesture responses for the respective user interface element of the second accessible-response type.

27. The computer readable storage medium of claim 23, wherein the instructions further cause the device to:

while the electronic device is operating in the first accessible mode, detect an input; and

,

in response to detecting the input, change from the first accessible mode to a second accessible mode distinct from the first accessible mode.

28. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

detecting a first finger input on the touch-sensitive surface;

in response to detecting the first finger input;

placing a current focus on a user interface element of a first accessible-response type, and

outputting audible accessibility information associated with the user interface element of a first accessible-response type;

detecting a second finger input on the touch-sensitive surface;

in response to detecting the second finger input;

changing the current focus from the user interface element of a first accessible-response type to a set of one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, and

outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type;

after outputting the audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type, detecting a series of finger inputs on the touch-sensitive surface; and

,

in response to detecting the series of finger inputs;

when the application is in a first accessibility mode;

executing a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

foregoing outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type; and

when the application is in a second accessibility mode distinct from the first accessibility mode;

executing a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type.

29. The device of claim 28, wherein the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the first accessibility mode is distinct from the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the second accessibility mode.

30. A method, comprising:

at an electronic device with a display and a touch-sensitive surface;

presenting in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

detecting a first finger input on the touch-sensitive surface;

in response to detecting the first finger input;

placing a current focus on a user interface element of a first accessible-response type, and

outputting audible accessibility information associated with the user interface element of a first accessible-response type;

detecting a second finger input on the touch-sensitive surface;

in response to detecting the second finger input;

changing the current focus from the user interface element of a first accessible-response type to a set of one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, and

outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type;

after outputting the audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type, detecting a series of finger inputs on the touch-sensitive surface; and

,

in response to detecting the series of finger inputs;

when the application is in a first accessibility mode;

executing a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

foregoing outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type; and

when the application is in a second accessibility mode distinct from the first accessibility mode;

executing a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type.

31. The method of claim 30, wherein the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the first accessibility mode is distinct from the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the second accessibility mode.

32. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

present in an application one or more user interface elements of a first accessible-response type and one or more user interface elements of a second accessible-response type distinct from the first accessible-response type;

detect a first finger input on the touch-sensitive surface;

in response to detecting the first finger input;

place a current focus on a user interface element of a first accessible-response type, and

output audible accessibility information associated with the user interface element of a first accessible-response type;

detect a second finger input on the touch-sensitive surface;

in response to detecting the second finger input;

change the current focus from the user interface element of a first accessible-response type to a set of one or more user interface elements of a second accessible-response type distinct from the first accessible-response type, and

output audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type;

after outputting the audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type, detect a series of finger inputs on the touch-sensitive surface; and

,

in response to detecting the series of finger inputs;

when the application is in a first accessibility mode;

execute a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

forego outputting audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type; and

when the application is in a second accessibility mode distinct from the first accessibility mode;

execute a corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type, and

output audible accessibility information associated with the set of one or more user interface elements of the second accessible-response type.

33. The computer readable storage medium of claim 32, wherein the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the first accessibility mode is distinct from the corresponding series of actions associated with the set of one or more user interface elements of the second accessible-response type in the second accessibility mode.