Accessibility techniques for presentation of symbolic expressions

• US 10,540,792 B2
• Filed: 01/19/2016
• Issued: 01/21/2020
• Est. Priority Date: 01/25/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Fleizach, Christopher B., Seymour, Eric T., Hughes, Gregory F., Pedersen, Mike
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Shin, Andrew
• Application Number:
  US15/001,028
• Publication Number:
  US 20160148409A1
• Time in Patent Office: 1,463 Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G10L 13/07 Concatenation rules
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

Methods for presenting symbolic expressions such as mathematical, scientific, or chemical expressions, formulas, or equations are performed by a computing device. One method includes: displaying a first portion of a symbolic expression within a first area of a display screen; while in a first state in which the first area is selected for aural presentation, aurally presenting first information related to the first portion of the symbolic expression; while in the first state, detecting particular user input; in response to detecting the particular user input, performing the steps of: transitioning from the first state to a second state in which a second area, of the display, is selected for aural presentation; determining second information associated with a second portion, of the symbolic expression, that is displayed within the second area; in response to determining the second information, aurally presenting the second information.

Claims

1. A method, comprising:

at a computing device with a display and a touch-sensitive surface and one or more audible output devices;

while displaying a symbolic expression including at least one portion having a plurality of sub-portions on the display, detecting a first tap gesture on the touch-sensitive surface directed to the portion of the symbolic expression, wherein the first contact corresponds to a location-dependent navigation input;

in response to detecting the first tap gesture, placing a focus on the portion of the symbolic expression and outputting audible information that corresponds to the portion of the symbolic expression;

detecting a first flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein the first flick gesture corresponds to a location-independent navigation input;

in response to detecting the first flick gesture, moving the focus from the portion of the symbolic expression to a first sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the first sub-portion;

detecting a second flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein second first flick gesture corresponds to a location-independent navigation input; and

in response to detecting the second flick gesture, moving the focus from the first sub-portion of the symbolic expression to a second sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the second sub-portion.

2. The method of claim 1, wherein the audible information corresponding to the portion of the symbolic expression comprises a description of the symbolic expression within the portion.

3. The method of claim 1, wherein the audible information corresponding to the first sub-portion of the symbolic expression comprises a description of the symbolic expression within the first sub-portion and a description of a location of the first sub-portion within the portion.

4. The method of claim 1, wherein outputting the audible information corresponding to the second sub-portion comprises outputting a transition sound prior to outputting the audible information corresponding to the second sub-portion.

5. The method of claim 4, wherein the transition sound is output in response to the moving of the focus from the first sub-portion to the second sub-portion.

6. The method of claim 1, further comprising:

detecting movement of a contact that traces a symbol within the symbolic expression, wherein feedback is provided to a user while the movement of the contact traces the symbol.

7. The method of claim 1, wherein the audible information corresponding to the first sub-portion and the second sub-portion includes additional information about the locations of the first sub-portion and the second sub-portion that is not included in the audible information corresponding to the portion.

8. One or more non-transitory computer-readable media storing one or more programs which, when executed by a computing device with a touch-sensitive surface and one or more audible output devices, cause the computing device to perform steps comprising:

determining portions within a symbolic expression, the symbolic expression including at least one portion having a plurality of sub-portions;

detecting a first tap gesture on the touch-sensitive surface directed to the portion of the symbolic expression, wherein the first contact corresponds to a location-dependent navigation input;

in response to detecting the first tap gesture, placing a focus on the portion of the symbolic expression and outputting audible information that corresponds to the portion of the symbolic expression;

detecting a first flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein the first flick gesture corresponds to a location-independent navigation input;

in response to detecting the first flick gesture, moving the focus from the portion of the symbolic expression to a first sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the first sub-portion;

detecting a second flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein second first flick gesture corresponds to a location-independent navigation input; and

in response to detecting the second flick gesture, moving the focus from the first sub-portion of the symbolic expression to a second sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the second sub-portion.

9. The computer-readable medium of claim 8, wherein the audible information corresponding to the portion of the symbolic expression comprises a description of the symbolic expression within the portion.

10. The computer-readable medium of claim 8, wherein the audible information corresponding to the first sub-portion of the symbolic expression comprises a description of the symbolic expression within the first sub-portion and a description of a location of the first sub-portion within the portion.

11. The computer-readable medium of claim 8, wherein outputting the audible information corresponding to the second sub-portion comprises outputting a transition sound prior to outputting the audible information corresponding to the second sub-portion.

12. The computer-readable medium of claim 11, wherein the transition sound is output in response to the moving of the focus from the first sub-portion to the second sub-portion.

13. The computer-readable medium of claim 8, further comprising:

detecting movement of a contact that traces a symbol within the symbolic expression, wherein feedback is provided to a user while the movement of the contact traces the symbol.

14. The computer-readable medium of claim 8, wherein the audible information corresponding to the first sub-portion and the second sub-portion includes additional information about the locations of the first sub-portion that is not included in the audible information corresponding to the portion.

15. A computing device, comprising:

a touch-sensitive surface;

one or more audible output devices;

one or more processors;

memory; and

one or more programs stored in the memory and which, when executed by the one or more processors, cause the computing device to determine portions within a symbolic expression, the symbolic expression including at least one portion having a plurality of sub-portions;

detect a first tap gesture on the touch-sensitive surface directed to the portion of the symbolic expression, wherein the first contact corresponds to a location-dependent navigation input;

in response to detecting the first tap gesture, place a focus on the portion of the symbolic expression and outputting audible information that corresponds to the portion of the symbolic expression;

detect a first flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein the first flick gesture corresponds to a location-independent navigation input;

in response to detecting the first flick gesture, move the focus from the portion of the symbolic expression to a first sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the first sub-portion;

detect a second flick gesture outside of the symbolic expression on the touch-sensitive surface that corresponds to movement through the portion of the symbolic expression, wherein second first flick gesture corresponds to a location-independent navigation input; and

in response to detecting the second flick gesture, move the focus from the first sub-portion of the symbolic expression to a second sub-portion of the portion of the symbolic expression and outputting audible information corresponding to the second sub-portion.

16. The computing device of claim 15, wherein the audible information corresponding to the portion of the symbolic expression comprises a description of the symbolic expression within the portion.

17. The computing device of claim 15, wherein the audible information corresponding to the first sub-portion of the symbolic expression comprises a description of the symbolic expression within the first sub-portion and a description of a location of the first sub-portion within the portion.

18. The computing device of claim 15, wherein outputting the audible information corresponding to the second sub-portion comprises outputting a transition sound prior to outputting the audible information corresponding to the second sub-portion.

19. The computing device of claim 18, wherein the transition sound is output in response to the moving of the focus from the first sub-portion to the second sub-portion.

20. The computing device of claim 15, further comprising:

the detect movement of a contact that traces a symbol within the symbolic expression, wherein feedback is provided to a user while the movement of the contact traces the symbol.

21. The computing device of claim 15, wherein the audible information corresponding to the first sub-portion and the second sub-portion includes additional information about the locations of the first sub-portion and the second sub-portion that is not included in the audible information corresponding to the portion.