Assistive reading interface

• US 10,339,833 B2
• Filed: 07/21/2017
• Issued: 07/02/2019
• Est. Priority Date: 09/12/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kirschner, Douglas R., Wrem, Nicholas H., Grieves, Jason Anthony, Patel, Amish
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US15/656,825
• Publication Number:
  US 20170323585A1
• Time in Patent Office: 711 Days
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G09B 21/006 using audible presentation of the information

Abstract

A screen reader for a computing device with a touch sensitive user interface. The screen reader operates in an investigatory mode in which touch inputs detected through the user interface are interpreted as inputs designating content on the display to verbally render. Each selection may be interpreted as designating a unit of content at the designated location. The size of the unit may be determined based on a direction of motion of pointing device arriving at the location on the display where the unit of content is designated. In this way, a user may, through simple and intuitively learned commands, designate content in units of different sizes such as characters, words, lines or paragraphs, when the content is text, and/or individual user interface elements when the displayed content being selected includes graphical elements.

Claims

1. A method of operating a computing device to verbally render information on a display, the method comprising:

receiving, user input representing a single contact gesture identifying a desired location on the display and indicating a direction of motion of a pointing device across the display, the motion crossing over content;

determining whether the direction of motion with the user interface is at least one of a horizontal motion or a vertical motion;

mapping, a unit of size for verbally rendering content to at least one of a first size corresponding to at least one of individual words or characters based on determining that the direction of motion corresponds to the horizontal motion associated with the single contact gesture or to a second size corresponding to one or more entire lines of text based on determining that the direction of motion corresponds to the vertical motion associated with the single contact gesture, wherein mapping the unit of size includes selecting content to be rendered according to the first size or the second size based on the direction of motion relative to a threshold angle that differentiates between motions signifying different sized content units; and

verbally rendering information on the display according to the unit of size and based on the direction of motion with the user interface.

2. The method of claim 1, wherein the information is presented on the display as lines of text.

3. The method of claim 1, wherein the first size further comprises individual graphical elements.

4. The method of claim 3, wherein:

the graphical elements comprise graphical user interface control objects; and

verbally rendering information on the display in the unit of size comprises rendering verbally a label associated with the control object.

5. The method of claim 1, further comprising:

monitoring motion of the pointing device across the screen while operating in a first mode in which information is rendered in the first size as the pointing device indicates each unit of information of the first size;

based on the monitored motion, entering a mode in which information is rendered in the second size as the pointing device indicates each unit of information of the second size.

6. The method of claim 1, wherein the display is a touch screen.

7. The method of claim 1, wherein in response to the direction of motion exceeding the threshold angle, determining that the direction of motion corresponds to the vertical motion;

and

in response to the direction of motion being less than or equal to the threshold angle, determining that the direction of motion corresponds to the horizontal motion.

8. A non-transitory computer-readable medium storing instructions executable by at least one processor of a computing device having a display and a sound output, comprising at least one instruction for:

receiving user input representing a single contact gesture identifying a desired location on the display and indicating a direction of motion of a pointing device across the display, the motion crossing over content;

determining whether the direction of motion with the user interface is at least one of a horizontal motion or a vertical motion;

mapping a unit of size for verbally rendering content to at least one of a first size corresponding to at least one of individual words or characters based on determining that the direction of motion corresponds to the horizontal motion associated with the single contact gesture or to a second size corresponding to one or more entire lines of text based on determining that the direction of motion corresponds to the vertical motion associated with the single contact gesture, wherein mapping the unit of size includes selecting content to be rendered according to the first size or the second size based on the direction of motion relative to a threshold angle that differentiates between motions signifying different sized content units; and

verbally rendering information on the display according to the unit of size and based on the direction of motion with the user interface.

9. The non-transitory computer-readable medium of claim 8, further comprising at least one instruction for receiving user input to designate a mode of interaction for investigation of displayed content.

10. The non-transitory computer-readable medium of claim 9, wherein:

the mode of interaction comprises a first mode; and

the computer-readable medium;

further comprises computer executable instructions for operating the computing device is a second mode of interaction in which content is rendered verbally without receiving express user input through the display.

11. The non-transitory computer-readable medium of claim 8, wherein selecting the unit of content comprises selecting a word when in response to the direction of motion corresponds corresponding to the horizontal motion and selecting a line of text when in response to the direction of motion corresponds corresponding to the vertical motion based on a downward motion.

12. The non-transitory computer-readable medium of claim 8, wherein selecting the unit of content comprises selecting a paragraph when in response to the direction of motion corresponds corresponding to the vertical motion based on an upward motion.

13. The non-transitory computer-readable medium of claim 8, wherein selecting the unit of content comprises selecting a textual description of a control element when in response to the direction of motion corresponds corresponding to the horizontal motion and selecting the textual description of the control element in conjunction with a textual description of adjacent displayed information when in response to the direction of motion corresponds corresponding to the vertical motion based on a downward motion.

14. A computing device, comprising:

a touch screen;

a speaker; and

at least one processor configured to;

receive user input representing a single contact gesture identifying a desired location on the display and indicating a direction of motion of a pointing device across the display, the motion crossing over content;

determine whether the direction of motion with the user interface is at least one of a horizontal motion or a vertical motion;

map a unit of size for verbally rendering content to at least one of a first size corresponding to at least one of individual words or characters based on determining that the direction of motion corresponds to the horizontal motion associated with the single contact gesture or to a second size corresponding to one or more entire lines of text based on determining that the direction of motion corresponds to the vertical motion associated with the single contact gesture, wherein mapping the unit of size includes selecting content to be rendered according to the first size or the second size based on the direction of motion relative to a threshold angle that differentiates between motions signifying different sized content units; and

verbally render information on the display according to the unit of size and based on the direction of motion with the user interface.

15. The computing device of claim 14, further comprising a text-to-speech converter, and generating control signals to the speaker comprises providing the selected unit of content to the text-to-speech converter.

16. The computing device of claim 14, wherein the processor is configured to select a paragraph in response to the direction of motion corresponding to the vertical motion based on an upward motion.

17. The computing device of claim 14, wherein:

the computing device further comprises an orientation sensor; and

selecting content comprises selecting a word in response to the direction of motion corresponding to the horizontal motion and selecting a line of text in response to the direction of motion corresponding to the vertical motion based on a downward motion.

18. The computing device of claim 14, wherein, in response to the identified unit of content being a control element, selecting content comprises selecting a textual description of the control element in response to the direction of motion corresponding to the horizontal motion and selecting the textual description of the control element in conjunction with a textual description of adjacent displayed information in response to the direction of motion corresponding to the vertical motion based on a downward motion.

19. The computing device of claim 14, wherein the computing device has a tablet form factor.