Narration of unfocused user interface controls using data retrieval event

• US 10,268,446 B2
• Filed: 02/19/2013
• Issued: 04/23/2019
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Canitz, James Andrew
• Primary Examiner(s): Pham, Linh K
• Application Number:
  US13/769,825
• Publication Number:
  US 20140237369A1
• Time in Patent Office: 2,254 Days
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

In one embodiment, a user interface narrator may repurpose a data retrieval event 232 to narrate user interface items 304 not under input control focus 306. The user interface narrator may use a narration control 212 of an operating system 210 to vocalize a user interface 222. The user interface narrator may execute an application task outside of an input control focus 306. The user interface narrator may receive a repurposed data retrieval event 232 for an application binary interface 230 of the operating system 210 to alert a narration control 212 of a task event for the application task.

Claims

1. A machine-implemented method, comprising:

vocalizing, by a narration control of an operating system, a user interface item selected as an input control focus by a user, the input control focus corresponding to a displayed element of a graphical user interface prepared to receive one or more user selections, wherein the user interface item corresponds to an application operating in a foreground, and wherein the narration control converts text from an application module to an audio format to be played for the user;

executing and updating, by the application module while the user interface item operating in the foreground is receiving user input, an application task operating in a background outside of the input control focus without user input, wherein the application task differs from a foreground task associated with the input control focus corresponding to the application operating in the foreground;

receiving a data retrieval event for an application binary interface of the operating system that alerts the operating system that content associated with the application task operating in the background is available for narration, wherein the data retrieval event is repurposed to cause narration of an unfocused user interface item, the unfocused user interface item corresponding to a second user interface item without the input control focus; and

reading text content from the data retrieval event to vocalize the unfocused user interface item to the user, wherein the unfocused user interface item is vocalized automatically while not under the input control focus.

2. The method of claim 1, wherein the data retrieval event is created using an extensible application markup language.

3. The method of claim 1, further comprising:

presenting a virtual navigation option to the user.

4. The method of claim 1, further comprising:

narrating an input control focus change.

5. The method of claim 1, further comprising:

narrating a keyboard focus change.

6. The method of claim 1, further comprising:

alerting the user to an error notification.

7. The method of claim 1, further comprising:

alerting the user of a loss of service to receive a credential reauthorization.

8. The method of claim 1, further comprising:

describing a dialogue box to the user.

9. A machine-readable storage device having a set of instructions detailing a method stored thereon that when executed by one or more processors cause the one or more processors to perform the method, the method comprising:

vocalizing, by a narration control of an operating system, a user interface item selected as an input control focus by a user, the input control focus corresponding to a displayed element of a graphical user interface prepared to receive one or more user selections, wherein the user interface item corresponds to an application operating in a foreground, and wherein the narration control converts text from an application module to an audio format to be played for the user;

executing and updating, by the application module while the user interface item operating in the foreground is receiving user input, an application task operating in a background outside of the input control focus without user input, wherein the application task differs from a foreground task associated with the input control focus corresponding to the application operating in the foreground;

repurposing with the application module a data retrieval event for an application binary interface of the operating system that alerts the operating system that content associated with the application task operating in the background is available for narration, wherein the data retrieval event is repurposed to narration of an unfocused user interface item, the unfocused user interface item corresponding to a second user interface item without the input control focus; and

producing text content for the data retrieval event to vocalize the unfocused user interface item to the user, wherein the unfocused user interface item is vocalized automatically while not under the input control focus.

10. The machine-readable storage device of claim 9, wherein the method further comprises:

detecting the task event for the application task.

11. The machine-readable storage device of claim 9, wherein the method further comprises:

determining a virtual navigation option for the user.

12. The machine-readable storage device of claim 9, wherein the method further comprises:

detecting an error event.

13. The machine-readable storage device of claim 9, wherein the method further comprises:

detecting a loss of service.

14. The machine-readable storage device of claim 9, wherein the method further comprises:

producing a dialogue box for the user.

15. The machine-readable storage device of claim 14, wherein the method further comprises:

cataloguing a box content of the dialogue box.

16. The machine-readable storage device of claim 9, wherein the method further comprises:

using extensible application markup language to create the data retrieval event.

17. A user interface narrator, comprising:

a memory configured to store a user interface having a user interface item;

a processor configured to execute;

an operating system having a narration control to vocalize the user interface item selected as an input control focus by a user, the input control focus corresponding to a displayed element of a graphical user interface prepared to receive one or more user selections, wherein the user interface item corresponds to an application operating in a foreground, and wherein the narration control converts text from an application module to an audio format to be played for the user; and

an application module, operating in a background outside of the input control focus, to execute and update an application task while the user interface item operating in the foreground is receiving user input without user input, wherein the application task differs from a foreground task associated with the input control focus corresponding to the application operating in the foreground, and to repurpose a data retrieval event for an application binary interface of the operating system to alert the operating system that content associated with the application task operating in the background is available for narration, wherein the data retrieval event is repurposed to cause narration of an unfocused user interface item, the unfocused user interface item corresponding to a second user interface item without the input control focus; and

an audio output configured to vocalize the unfocused user interface item from the data retrieval event, wherein the unfocused user interface item is vocalized automatically while not under the input control focus.

18. The user interface narrator of claim 17, wherein the narration control reads text content from the repurposed data retrieval event.

19. The user interface narrator of claim 17, wherein the narration control presents a virtual navigation option to the user.

20. The user interface narrator of claim 17, wherein the narration control describes a dialogue box to the user.