Multi-tiered voice feedback in an electronic device

• US 8,768,702 B2
• Filed: 09/05/2008
• Issued: 07/01/2014
• Est. Priority Date: 09/05/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Mason, James Eric, Boettcher, Jesse
• Primary Examiner(s): Abebe, Daniel D
• Application Number:
  US12/205,780
• Publication Number:
  US 20100063818A1
• Time in Patent Office: 2,125 Days
• Field of Search: 704/258, 704/260, 345/177
• US Class Current: 704/258
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons

Abstract

This invention is directed to providing voice feedback to a user of an electronic device. Because each electronic device display may include several speakable elements (i.e., elements for which voice feedback is provided), the elements may be ordered. To do so, the electronic device may associate a tier with the display of each speakable element. The electronic device may then provide voice feedback for displayed speakable elements based on the associated tier. To reduce the complexity in designing the voice feedback system, the voice feedback features may be integrated in a Model View Controller (MVC) design used for displaying content to a user. For example, the model and view of the MVC design may include additional variables associated with speakable properties. The electronic device may receive audio files for each speakable element using any suitable approach, including for example by providing a host device with a list of speakable elements and directing a text to speech engine of the host device to generate and provide the audio files.

Claims

1. A method for providing voice feedback to a user of an electronic device, comprising:

displaying a plurality of elements;

identifying at least two of the plurality of elements for which to provide voice feedback;

determining tiers associated with the display of each of the identified elements;

generating an initial queue comprising the identified elements in response to identifying and determining;

ordering the identified elements in the initial queue based on the determined tiers; and

providing voice feedback for the identified elements in the order of the elements in the initial queue.

2. The method of claim 1, further comprising:

retrieving audio files associated with each of the identified elements; and

playing back the retrieved audio files.

3. The method of claim 1, further comprising:

changing at least one of the displayed plurality of elements; and

updating at least a portion of the initial queue in response to changing.

4. The method of claim 3, further comprising:

re-identifying at least two of the plurality of elements for which to provide voice feedback in response to changing;

re-determining tiers associated with the display of each of the re-identified elements; and

generating a revised queue comprising the re-identified elements.

5. The method of claim 4, further comprising, when the changing occurs during the providing of the voice feedback for one of the identified elements:

detecting the element for which voice feedback was provided during changing;

determining that the detected element has the same position and tier in the initial queue and the revised queue; and

providing voice feedback from the revised queue starting with the detected element.

6. The method of claim 5, further comprising:

completing the voice feedback for the detected element.

7. The method of claim 4, further comprising, when the changing occurs during the providing of the voice feedback for one of the identified elements:

detecting the element for which voice feedback was provided during changing;

comparing the initial queue and the revised queue to identify common portions of the queues;

determining that the detected element is not in a portion of the revised queue that is in common with the initial queue; and

stopping providing voice feedback for the detected element.

8. The method of claim 7, further comprising:

identifying the first element of the revised queue following the common portions of the queues; and

providing voice feedback from the revised queue starting with the identified first element.

9. The method of claim 1, wherein providing the voice feedback comprises providing the voice feedback for the identified elements sequentially and without human intervention.

10. An electronic device operative to provide audio feedback for displayed content, comprising a processor, a display, and an audio output, the processor operative to:

direct the display to display a plurality of elements in views, wherein speakable properties are associated with at least two of the plurality of elements;

determine tiers associated with the views of each of the at least two elements from the associated speakable properties;

generate a queue comprising the at least two elements, wherein the order of the queue elements is set by the determined tiers; and

direct the audio output to sequentially speak each queue element in the order of the queue.

11. The electronic device of claim 10, wherein the processor is further operative to:

direct the display to display at least two text strings; and

direct the audio output to provide voice feedback for the at least two text strings.

12. The electronic device of claim 10, wherein the processor is further operative to:

detect a transaction; and

generate a revised queue comprising modified elements with which a speakable property is associated.

13. The electronic device of claim 12, wherein the processor is further operative to:

direct the audio output to provide audio associated with each element of the revised queue in the order of the revised queue.

14. The electronic device of claim 12, wherein the processor is further operative to:

determine that at least one of the displayed elements with which a speakable property is associated has changed; and

detect a transaction.

15. The electronic device of claim 14, wherein the processor is further operative to:

detect a user input changing at least one of the displayed elements with which a speakable property is associated; and

detect a transaction.

16. The electronic device of claim 15, wherein the input comprises at least one of a user selection of a displayed option and a user instruction to move a highlight region.

17. The electronic device of claim 10, wherein the processor operative to direct the audio output is further operative to direct the audio output to speak each queue element sequentially and without human intervention.

18. A method for speaking text of elements displayed by an electronic device, comprising:

defining a plurality of elements with which speakable properties are associated;

displaying the plurality of elements in a plurality of views, wherein each view is associated with a speakable order;

generating a queue comprising the plurality of elements, wherein the order of the plurality of elements in the queue is set from the speakable order;

pausing for a first timeout;

identifying audio files associated with each of the plurality of elements of the queue, wherein the audio files correspond to text to speak for each element;

sequentially playing back the identified audio files in the order of the queue; and

pausing for a second timeout.

19. The method of claim 18, wherein identifying further comprises retrieving audio files associated with each of the plurality of elements from a hash of the text to speak.

20. The method of claim 18, wherein the audio files are received from a host device.

21. The method of claim 20, wherein the host device generates the audio files using a text to speech engine.

22. The method of claim 21, further comprising:

providing the text to speak for each of the plurality of elements to the host device; and

receiving audio files generated using a text to speech engine applied to the provided text to speak for each of the plurality of elements.

23. The method of claim 18, further comprising:

changing at least one of the displayed plurality of elements; and

generating a revised queue comprising the changed displayed plurality of elements ordered from the speakable orders associated with the displayed views.

24. The method of claim 18, wherein the identified audio files are sequentially played back in the order of the queue without human intervention.

25. A non-transitory computer readable storage media for providing voice feedback to a user of an electronic device, the computer readable media comprising computer program logic recorded thereon for:

displaying a plurality of elements;

identifying at least two of the plurality of elements for which to provide voice feedback;

determining tiers associated with the display of each of the identified elements;

generating an initial queue comprising the identified elements in response to identifying and determining;

ordering the identified elements in the initial queue based on the determined tiers; and

providing voice feedback for the identified elements in the order of the determined tiers.

26. The non-transitory computer readable storage media of claim 25, wherein the computer program logic for providing the voice feedback further comprises computer program logic for providing the voice feedback for the identified elements sequentially and without human intervention.