Adaptive audio feedback system and method

• US 8,381,107 B2
• Filed: 01/13/2010
• Issued: 02/19/2013
• Est. Priority Date: 01/13/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Rottler, Benjamin Andrew, Lindahl, Aram, Haughay, Allen Paul Jr., Ellis, Shawn A., Wood, Policarpo
• Primary Examiner(s): RIEGLER, PATRICK F
• Application Number:
  US12/686,876
• Publication Number:
  US 20110173539A1
• Time in Patent Office: 1,133 Days
• Field of Search: 715/727, 715/716, 715/729, 715/978
• US Class Current: 715/727
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence

Abstract

Various techniques for adaptively varying audio feedback data on an electronic device are provided. In one embodiment, an audio user interface implementing certain aspects of the present disclosure may devolve or evolve the verbosity of audio feedback in response to user interface events based at least partially upon the verbosity level of audio feedback provided during previous occurrences of the user interface event. In another embodiment, an audio user interface may be configured to vary the verbosity of audio feedback associated with a navigable list of items based at least partially upon the speed at which a user navigates the list. In a further embodiment, an audio user interface may be configured to vary audio feedback verbosity based upon the contextual importance of a user interface event. Electronic devices implementing the present techniques provide an improved user experience with regard to audio user interfaces.

Claims

1. A method, comprising:

using a processor on an electronic device to;

detect the occurrence of a user interface event requesting audio feedback on the electronic device at a first time;

identify a plurality of audio items associated with the user interface event, wherein the plurality of audio items comprises at least two verbal audio items and at least one non-verbal audio item, and wherein each verbal audio item of the plurality of audio items has a different level of verbosity comprising a different amount of spoken words;

select a first audio item from the plurality of audio items that corresponds to a first desired verbosity level determined based at least partially upon whether the user interface event previously occurred within a particular time interval prior to the first time; and

provide the audio feedback by playing back the selected first audio item on the electronic device using an audio output device.

2. The method of claim 1, wherein selecting the first audio item comprises:

if the user interface event did not previously occur within the particular time interval prior to the first time, selecting an audio item having the highest level of verbosity from the plurality of audio items; and

if the user interface event previously occurred within the particular time interval prior to the first time, determining the verbosity level of an audio item selected from the plurality of audio items during the previous occurrence of the user interface event, and selecting an audio item from the plurality of audio items that is at least one step less verbose relative to the previously selected audio item.

3. The method of claim 2, comprising:

detecting a subsequent occurrence of the user interface event on the electronic device at a second time;

selecting a second audio item from the plurality of audio items that corresponds to a second desired verbosity level determined based upon the duration between the first time and the second time; and

playing the selected second audio item on the electronic device using the audio output device.

4. The method of claim 3, wherein selecting the second audio item comprises:

if the duration between the first and second times is less than or equal to the particular time interval, selecting an audio item from the plurality of audio items that is at least one step less verbose relative to the first audio item; and

if the duration between the first and second times is greater than the particular time interval, selecting an audio item from the plurality of audio items that is at least one step more verbose relative to the first audio item.

5. The method of claim 4, wherein selecting the audio item that is at least one step more verbose relative to the first audio item comprises selecting the audio item having the highest verbosity level within the plurality of audio items regardless of the verbosity level of the first audio item.

6. The method of claim 2, wherein selecting the first audio item comprises:

determining whether a playback termination event was detected during the previous occurrence of the user interface event; and

if a playback termination event was detected during the previous occurrence, selecting an audio item from the plurality of audio items having a lower level of verbosity regardless of the verbosity level of the previously selected audio item.

7. The method of claim 6, wherein the audio item having the lower level of verbosity comprises a non-verbal audio item.

8. The method of claim 6, wherein a user termination event comprises a user input or a gesture that prematurely terminated the playback of the previously selected audio item.

9. The method of claim 6, comprising:

if a playback termination event was not detected during the previous occurrence of the user interface event, using a graphical user interface to display a first visual notification at least partially concurrently with the playback of the selected first audio item; and

if a playback termination event was detected during the previous occurrence of the user interface event, using a graphical user interface to display a second visual notification substantially concurrently with the playback of the selected first audio item, wherein the second visual notification is less verbose relative to the first visual notification.

10. A method, comprising:

using a processor on an electronic device to;

identify a navigable set of data items stored on the electronic device;

receive an input from a user of the electronic device corresponding to a navigation event, wherein the navigation event includes a transition from a first data item to a second data item within the navigable set, such that the second data item becomes a currently selected data item;

detect a navigation speed corresponding to the navigation event;

vary audio feedback associated with the currently selected data item based at least partially upon the navigation speed;

determine whether the currently selected data item is newly acquired by the electronic device using a newness threshold;

play back a first non-verbal audio item if the currently selected data item is not newly acquired by the electronic device; and

play back a second non-verbal audio item if the currently selected data item is newly acquired by the electronic device, wherein the second non-verbal audio item is more audibly distinct relative to the first non-verbal audio item.

11. The method of claim 10, wherein varying the audio feedback comprises:

at a first navigation speed, playing back a first audio item having full verbosity; and

at a second navigation speed, playing back a second audio item that is less verbose relative to the first audio item;

wherein the second navigation speed is greater than the first navigation speed.

12. The method of claim 11, the navigable set of data items is ordered alphabetically on the electronic device, and wherein selecting and playing back the second audio item comprises:

if the currently selected data item is the first item of an alphabetical letter group, playing back the letter corresponding to the alphabetical letter group, or else playing back a non-verbal audio item.

13. The method of claim 10, wherein determining whether the currently selected data item includes new content comprises:

determining the time at which the currently selected data item was acquired by the electronic device;

determining whether the acquisition time occurred within a particular time interval prior to the present time;

and identifying the currently selected data item as new content if the acquisition time occurred within the particular time interval, or else, identifying the currently selected data item as not being new content.

14. The method of claim 10, wherein the navigable set of data items stored on the electronic device is displayable as a list on a display device, and wherein the input from the user corresponds to a scroll command configured to navigate the displayed list.

15. A method, comprising:

using a processor on an electronic device to;

detect the occurrence of a user interface event requesting audio feedback on the electronic device;

identify a plurality of audio items associated with the user interface event, wherein the plurality of audio items comprises at least two verbal audio items and at least one non-verbal audio item;

select an audio item from the plurality of audio items based at least partially on whether the user interface event requests a decision input from the user; and

provide the audio feedback by playing back the selected audio item on the electronic device using an audio output device.

16. The method of claim 15, wherein the plurality of audio items comprises at least a first audio item and a second audio item, wherein the first and second audio items have the same level of verbosity.

17. The method of claim 16, wherein selecting the audio item comprises determining a contextual importance comprising:

selecting the first audio item if the contextual importance is high; and

selecting the second audio item if the contextual importance is low;

wherein the first audio item is more audibly distinct relative to the first audio item.

18. The method of claim 17, wherein the contextual importance of the user interface event is determined to be high if the user interface event requests the decision input from the user of the electronic device.

19. The method of claim 17, wherein the first and second audio items each comprise non-verbal audio items.

20. The method of claim 17, wherein the first audio item has a more audibly distinct pitch, volume, playback time, or a combination thereof.

21. The method of claim 15, wherein the contextual importance of the user interface event is determined based upon information stored on the device, wherein the information stored on the device comprises one of pre-programmed information, or user-specified information configured through a set of user preference settings, or a combination thereof.

22. The method of claim 15, wherein the contextual importance of the user interface event is determined based upon the frequency at which the same type of user interface event occurred within a period of time just prior to the occurrence of the user interface event.

23. An electronic device, comprising:

at least one storage device configured to store a plurality of audio items;

a user interface comprising an audio user interface, wherein the audio user interface comprises;

audio feedback selection logic configured to;

detect the occurrence of a user interface event requesting audio feedback;

identify a set of audio items of the plurality of audio items associated with the user interface event, wherein the set of audio items comprises verbal and non-verbal audio items, and wherein the set of audio items has varying levels of verbosity comprising a lowest level that includes no spoken words, a highest level that includes a plurality of spoken words, and a level therebetween that includes at least one spoken word and fewer spoken words than the highest level;

determine a desired verbosity level based upon statistical data associated with the user interface event;

select from the set of audio items an audio item having a verbosity level corresponding to the desired verbosity level; and

initiate the playback of the selected audio item.

24. The electronic device of claim 23, wherein the selected audio item is played back on an audio output device comprising at least one of an integrated audio output device or an external audio output device coupled to the electronic device via an audio connection port, or a combination thereof.

25. The electronic device of claim 23, comprising a display, wherein the user interface comprises a graphical user interface configured to display a visual notification in response to the user interface event on the display, wherein the playback of the selected audio item is at least partially concurrent with the display of the visual notification.

26. The electronic device of claim 23, wherein the statistical data associated with the user interface event is stored in the at least one storage device.

27. The electronic device of claim 26, wherein the statistical data associated with the user interface event comprises data regarding previous occurrences of the user interface event, and wherein the determination of the desired verbosity level is based at least partially upon the verbosity level of an audio item selected during the most recent previous occurrence of the user interface event.

28. Audio processing circuitry comprising:

one or more memory devices configured to store a plurality of audio items, wherein each of the plurality of audio items is selectable for playback in response to detection of an audio user interface event requesting audio feedback; and

audio feedback selection logic configured to detect the occurrence of an audio user interface event requesting audio feedback, identify from the plurality of audio items stored in the one or more memory devices a set of audio items comprising verbal and non-verbal audio items having varying levels of verbosity and being associated with the audio user interface event, wherein the varying levels of verbosity comprise a lowest level that includes no spoken words, a highest level that includes a plurality of spoken words, and a level therebetween that includes at least one spoken word and fewer spoken words than the highest level, determine a desired verbosity level, select from the set of audio items an audio item having a verbosity level corresponding to the desired verbosity level, and output the selected audio item to an audio output device.

29. The audio processing logic of claim 28, wherein the desired verbosity level is determined based upon at least one of a set of user preference settings or statistical data associated with the audio user interface event, wherein the user preference settings and the statistical data are stored in the one or more memory devices.