Audio user interface for computing devices

• US 7,779,357 B2
• Filed: 04/09/2007
• Issued: 08/17/2010
• Est. Priority Date: 11/04/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Naik, Devang K.
• Primary Examiner(s): Lee; Ting
• Assistant Examiner(s): ZAHR, ASHRAF A
• Application Number:
  US11/733,126
• Publication Number:
  US 20070180383A1
• Time in Patent Office: 1,226 Days
• Field of Search: 715270-271, 715/716, 715/727, 715810-845
• US Class Current: 715/727
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

An audio user interface that generates audio prompts that help a user interact with a user interface of a computing device is disclosed. The audio prompts can provide audio indicators that allow a user to focus his or her visual attention upon other tasks such as driving an automobile, exercising, or crossing a street, yet still enable the user to interact with the user interface. As examples, the audio prompts provided can audiblize the spoken version of a user interface selection, such as a selected function or a selected (e.g., highlighted) menu item of a display menu. The computing device can be, for example, a media player such as an MP3 player, a mobile phone, or a personal digital assistant.

Claims

1. A hand-held electronic device comprising:

a media database that stores media content;

a display device that is capable of visually displaying a user interface that enables a user to interact with the hand-held electronic device, the user interface presenting at least one menu, the at least one menu having a plurality of menu items;

a user input device capable of receiving a user input pertaining to a selection of at least one of the menu items in the at least one menu;

a memory, wherein the memory stores media files, each media file having an audio tag containing a pointer to an associated audio cue;

a user interface control module that, in response to the user input pertaining to a selection of the at least one menu item in the at least one menu receives a manually entered text string corresponding to the at least one menu item, wherein the manually entered text string does not replace a text string currently stored on the hand-held electronic device, and causes the manually entered text string to be passed to a host computer, wherein the host computer converts the manually entered text string to a corresponding audio cue and passes the corresponding audio cue back to the hand-held electronic device.

2. A hand-held electronic device as recited in claim 1, wherein if the user wishes to play the audio file associated with the played audio cue, then the user selects the at least one menu item again, otherwise, the user selects a different menu item.

3. A hand-held electronic device as recited in claim 1, wherein the user input device comprises a touch pad apparatus.

4. A hand-held electronic device as recited in claim 1, wherein the user interface presents at least a series of menus configured in a nested arrangement, each of the menus having at least one menu item.

5. A hand-held electronic device as recited in claim 4,

wherein the user interface presents at least one of the menus of the series of menus,

wherein the user input device is capable of receiving a user input pertaining to a selection of at least one of the menu items in the presented at least one menu of the series of menus,

wherein the memory stores audio files associated with the menu items of the menus of the series of menus, and

wherein the user interface control module that plays one of the audio files in response to the user input pertaining to a selection of at least one of the menu items in the presented at least one menu of the series of menus.

6. A hand-held electronic device as recited in claim 1, wherein the hand held electronic device is a portable media player.

7. A hand-held electronic device as recited in claim 1, wherein the audio files are provided to the hand-held electronic device by a host computer.

8. A hand-held electronic device as recited in claim 1, wherein the audio files are provided to the hand-held electronic device by a remote computer.

9. A hand-held electronic device as recited in claim 1, wherein the hand-held electronic device further comprises:

a communications port for receiving audio files created by a host computer and provided to the hand-held electronic device via the host computer.

10. A hand-held electronic device as recited in claim 1, wherein an audio tag is provided with at least one of the media files.

11. A hand-held electronic device as recited in claim 1, wherein each of at least a plurality of the media files comprises:

an audio tag that is provided within one of the media files; and

a pointer that is contained within the audio tag, the pointer pointing to an associated audio file.

12. A hand-held electronic device as recited in claim 1, wherein the hand-held electronic device further comprises:

a text-to-speech conversion engine that creates a respective audio file based upon a text string.

13. A hand-held electronic device as recited in claim 12, wherein the text string describes an attribute of a media file.

14. A hand-held electronic device as recited in claim 12, wherein the text string is song title, an album name, or an artist name.

15. A hand-held electronic device as recited in claim 1, wherein the hand-held electronic device further comprises:

a communications port for receiving audio files created by a host computer and provided to the hand-held electronic device via the host computer.

16. A hand-held electronic device as recited in claim 1, wherein the audio files are provided to the hand-held electronic device by a remote computer.

17. A hand-held electronic device as recited in claim 1 wherein the hand-held electronic device further comprises:

text-to-speech conversion engine that creates at least a portion of the audio files.

18. The hand-held electronic device, as recited in claim 1, wherein if the selected menu item is a play icon, then the user interface control module determines if an audio file is available for the control selection made by the user.

19. The hand-held electronic device as recited in claim 18, wherein if the user interface control determines that there is no audio file is available, then the user interface control is configured to receive a next control selection from the user.

20. The hand-held electronic device as recited in claim 18, wherein if the user interface control determines that an audio file exists, then the user interface control module retrieves the audio file associated with the control selection.

21. The hand-held electronic device as recited in claim 20, wherein the audio file to be retrieved can correspond to a user interface control, or information descriptive of a media file, or a menu item wherein the menu item includes text strings that can appear on the display screen.

22. The hand-held electronic device as recited in claim 21, further comprising:

a user selectable audio interface mode control module that causes the audio interface to operate in any of a number of user selectable modes of operation.

23. The hand-held electronic device as recited in claim 22, wherein the user selectable modes of operation includes generating an audio prompt for every action and menu item selected by the user or only generating an audio prompt for a highlighted menu item or only generating an audio prompt for a top level menu item.

24. The hand-held electronic device as recited in claim 23, wherein when a media file is being played back, the audio interface operation mode can be set such that the user selectable audio interface mode control module mixes the media file and audio file playback in different manners.

25. The hand-held electronic device as recited in claim 24, wherein a first manner of playing back the audio file is playing back the audio file at reduced volume when the audio prompt is to be played.

26. The hand-held electronic device as recited in claim 24, wherein a second manner of playback of a media file is the playback of the media file is paused during the playback of an audio prompt and then restarted after the audio prompt has been played.

27. The hand-held electronic device as recited in claim 24 wherein if the user interface control module detects that the user is making multiple user control selections in a certain time frame, the playback of the media file can be paused for a short period of time so that the playback of the media file need not be paused and restarted multiple times.

28. The hand-held electronic device of claim 1, wherein the at least one menu item is not uniquely associated with a piece of media content other than the audio cue.

29. A method executed by a processor included in a hand held electronic device, comprising:

displaying a user interface that enables a user to interact with the hand-held electronic device, the user interface presenting at least one menu, the at least one menu having a plurality of menu items;

receiving a text string, wherein the text string does not replace a text string currently stored on the hand-held electronic device;

sending the text string to a host computer that converts the replacement text string to an audio cue and passes the audio cue back to the hand held electronic device;

responding to a user input pertaining to a selection of the at least one menu item associated with the audio cue by playing the audio cue; and

wherein the hand held electronic device further stores a plurality of media files, each media file having an audio tag containing a pointer to an associated audio cue.

30. The method as recited in claim 29, further comprising:

storing the audio file having the embedded audio cue embedded in a memory unit included in the hand held electronic device.

31. The method as recited in claim 30 further comprising:

creating at least a portion of the audio cue using a text-to-speech conversion engine based upon a text string.

32. The method as recited in claim 31, wherein the text string describes an attribute of a media file.

33. The method as recited in claim 32, wherein the attribute of the media file is a song title, an album name, or an artist name.

34. The method as recited in claim 29, wherein if the user wishes to play the audio file corresponding to the played audio cue, then the user selects the at least one menu item again, otherwise, the user selects a different menu item.

35. The method as recited in claim 29, wherein the hand-held electronic device further comprises a communications port.

36. The method as recited in claim 35, further comprising:

receiving audio files created by the host computer at the communications port.

37. The method as recited in claim 36, wherein the audio files are received as part of a synchronization procedure between the host computer and the hand held electronic device.

38. The method of claim 29, wherein the at least one menu item is not uniquely associated with a piece of media content other than the audio cue.