Audio renderings for expressing non-audio nuances

• US 7,062,437 B2
• Filed: 02/13/2001
• Issued: 06/13/2006
• Est. Priority Date: 02/13/2001
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Stern, Edith H., Willner, Barry E., Kovales, Renee M., Mathewson, James M. II
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US09/782,564
• Publication Number:
  US 20020110248A1
• Time in Patent Office: 1,946 Days
• Field of Search: 704/260, 704/270.1, 704/3, 704/270, 704/275, 704/235, 705/10, 846/50, 379/201.01, 379/67.1, 379/88.15, 379/88.17
• US Class Current: 704/260
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

Methods, systems, computer program products, and methods of doing business by adapting audio renderings of non-audio messages (for example, e-mail messages that are processed by a text-to-speech translator) to reflect various nuances of the non-audio information. Audio cues are provided for this purpose, which are sounds that are “mixed” in with the audio rendering as a separate (background) audio stream. Audio cues may reflect information such as the topical structure of a text file, or changes in paragraphs. Or, audio cues may be used to signal nuances such as changes in the color or font of the source text. Audio cues may also be advantageously used to reflect information about the translation process with which the audio rendering of a text file was created, such as using varying background tones to convey the degree of certainty in the accuracy of translating text to audio using a text-to-speech translation system, or of translating audio to text using a voice recognition system, or of translating between languages, and so forth. Stylesheets, such as those encoded in the Extensible Stylesheet Language (“XSL”), may optionally be used to customize the audio cues. For example, a user-specific stylesheet customization may be performed to override system-wide default audio cues for a particular user, enabling her to hear a different background sound for messages on a particular topic than other users will hear.

Claims

1. A method of enhancing audio renderings of non-audio data sources, comprising:

detecting a nuance of a non-audio data source;

locating an audio cue corresponding to the detected nuance; and

associating the located audio cue with the detected nuance for playback to a listener, wherein detecting a nuance of a non-audio data source detects a plurality of nuances of the non-audio data source, locating an audio cue locates audio cues for each of the detected nuances, and associating the located audio cue with the detected nuance for playback to a listener associates each of the located audio cues with the respective detected nuance, and further comprising;

creating an audio rendering of the non-audio data source; and

mixing the associated audio cues in with the audio rendering to generate integrated sounds therefrom to the listener.

2. The method according to claim 1, wherein mixing the associated audio cues occurs while playing the audio rendering to the listener.

3. The method according to claim 1, wherein the non-audio data source is a text file and wherein creating an audio rendering of the non-audio data source further comprises processing the text file with a text-to-speech translator.

4. The method according to claim 1, wherein at least one of the detected nuances is presence of a formatting tag.

5. The method according to claim 4, wherein the formatting tag is a new paragraph tag.

6. The method according to claim 1, wherein the non-audio data source is a text file and at least one of the detected nuances is a change in color of text in the text file.

7. The method according to claim 1, wherein the non-audio data source is a text file and the detected nuance is a change in font of text in the text file.

8. The method according to claim 1, wherein the non-audio data source is a text file and the detected nuance is presence of a keyword for the text file.

9. The method according to claim 8, wherein the keyword is supplied by a creator of the text file.

10. The method according to claim 8, wherein the keyword is programmatically detected by evaluating text in the text file.

11. The method according to claim 1, wherein the non-audio data source is a text file and at least one of the detected nuances is presence of an emoticon in the text file.

12. The method according to claim 1, wherein the detected nuance is a change of topic in the non-audio data source.

13. The method according to claim 1, wherein at least one of the detected nuances is a degree of certainty in translation of the non-audio data source from another format.

14. The method according to claim 13, wherein detecting a nuance of a non-audio data source detects at least two different degrees of certainty, and wherein the located audio cues comprise changes in a pitch of a voice used in the audio rendering for each of the different degrees of certainty.

15. The method according to claim 13, wherein detecting a nuance of a non-audio data source detects at least two different degrees of certainty, and further comprising changing a pitch of the associated audio cue used by mixing the associated audio cues in with the audio rendering for each of the different degrees of certainty.

16. The method according to claim 13, wherein detecting a nuance of a non-audio data source detects at least two different degrees of certainty, and wherein mixing the associated audio cues in with the audio rendering further comprises alternating between two of the located audio cues to audibly indicate the different degrees of certainty.

17. The method according to claim 13, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects accuracy of the audio-to-text translation.

18. The method according to claim 13, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects identification of a speaker who created the input audio data source.

19. The method according to claim 13, wherein the other format is a source text file and the non-audio data source is an output text file, and the translation is a text-to-text translation from the source text file to the output text file, and wherein the degree of certainty reflects accuracy of the text-to-text translation.

20. The method according to claim 19, wherein the source text file contains text in a first language and the output text file contains text in a second language.

21. A system for enhancing audio renderings of non-audio data sources, comprising:

means for detecting one or more nuances of a non-audio data source;

means for locating an audio cue corresponding to each of the detected nuances;

means for associating the located audio cues with their respective detected nuances for playback to a listener;

means for creating an audio rendering of the non-audio data source, wherein the non-audio segment is associated with the nuance; and

means for mixing the associated audio cues in with the audio rendering to generate integrated sounds therefrom to the listener.

22. The system according to claim 21, wherein the non-audio data source is a text file and wherein the means for creating further comprises means for processing the text file with a text-to-speech translator.

23. The system according to claim 21, wherein at least one of the detected nuances is presence of a formatting tag.

25. The system according to claim 21, wherein the non-audio data source is a text file and the detected nuance is a change in font of text in the text file.

26. The system according to claim 21, wherein the non-audio data source is a text file and at least one of the detected nuances is presence of an emoticon in the text file.

27. The system according to claim 21, wherein the detected nuance is a change of topic in the non-audio data source.

28. The system according to claim 21, wherein at least one of the detected nuances is a degree of certainty in translation of the non-audio data source from another format.

29. The system according to claim 28, wherein the means for detecting detects at least two different degrees of certainty, and wherein the located audio cues comprise changes in a pitch of a voice used in the audio rendering for each of the different degrees of certainty.

30. The system according to claim 28, wherein the means for detecting detects at least two different degrees of certainty, and further comprising means for changing a pitch of the associated audio cue used by the means for mixing for each of the different degrees of certainty.

31. The system according to claim 28, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects accuracy of the audio-to-text translation.

32. The system according to claim 28, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects identification of a speaker who created the input audio data source.

33. The system according to claim 28, wherein the other format is a source text file and the non-audio data source is an output text file, and the translation is a text-to-text translation from the source text file to the output text file, and wherein the degree of certainty reflects accuracy of the text-to-text translation.

34. The system according to claim 21, wherein the non-audio data source is an e-mail message and at least one of the detected nuances is an e-mail convention found in the e-mail message.

35. The system according to claim 21, wherein the non-audio data source is text provided by a user.

36. The system according to claim 21, wherein the detected nuance is embedded within the non-audio file.

37. The system according to claim 21, wherein the detected nuance comprises metadata associated with the non-audio file.

38. A computer program product for enhancing audio renderings of non-audio data sources, the computer program product embodied on one or more computer-readable media and comprising:

computer-readable program code that is configured to detect one or more nuances of a non-audio data source;

computer-readable program code that is configured to locate an audio cue corresponding to each of the detected nuances;

computer-readable program code that is configured to associate the located audio cues with their respective detected nuances for playback to a listener;

computer-readable program code that is configured to create an audio rendering of a non-audio segment of the non-audio data source, wherein the non-audio segment is associated with the nuance; and

computer-readable program code that is configured to mix the associated audio cue with the audio rendering of the segment to generate integrated sounds therefrom to the listener.

24. The system according to claim 46, wherein the formatting tag is a new paragraph tag.

39. The computer program product according to claim 38, wherein the non-audio data source is a text file and wherein the computer-readable program code that is configured to create further comprises computer-readable program code that is configured to process the text file with a text-to-speech translator.

40. The computer program product according to claim 38, wherein the non-audio data source is a text file and at least one of the detected nuances is a change in color of text in the text file.

41. The computer program product according to claim 38, wherein the non-audio data source is a text file and the detected nuance is presence of a keyword for the text file.

42. The computer program product according to claim 41, wherein the keyword is supplied by a creator of the text file.

43. The computer program product according to claim 41, wherein the keyword is programmatically detected by evaluating text in the text file.

44. The computer program product according to claim 38, wherein at least one of the detected nuances is a degree of certainty in translation of the non-audio data source from another format.

45. The computer program product according to claim 44, wherein the computer-readable program code that is configured to detect detects at least two different degrees of certainty, and wherein the located audio cues comprise changes in a pitch of a voice used in the audio rendering for each of the different degrees of certainty.

46. The computer program product according to claim 44, wherein the computer-readable program code that is configured to detect detects at least two different degrees of certainty, and further comprising changing a pitch of the associated audio cue used by the computer-readable program code that is configured to mix for each of the different degrees of certainty.

47. The computer program product according to claim 44, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects accuracy of the audio-to-text translation.

48. The computer program product according to claim 44, wherein the other format is an input audio data source and the non-audio data source is a text file, and the translation is an audio-to-text translation from the input audio data source to the text file, and wherein the degree of certainty reflects identification of a speaker who created the input audio data source.

49. The computer program product according to claim 44, wherein the other format is a source text file and the non-audio data source is an output text file, and the translation is a text-to-text translation from the source text file to the output text file, and wherein the degree of certainty reflects accuracy of the text-to-text translation.

50. The computer program product according to claim 49, wherein the source text file contains text in a first language and the output text file contains text in a second language.

51. The computer program product according to claim 38, wherein at least one of the detected nuances is an identification of a creator of the non-audio data source.

52. The computer program product according to claim 51, wherein the identification is used to locate stored preferences of the creator.

53. The computer program product according to claim 38, wherein the non-audio data source is an e-mail message.

54. The computer program product according to claim 38, wherein the detected nuance is embedded within the non-audio file.

55. The computer program product according to claim 38, wherein the detected nuance comprises metadata associated with the non-audio file.