Voice prompts for use in speech-to-speech translation system

• US 8,560,326 B2
• Filed: 05/05/2008
• Issued: 10/15/2013
• Est. Priority Date: 05/06/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gao, Yuqing, Gu, Liang, Liu, Fu-Hua
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US12/115,205
• Publication Number:
  US 20080243476A1
• Time in Patent Office: 1,989 Days
• Field of Search: 704 1- 5, 704 7- 9, 704/200, 704/231, 704/257, 704/258, 704270-277, 704E17001-E17003
• US Class Current: 704/277
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Techniques for employing improved prompts in a speech-to-speech translation system are disclosed. By way of example, a technique for use in indicating a dialogue turn in an automated speech-to-speech translation system comprises the following steps/operations. One or more text-based scripts are obtained. The one or more text-based scripts are synthesizable into one or more voice prompts. At least one of the one or more voice prompts is synthesized for playback from at least one of the one or more text-based scripts, the at least one synthesized voice prompt comprising an audible message in a language understandable to a speaker interacting with the speech-to-speech translation system, the audible message indicating a dialogue turn in the automated speech-to-speech translation system.

Claims

1. A method for use in indicating a dialogue turn in an automated speech-to-speech translation system, comprising the steps of:

translating speech input between a plurality of speakers having a multilingual conversation using an automated speech-to-speech translation system; and

providing an indication to each speaker of the plurality of speakers of when it is a turn of each speaker to commence speaking in a dialog interaction between the plurality of speakers and provide speech input to the automated speech-to-speech translation system, wherein providing an indication comprises;

obtaining one or more previously-generated text-based scripts, the one or more text-based scripts being synthesizable into one or more voice prompts in different languages of the plurality of speakers, wherein the voice prompts are audible messages that notify a given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system;

synthesizing for playback at least one of the one or more voice prompts from at least one of the one or more text-based scripts, the at least one synthesized voice prompt comprising an audible message in a language understandable to the given speaker to notify the given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system; and

playing the at least one synthesized voice prompt to provide the audible message to the given speaker to notify the given speaker that it is the given speaker'"'"'s turn for inputting speech to the automated speech-to-speech translation system.

2. The method of claim 1, further comprising the step of detecting a language spoken by the given speaker interacting with the speech-to-speech translation system such that a voice prompt in the detected language is synthesized for playback to the given speaker.

3. The method of claim 2, wherein an initial voice prompt is synthesized for playback in a default language until the actual language of the given speaker is detected.

4. The method of claim 1, further comprising the step of displaying text of the at least one voice prompt synthesized for playback.

5. The method of claim 1, further comprising the step of recognizing speech uttered by the given speaker interacting with the automated speech-to-speech translation system.

6. The method of claim 1, wherein the plurality of speakers include a system user that operates the automated speech-to-speech translation system and a foreign language speaker that interacts with the automated speech-to-speech translation system.

7. The method of claim 6, wherein at least a portion of the speech uttered by the foreign language speaker or the system user is translated from one language to another language.

8. The method of claim 7, further comprising displaying text of at least a portion of the translated speech.

9. A method of providing an interface for use in an automated speech-to-speech translation system, the automated speech-to-speech translation system being operated by a system user and interacted with by a foreign language speaker, the method comprising the steps of:

translating speech input between the foreign language speaker and the system user having a multilingual conversation using the automated speech-to-speech translation system; and

utilizing an interface of the automated speech-to-speech translation to provide an indication to the foreign language speaker of when it is a turn of the foreign language speaker to commence speaking in a dialog interaction with the system user and provide speech input to the automated speech-to-speech translation system by the foreign language speaker, wherein utilizing an interface comprises;

the system user enabling a microphone of the automated speech-to speech translation system via the interface;

synthesizing at least one previously-generated text-based scripts into a voice prompt for playback to the foreign language speaker, the voice prompt comprising an audible message in a language understandable to the foreign language speaker to notify the foreign language speaker when it is a turn of the foreign language speaker to input speech to the automated speech-to-speech translation system;

playing the audible message to the foreign language speaker to notify the foreign language speaker that is the foreign language speaker'"'"'s turn for inputting speech to the automated speech-to-speech translation system; and

receiving speech uttered into the microphone by the foreign language speaker for translation by the automated speech-to speech translation system.

10. The method of claim 9, further comprising the step of displaying text in a first field of the interface representing translated speech uttered by the system user.

11. The method of claim 10, further comprising the step of displaying text in a second field of the interface representing translated speech uttered by the speaker.

12. An apparatus for use in indicating a dialogue turn in an automated speech-to-speech translation system, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

(i) translate speech input from a plurality of speakers having a multilingual conversation using an automated speech-to-speech translation system;

(ii) provide an indication to each speaker of the plurality of speakers of when it is a turn of each speaker to commence speaking in a dialog interact between the plurality of speakers and provide speech input to the automated speech-to-speech translation system, wherein the at least one processor is operative to provide an indication by;

obtaining one or more previously-generated text-based scripts, the one or more text-based scripts being synthesizable into one or more voice prompts in different languages of the plurality of speakers, wherein the voice prompts are audible messages that notify a given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system;

synthesizing for playback at least one of the one or more voice prompts from at least one of the one or more text-based scripts, the at least one synthesized voice prompt comprising an audible message in a language understandable to the given speaker to notify the given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system; and

playing the at least one synthesized voice prompt to provide the audible message to the given speaker to notify the given speaker that it is the given speaker'"'"'s turn for inputting speech to the automated speech-to-speech translation system.

13. The apparatus of claim 12, wherein the at least one processor is further operative to detect a language spoken by the given speaker interacting with the speech-to-speech translation system such that a voice prompt in the detected language is synthesized for playback to the given speaker.

14. The apparatus of claim 13, wherein an initial voice prompt is synthesized for playback in a default language until the actual language of the given speaker is detected.

15. The apparatus of claim 12, wherein the at least one processor is further operative to display text of the at least one voice prompt synthesized for playback.

16. The apparatus of claim 12, wherein the at least one processor is further operative to recognize speech uttered by the given speaker interacting with the speech-to-speech translation system.

17. The apparatus of claim 16, wherein the plurality of speakers includes a system user that operates the speck-to-speech translation system and a foreign language speaker.

18. The apparatus of claim 17, wherein at least a portion of the speech uttered by the foreign language speaker or the system user is translated from one language to another language.

19. The apparatus of claim 18, wherein at least a portion of the translated speech is displayed as text.

20. An interface for use in an automated speech-to-speech translation system, the automated speech-to-speech translation system being operated by a system user and interacted with by a foreign language speaker, the interface comprising:

a display to display a graphical user interface of the automated speech-to-speech translation system, wherein the graphical user interface comprises;

a first field for use by the system user to enable a microphone of the automated speech-to-speech translation system;

a second field for use by the system user for at least one of displaying text of speech uttered by the system user and displaying text of translated speech uttered by the foreign language speaker; and

a third field for use by the foreign language speaker for at least one of displaying text of speech uttered by the speaker and displaying text of translated speech uttered by the system user;

wherein the automated speech-to-speech translation system synthesizes for audible output at least one previously-generated voice prompt to the foreign language speaker in a language understandable to the foreign language speaker to notify the foreign language speaker when it is a turn of the foreign language speaker for inputting speech to the automated speech-to-speech translation system, and wherein the automated speech-to-speech translation system receives speech uttered into the microphone by the foreign language speaker for translation by the automated speech-to-speech translation system.

21. The interface of claim 20, the graphical user interface further comprising a fourth field for use by the system user to enable a microphone of the automated speech-to-speech translation system such that speech uttered by the system user is captured by the automated speech-to-speech translation system.

22. An article of manufacture for use in indicating a dialogue turn in an automated speech-to-speech translation system, comprising a non-transitory computer readable storage medium containing one or more programs which when executed implement the steps of:

translating speech input between a plurality of speakers having a multilingual conversation using an automated speech-to-speech translation system; and

providing an indication to each speaker of the plurality of speakers of when it is a turn of each speaker to commence speaking in a dialog interaction between the plurality of speakers and provide speech input to the automated speech-to-speech translation system, wherein providing an indication comprises;

obtaining one or more previously-generated text-based scripts, the one or more text-based scripts being synthesizable into one or more voice prompts in different languages of the plurality of speakers, wherein the voice prompts are audible messages that notify a given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system;

synthesizing for playback at least one of the one or more voice prompts from at least one of the one or more text-based scripts, the at least one synthesized voice prompt comprising an audible message in a language understandable to the given speaker to notify the given speaker when it is a turn of the given speaker for inputting speech to the automated speech-to-speech translation system; and

playing the at least one synthesized voice prompt to provide the audible message to the given speaker to notify the given speaker that it is the given speaker'"'"'s turn for inputting speech to the automated speech-to-speech translation system.