Speech morphing communication system

• US 9,053,095 B2
• Filed: 10/30/2011
• Issued: 06/09/2015
• Est. Priority Date: 10/31/2010
• Status: Active Grant

• Current Assignee: Speech Morphing, Inc.
• Original Assignee: Speech Morphing, Inc.
• Inventors: Yassa, Fathy
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US13/284,979
• Publication Number:
  US 20120109629A1
• Time in Patent Office: 1,318 Days
• Field of Search: 704/2, 704/9, 704/4, 704/277, 704/275, 704/270, 704/268, 704/266, 704/260, 704/258, 704/254, 704/250, 704/240, 704/235, 704/224, 704/221, 704/211, 704/209, 704/207, 704/10, 709/224, 705/17, 455/557
• US Class Current: 1/1
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A communication system is described that includes an automatic speech recognizer configured to receive a speech signal and to convert the speech signal into a text sequence and includes a speech analyzer configured to receive the translator speech signal and to convert the translator speech signal into a text sequence. The speech analyzer configured to receive the translator signal. The speech analyzer configured to extract paralinguistic characteristics from the translator speech signal. The communication system includes a voice analyzer configured to receive a speech input signal. The voice analyzer configured to generate one or more phonemes based on the speech input signal. And, a speech output device coupled with the automatic speech recognizer, the speech analyzer, and the voice analyzer. The speech output device configured to convert the text sequence into an output speech signal based on the extracted paralinguistic characteristics and based on the one or more phonemes.

Claims

1. A communication system comprising:

an automatic speech recognizer configured to receive a translator speech signal of a first speaker and to convert said translator speech signal into a text sequence;

a speech analyzer configured to receive said translator speech signal, said speech analyzer configured to extract paralinguistic characteristics from said translator speech signal;

a voice analyzer configured to receive a speech input signal of a second speaker, said voice analyzer configured to generate one or more phonemes based on said speech input signal of said second speaker; and

a speech output device coupled with said automatic speech recognizer, said speech analyzer, and said voice analyzer, said speech output device configured to convert said text sequence of said translator speech signal into a converted speech signal of said second speaker based on said one or more phonemes of said speech input signal, transform the paralinguistic characteristics of said translator speech signal to paralinguistic characteristics of said speech input signal, and convert said converted speech signal into an output speech signal of said second speaker by applying said paralinguistic characteristics of said speech input signal to said converted speech signal.

2. The communication system of claim 1, wherein said speech output device includes a personifier.

3. The communication system of claim 1, wherein said automatic speech recognizer converts said translator speech signal into said text sequence based on a first algorithm.

4. The communication system of claim 3, further comprising a second automatic speech recognizer, said second automatic speech recognizer configured to convert said translator speech signal to a second text sequence based on a second algorithm.

5. The communication system of claim 4, further comprising a text sequence comparator configured to compare said text sequence with said second text sequence.

6. The communication system of claim 5, wherein said text sequence comparator is configured to generate an error corrected text sequence based on the comparison of said text sequence and said second text sequence.

7. The communication system of claim 5, wherein said text sequence comparator is configured to generate one or more confidence scores for said text sequence and for said second text sequence.

8. The communication system of claim 1, wherein said speech output device converts said text sequence into the converted speech signal using a text-to-speech algorithm.

9. A communication system comprising:

a memory;

one or more processors; and

one or more modules stored in memory and configured for execution by the one or more processors, the one or more modules comprising;

a translator speech input module configured to receive a translator speech signal of a first speaker;

a speech input module configured to receive a speech signal of a second speaker;

an automatic speech recognizer module coupled with said translator speech input module, said automatic speech recognizer module configured to convert said translator speech signal into a text sequence;

a speech analyzer module coupled with said translator speech input module, said speech analyzer module configured to extract paralinguistic characteristics from said translator speech signal;

a voice analyzer coupled with said speech input module, said voice analyzer configured to generate one or more phonemes based on said speech signal of said second speaker; and

an output speech module coupled with said automatic speech recognizer module, said speech analyzer module, and said voice analyzer module, said output speech module configured to convert said text sequence of said translator speech signal into a converted speech signal of said second speaker based on said one or more phonemes of said speech signal of said second speaker, transform the paralinguistic characteristics of said translator speech signal to paralinguistic characteristics of said speech signal of said second speaker, and convert said converted speech signal into an output speech signal of said second speaker by applying said paralinguistic characteristics of said speech signal to said converted speech signal.

10. The communication system of claim 9, wherein said speech output module includes a personifier.

11. The communication system of claim 9, wherein said automatic speech recognizer converts said translator speech signal into said text sequence based on a first algorithm.

12. The communication system of claim 11, further comprising a second automatic speech recognizer, said second automatic speech recognizer configured to convert said translator speech signal to a second text sequence based on a second algorithm.

13. The communication system of claim 12, further comprising a text sequence comparator configured to compare said text sequence with said second text sequence.

14. The communication system of claim 13, wherein said text sequence comparator is configured to generate an error corrected text sequence based on the comparison of said text sequence and said second text sequence.

15. The communication system of claim 13, wherein text sequence comparator module is configured to generate one or more confidence scores for said text sequence and for said second text sequence.

16. The communication system of claim 9, wherein said speech output device converts said text sequence into the converted speech signal using a text-to-speech algorithm.

17. A speech morphing communication system comprising:

a translator speech input device configured to receive a translator speech signal of a first speaker;

an speech input device configured to receive a speech signal of a second speaker;

a first automatic speech recognizer and a second automatic speech recognizer coupled with said translator speech input device, said first automatic speech recognizer configured to convert said translator speech signal into a first text sequence and said second automatic speech recognizer configured to convert said translator speech signal into a second text sequence;

a text sequence comparator coupled with said first automatic speech recognizer a second automatic speech recognizer, said text sequence comparator configured to select one text sequence from among said first text sequence and said second text sequence;

a speech analyzer coupled with said translator speech input device, said speech analyzer configured to extract paralinguistic characteristics from said translator speech signal;

a voice analyzer coupled with said speech input device, said voice analyzer configured to generate one or more phonemes based on said speech input signal of said second speaker; and

a speech output device coupled with said automatic speech recognizer, said speech analyzer, and said voice analyzer, said speech output device configured to convert said text sequence of said translator speech signal into a converted speech signal of said second speaker based on said one or more phonemes of said speech signal. transform the paralinguistic characteristics of said translator speech signal to paralinguistic characteristics of said speech signal and convert said converted speech signal into an output speech signal of said second speaker by applying said paralinguistic characteristics of said speech signal to said converted speech signal.

18. A method for converting speech into text comprising:

receiving a translator speech signal of a first speaker;

receiving a speech signal of a second speaker;

extracting paralinguistic characteristics from said translator speech signal;

converting said translator speech signal to a text sequence;

generating one or more phonemes from said speech signal of said second speaker;

converting said text sequence of said translator speech signal into a converted speech signal of said second speaker based on said one or more phonemes of said speech input signal;

transforming the paralinguistic characteristics of said translator speech signal to paralinguistic characteristics of said speech input signal; and

converting said converted speech signal into an output speech signal of said second speaker by applying said paralinguistic characteristics of said speech signal to said converted speech signal.

19. The method of claim 18 wherein, converting said text sequence comprises converting said text sequence using one or more text-to-speech algorithms.

20. The method of claim 19, wherein the transforming comprises transforming said paralinguistic characteristics into paralinguistic characteristics of a second language.