Mobile speech-to-speech interpretation system

• US 8,775,181 B2
• Filed: 07/02/2013
• Issued: 07/08/2014
• Est. Priority Date: 01/09/2008
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Fluential, LLC
• Inventors: Ehsani, Farzad, Master, Demitrios, Zuber, Elaine Drom
• Primary Examiner(s): SAINT CYR, LEONARD
• Application Number:
  US13/934,194
• Publication Number:
  US 20130297288A1
• Time in Patent Office: 371 Days
• Field of Search: 704 2- 10, 704/246, 704/247, 704/251, 704/252
• US Class Current: 704/246
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Interpretation from a first language to a second language via one or more communication devices is performed through a communication network (e.g. phone network or the internet) using a server for performing recognition and interpretation tasks, comprising the steps of: receiving an input speech utterance in a first language on a first mobile communication device; conditioning said input speech utterance; first transmitting said conditioned input speech utterance to a server; recognizing said first transmitted speech utterance to generate one or more recognition results; interpreting said recognition results to generate one or more interpretation results in an interlingua; mapping the interlingua to a second language in a first selected format; second transmitting said interpretation results in the first selected format to a second mobile communication device; and presenting said interpretation results in a second selected format on said second communication device.

Claims

1. A system for interpreting from a first language to a second language, comprising:

an audio input device configured to receive an input speech utterance;

an output device configured to present a communication to a user; and

a server having a database and a recognition module and an interpretation module collectively configured to;

condition the input speech utterance to produce a conditioned input speech utterance;

recognize the conditioned input speech utterance to generate a set of first recognition results;

optimize the set of first recognition results by rescoring word-recognition lattice results in conjunction with interpretation lattice results to generate a set of second recognition results;

interpret the set of second recognition results by generating one or more interpretation results comprising abstract paraphrased specific concepts;

translate the abstract paraphrased specific concepts into a form of a language-independent abstract interlingua;

map the abstract paraphrased specific concepts from the language-independent interlingua to a second language in a first selected format; and

present the one or more interpretation results in a second selected format on the output device.

2. The system of claim 1, wherein the database comprises user profile data that includes one or more of:

domain selection, voice identification data, location data, and users'"'"' preferences.

3. The system of claim 1, wherein the audio input device and the output device are located on a first communication device.

4. The system of claim 3, wherein the server resides on the first communication device.

5. The system of claim 3, wherein the server resides remotely with respect to the first communication device.

6. The system of claim 5, further comprising a data connection between the first communication device and the server.

7. The system of claim 1, wherein the audio input device and server are located on a first communication device and the output device is located on a second communication device.

8. The system of claim 1, wherein the output device is one or more of:

a graphical user interface (GUI) and a voice user interface (VUI).

9. The system of claim 1, wherein the output device includes a graphical user interface comprising:

a first domain-specific pane; and

a second domain-specific pane having a different domain usage than the first domain-specific pane.

10. The system of claim 9, wherein the graphical user interface further comprises a development pane.

11. The system of claim 9, wherein the graphical user interface is displayed on a touch screen.

12. The system of claim 11, wherein the first domain-specific pane includes one or more of:

a domain button, a status button, a verification field, a search button, and a menu button.

13. The system of claim 9, wherein the first domain-specific pane relates to one of:

medical services and travel services.

14. The method of claim 1, wherein the second selected format includes text, audio, images and videos.