Mobile speech-to-speech interpretation system

• US 9,251,142 B2
• Filed: 07/08/2014
• Issued: 02/02/2016
• Est. Priority Date: 01/09/2008
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Ehsani, Farzad, Master, Demitrios, Zuber, Elaine Drom
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US14/326,283
• Publication Number:
  US 20140316762A1
• Time in Patent Office: 574 Days
• Field of Search: 704 2- 10, 704/246, 704/247, 704/251, 704/252
• US Class Current: 1/1
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Interpretation from a first language to a second language via one or more communication devices is performed through a communication network (e.g. phone network or the internet) using a server for performing recognition and interpretation tasks, comprising the steps of: receiving an input speech utterance in a first language on a first mobile communication device; conditioning said input speech utterance; first transmitting said conditioned input speech utterance to a server; recognizing said first transmitted speech utterance to generate one or more recognition results; interpreting said recognition results to generate one or more interpretation results in an interlingua; mapping the interlingua to a second language in a first selected format; second transmitting said interpretation results in the first selected format to a second mobile communication device; and presenting said interpretation results in a second selected format on said second communication device.

Claims

1. A communication device comprising:

a language input device configured to detect a first language signal associated with a first language; and

a recognition and interpretation engine coupled with the language input device and configured to;

obtain the first language signal from the language input device;

generate a first recognition result set from the first language signal according to at least one of a grammar and statistical language model of the first language, said language model comprising a mobile interference model;

generate an improved recognition result set from the first recognition result set by rescoring the first recognition result set according to a domain-specific language model;

generate at least one interpretation result from the improved recognition results set;

map the at least one interpretation result to a second language representation of a second language; and

cause an output device to present an output interpretation according to the second language derived from the second language representation.

2. The device of claim 1, wherein the output interpretation comprises at least one of the following data formats:

text, audio, images, and video.

3. The device of claim 1, wherein the first language signal comprises an audio signal.

4. The device of claim 1, wherein the first language signal comprises a voice signal.

5. The device of claim 1, wherein the first language signal comprises a speech signal.

6. The device of claim 1, further comprising a mobile device that includes the language input device, and the recognition and interpretation engine.

7. The device of claim 1, wherein the output device comprises a second, different mobile device.

8. The device of claim 1, wherein the output device comprises a mobile communication device.

9. The device of claim 1, further comprising a server that includes the language input device and recognition and interpretation engine.

10. The device of claim 1, wherein the domain-specific language model includes an interpreted “

n best list”

.

11. The device of claim 1, wherein the domain-specific language model represents a user reject list of at least some of the first recognition results.

12. The device of claim 1, wherein the domain-specific language model include an interpretation lattice result.

13. The device of claim 1, wherein the domain-specific language model includes a location.

14. The device of claim 1, wherein the interference model represents a model of at least one the following:

a loss of the first language signal, a weak first language signal, a user profile, and a domain.

15. The device of claim 1, wherein the domain specific model is a user selectable domain.

16. The device of claim 1, wherein the language input device comprises a microphone.

17. The device of claim 1, wherein the domain-specific language model relates to a pharmacist.

18. The device of claim 1, wherein the domain-specific language model relates to a nurse.

19. The device of claim 1, wherein the domain-specific language model relates to a tour guide.

20. The device of claim 1, wherein the domain-specific language model relates to a sign language.

21. The device of claim 1, wherein the grammar and statistical language models comprise empirically determined mixtures and weightings.

22. The device of claim 1, wherein the second language representation comprises an language independent interlingua.

23. The device of claim 1, wherein the second language comprises a sign language.