Mobile speech-to-speech interpretation system

• US 9,430,467 B2
• Filed: 12/18/2015
• Issued: 08/30/2016
• Est. Priority Date: 01/09/2008
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Ehsani, Farzad, Master, Demitrios, Zuber, Elaine Drom
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US14/974,529
• Publication Number:
  US 20160103825A1
• Time in Patent Office: 256 Days
• Field of Search: 704 2- 10, 704/246, 704/247, 704/251, 704/252
• US Class Current: 1/1
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

Interpretation from a first language to a second language via one or more communication devices is performed through a communication network (e.g. phone network or the internet) using a server for performing recognition and interpretation tasks, comprising the steps of: receiving an input speech utterance in a first language on a first mobile communication device; conditioning said input speech utterance; first transmitting said conditioned input speech utterance to a server; recognizing said first transmitted speech utterance to generate one or more recognition results; interpreting said recognition results to generate one or more interpretation results in an interlingua; mapping the interlingua to a second language in a first selected format; second transmitting said interpretation results in the first selected format to a second mobile communication device; and presenting said interpretation results in a second selected format on said second communication device.

Claims

1. A computer translation system comprising

A language input device that detects a first language signal and a device location;

and

An interpretation engine coupled with the language input device and configured to;

obtain the first language signal and the device location;

activate a language model, including a mobile interference model, based on the first language signal and at least in part based on the device location;

update utterance probabilities of the language model based the device location;

extract a conceptual interlingua result set from the first language signal according to the updated language model and by applying the mobile interference model;

map the conceptual interlingua result set to a second language representation of a second language; and

cause an output device to present an output interpretation according to the second language derived from the second language representation.

2. The system of claim 1, wherein the language input device comprises the interpretation engine.

3. The system of claim 1, wherein the language input device comprises a mobile device.

4. The system of claim 3, wherein the output device is a second mobile device at a different location that the language input device.

5. The system of claim 1, wherein the output device is the language input device.

6. The system of claim 1, wherein the device location corresponds to a user position.

7. The system of claim 1, wherein the interpretation engine is further configured to boost scores for results in the conceptual interlingua result set for an entity in proximity to the language input device based on the device location.

8. The system of claim 7, wherein the entity includes at least one of the following:

a street name and a landmark.

9. The system of claim 7, wherein the entity is named in the first language signal.

10. The system of claim 1, wherein the output interpretation includes at least one of the following:

text, a spoken language according to the second language, audio, a still image, a video, and a sign language.

11. The system of claim 1, wherein the activated language model relates to a nurse.

12. The system of claim 1, wherein the activated language model relates to a pharmacist.

13. The system of claim 1, wherein the activated language model relates to a tour.

14. The system of claim 1, wherein the activated language model relates to a sign language.

15. The system of claim 1, wherein the first language signal comprises a voice signal.

16. The system of claim 1, wherein the activated language model comprises an “

n-best list”

.

17. The system of claim 1, wherein the activated language model comprises a reject list of at least some recognition results.

18. The system of claim 1, wherein the activated language model comprises a domain-specific language model.

19. The system of claim 18, wherein the domain-specific language model comprises a user domain selection.

20. The system of claim 1, wherein the activated language model comprises an empirically derived model.

21. The system of claim 1, wherein the interpretation engine is further configured to update the utterance probabilities of the language model based on a detected move of the device location.