Speech-enabled language translation system and method enabling interactive user supervision of translation and speech recognition accuracy

• US 7,539,619 B1
• Filed: 09/07/2004
• Issued: 05/26/2009
• Est. Priority Date: 09/05/2003
• Status: Active Grant

• Current Assignee: ZAMA Innovations LLC
• Original Assignee: Spoken Translation IND
• Inventors: Cain, Gerald Richard, Dillinger, Mike, Seligman, Mark, Friedland, Barton
• Primary Examiner(s): McFadden, Susan
• Application Number:
  US10/936,093
• Time in Patent Office: 1,722 Days
• Field of Search: 704/277
• US Class Current: 704/277
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A system and method for a highly interactive style of speech-to-speech translation is provided. The interactive procedures enable a user to recognize, and if necessary correct, errors in both speech recognition and translation, thus providing robust translation output than would otherwise be possible. The interactive techniques for monitoring and correcting word ambiguity errors during automatic translation, search, or other natural language processing tasks depend upon the correlation of Meaning Cues and their alignment with, or mapping into, the word senses of third party lexical resources, such as those of a machine translation or search lexicon. This correlation and mapping can be carried out through the creation and use of a database of Meaning Cues, i.e., SELECT. Embodiments described above permit the intelligent building and application of this database, which can be viewed as an interlingua, or language-neutral set of meaning symbols, applicable for many purposes. Innovative techniques for interactive correction of server-based speech recognition are also described.

Claims

1. A system providing cross-linguistic communication, comprising:

a client component capturing speech and tactile inputs, the client component providing a user interface configured to display and correct the inputs and meanings for the inputs, a translation of the input term into a different language than an input language, and a back translation of the translation, i.e., a retranslation from the target language back into the source language, intended to help the user to judge the translation quality; and

a server component providing the meanings, the translation and the back translation to the client component based upon the input term, the server component including,

an interaction manager requesting the translation of the inputs, the interaction manager accessing a database of Meaning Cues corresponding to the inputs, the corresponding Meaning Cues being displayed through the client component to facilitate word sense selection for the inputs.

2. The system of claim 1 further comprising:

at least two lexical resources, the first lexical resource including a dictionary database containing a plurality of words, wherein each of the plurality of words is associated with a meaning, the second lexical resource containing a plurality of word meanings in a central repository, each word meaning mapped to a corresponding word sense of the database of the Meaning Cues.

3. The system of claim 1 further comprising:

a memory database storing user profiles defining word sense preferences for each user profile.

4. The system of claim 1, wherein the server component includes a speech recognition module in communication with the interaction manager, the speech recognition module permitting use of voice commands captured through the client component.

5. The system of claim 1, wherein the server component includes a text-to-speech module providing an audio signal to the client component for pronunciation of the input term in the different, language and the input language.

6. The system of claim 2, wherein the second lexical resource is organized according to word senses and includes multiple Meaning Cues.

7. The system of claim 1, wherein the back translation reflects word sense selections for input terms.

8. The system of claim 1, wherein the Meaning Cues include one of synonyms, related words, definitions, example sentences, or graphic elements related to corresponding inputs.

10. The system of claim 7, wherein Meaning Cue information is organized by word sense.

11. The system of claim 7, wherein the interaction manager

receives the term for translation from the multi-modal user interface;

accesses the translation engine to locate a translated term corresponding to the term;

associates the term with Meaning Cues, the Meaning Cues including one of synonyms, related words, definitions, example sentences, or graphic elements; and

transmits the Meaning Cues to the multi-modal user interface for display.

9. A cross linguistic communication system, comprising:

a client component including a multi-modal user interface; and

a server component including an interaction manager in communication with the multi-modal user interface, the interaction manager providing semantic cross indexing or mapping between a term output from a translation engine accessible to the server component and a database comprised of Meaning Cues designed to clarify for users word sense options for the term, wherein the Meaning Cues are returned to the interaction manager for display through the multi-modal user interface to enable interactive user modification of the translation.

12. The system of claim 9, wherein the translation engine provides a back translation of the translation which reflects word sense selections for one of the input terms.

13. A computer-readable storage medium having computer-executable instructions stored thereon that, when executed, display a graphical user interface (GUI) enabling computer mediated communication, the GUI comprising:

a first section of the GUI displaying input terms in a first language;

a second section of the GUI displaying Meaning Cues for each of the input terms;

a third section of the GUI displaying a translation of the input terms in a second language; and

a fourth section of the GUI displaying a back translation in the first language, the back translation based upon the translation.

14. The computer-readable storage medium of claim 13, wherein selection of one of the Meaning Cues in the second section triggers a display of a fifth section, the fifth section enabling selection of a word sense for the input term.