Speech recognition using a foreign word grammar

• US 10,290,299 B2
• Filed: 07/17/2014
• Issued: 05/14/2019
• Est. Priority Date: 07/17/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hwang, Mei-Yuh, Zhang, Hua
• Primary Examiner(s): Mishra, Richa
• Application Number:
  US14/408,773
• Publication Number:
  US 20160267902A1
• Time in Patent Office: 1,762 Days
• Field of Search: None
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

Systems and methods are utilized for recognizing speech that is partially in a foreign language. The systems and methods receive speech input from a user and detect if a rule or sentence entry grammar structure utilizing a foreign word has been uttered. To recognize the foreign word, a foreign word grammar is utilized. The foreign word grammar includes rules for recognizing the uttered foreign word. Two rules may be included in the foreign word grammar for each legitimate or slang term included in the foreign word grammar. A first rule corresponds to the spoken form of the foreign word, and the second rule corresponds to the spelling form of the foreign word. The foreign word grammar may also utilize a prefix tree. Upon recognizing the foreign word, the recognized foreign word may be sent to an application to retrieve the pronunciation, translation, or definition of the foreign word.

Claims

1. A computer-implemented method for improved computerized speech recognition of speech uttered in at least two different languages, the method comprising:

receiving speech input, wherein the speech input includes the foreign word;

detecting, in the speech input, a sentence entry grammar structure from a set of predetermined sentence entry grammar structures, wherein the detected sentence entry grammar structure includes static text in a first language indicating that the foreign word in a second language is to be identified and a placeholder for the foreign word, wherein the foreign word is in the second language;

based on detecting the sentence entry grammar structure, triggering the use of a foreign word grammar for recognition of the foreign word in the placeholder;

utilizing the foreign word grammar storing a plurality of rules to recognize the foreign word, wherein the foreign word grammar includes a first rule in the form of a grammar structure for processing the spoken form of the foreign word and a second rule in the form of a grammar structure for processing the spelling form of the foreign word;

generating a response based on the recognized foreign word;

recording the recognized foreign word in an aggregated result database; and

adjusting statistical weights in the foreign word grammar.

2. The method of claim 1, further comprising sending the recognized foreign word to an application, wherein the application is selected based on at least the detected sentence entry grammar structure.

3. The method of claim 2, further comprising receiving results from the application, wherein the results from the application include a definition for the foreign word in the first language.

4. The method of claim 1, wherein speech input of the foreign word is received as a spelling of the foreign word.

5. The method of claim 1, wherein the foreign word grammar includes a prefix tree, and utilizing the foreign word grammar further comprises utilizing the prefix tree to recognize the foreign word, wherein probabilities are incorporated into transition arcs of the prefix tree.

6. The method of claim 1, wherein the speech input is captured on a client device and received from the client device at a cloud-based server;

and

wherein the detecting the sentence entry grammar structure and the utilizing the foreign word grammar to recognize the foreign word are performed on the cloud-based server.

7. The method of claim 1, wherein the foreign word grammar constrains the results of the speech recognition to only those words included within the foreign word grammar.

8. The method of claim 1, wherein the foreign word grammar includes multiple words in the second language, wherein all the words in the foreign word grammar are legitimate words or slang terms in the second language.

9. A computer storage device, having computer-executable instructions that, when executed by at least one processor, perform a method for speech recognition of a foreign word, the method comprising:

capturing speech input, wherein the speech input includes a spelling of the foreign word;

receiving a result for the speech recognition of the foreign word, wherein the result of the speech recognition is based on;

a detection of a sentence entry grammar structure in the captured speech, wherein the sentence entry grammar structure is from a set of predetermined sentence entry grammar structures and includes static text in a first language indicating that the foreign word in a second language is to be identified and a placeholder for the foreign word, wherein the foreign word is in a second language; and

based on the detection of the sentence entry grammar structure, a utilization of a foreign word grammar, storing a plurality of rules, to recognize the foreign word, wherein the foreign word grammar includes a first rule in the form of a grammar structure for the spoken form the foreign word and a second rule in the form of a grammar structure for the spelling of the foreign word, and the foreign word grammar constrains results of the recognition to words in the foreign word grammar, wherein the words in the foreign word grammar are only legitimate words or slang terms in the second language; and

generating a response based on the recognized foreign word;

recording the recognized foreign word in an aggregated result database; and

adjusting statistical weights in the foreign word grammar.

10. The computer storage device of claim 9, wherein the application is selected based on at least the detected sentence entry grammar structure.

11. The computer storage device of claim 10, wherein the action is retrieving a definition in the first language for the foreign word.

12. The computer storage device of claim 10, wherein the foreign word grammar includes a prefix tree, and utilizing the foreign word grammar further comprises utilizing the prefix tree to recognize the foreign word, wherein probabilities are incorporated into transition arcs of the prefix tree.

13. A system comprising:

at least one processor; and

a memory including instructions that when executed by the at least one processor performs a set of actions comprising;

receiving speech input, wherein the speech input includes the foreign word;

detecting, in the speech input, a sentence entry grammar structure from a set of predetermined sentence entry grammar structures, wherein the detected sentence entry grammar structure includes static text in a first language indicating that the foreign word in a second language is to be identified and a placeholder for the foreign word, wherein the foreign word is in a second language;

based on detecting the sentence entry grammar structure, triggering the use of a foreign word grammar for recognition of the foreign word in the placeholder;

utilizing the foreign word grammar storing a plurality of rules to recognize the foreign word, wherein the foreign word grammar includes a first rule in the form of a grammar structure for the spoken form of the foreign word and a second rule in the form of a grammar structure for the spelling form of the foreign word, wherein the foreign word grammar includes a prefix tree, and utilizing the foreign word grammar further comprises utilizing the prefix tree to recognize the foreign word, wherein probabilities are incorporated into transition arcs of the prefix tree; and

executing a function based on the detected sentence entry grammar structure.

14. The system of claim 13, wherein the application is selected based on at least the detected sentence entry grammar structure.

15. The system of claim 13, wherein the function includes retrieving a definition in the first language for the foreign word.

16. The system of claim 13, wherein the actions further comprise:

recording the recognized foreign word in an aggregated result database; and

adjusting statistical weights in the foreign word grammar.

17. The system of claim 13, wherein the speech input is captured on a client device and received from the client device at a cloud-based server;

and

wherein the detecting the sentence entry grammar structure and the utilizing the foreign word grammar to recognize the foreign word are performed on the cloud-based server.

18. The system of claim 13, wherein speech input of the foreign word is received as a spelling of the foreign word.