Automatic reading tutoring with parallel polarized language modeling

• US 20080177545A1
• Filed: 01/19/2007
• Published: 07/24/2008
• Est. Priority Date: 01/19/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ju, Yun-Cheng, Deng, Li, Li, Xiaolong, Acero, Alejandro
• Granted Patent: US 8,433,576 B2
• Time in Patent Office: Days
• US Class Current: 704/255
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A novel system for automatic reading tutoring provides effective error detection and reduced false alarms combined with low processing time burdens and response times short enough to maintain a natural, engaging flow of interaction. According to one illustrative embodiment, an automatic reading tutoring method includes displaying a text output and receiving an acoustic input. The acoustic input is modeled with a domain-specific target language model specific to the text output, and with a general-domain garbage language model, both of which may be efficiently constructed as context-free grammars. The domain-specific target language model may be built dynamically or “on-the-fly” based on the currently displayed text (e.g. the story to be read by the user), while the general-domain garbage language model is shared among all different text outputs. User-perceptible tutoring feedback is provided based on the target language model and the garbage language model.

Claims

1. A method comprising:

displaying a text output;

receiving an acoustic input;

modeling the acoustic input with a domain-specific target language model specific to the text output;

further modeling the acoustic input with a general-domain garbage language model; and

providing user-perceptible feedback based on the target language model and the garbage language model.

2. The method of claim 1, wherein the target language model is constructed with N-grams of a user-selectable order.

3. The method of claim 2, wherein the N-grams may be selected from among unigrams, bigrams, and trigrams.

4. The method of claim 1, wherein the domain-specific target language model is dynamically assembled based on a unit of the text output while it is being displayed.

5. The method of claim 4, further comprising:

iteratively displaying additional units of the text output, and dynamically assembling additional domain-specific target language models respectively based on each of the additional units of the text output while they are being displayed.

6. The method of claim 4, wherein the units of the text output are selected from among either sentences or paragraphs.

7. The method of claim 1, wherein the target language model and the garbage language model are used to detect elements of the acoustic model that do not correspond properly to the text output and to identify these elements as miscues, and wherein the user-perceptible feedback comprises providing user output with information on the miscues.

8. The method of claim 7, wherein the user-perceptible feedback comprises audible corrections to the miscues.

9. The method of claim 7, wherein the user-perceptible feedback comprises an evaluation of how well the acoustic input corresponds to the text output.

10. The method of claim 7, wherein the user-perceptible feedback comprises displaying a phonetic representation of a portion of the text output corresponding to a miscue.

11. The method of claim 7, wherein the user-perceptible feedback comprises displaying a translation into a different language of a portion of the text output for which the acoustic input includes a miscue.

12. The method of claim 7, wherein the miscues are identified with one or more miscue categories and the user-perceptible feedback is based in part on one of the miscue categories with which a miscue in the acoustic input is identified, wherein the miscue categories comprise one or more of:

word repetition, breath, partial word, pause, hesitation or elongation, wrong word, mispronunciation, background noise, interjection or insertion, non-speech sound, and hyperarticulation.

13. The method of claim 1, wherein the garbage language model comprises criteria based on an acoustic score for garbage words and a language score for garbage words, and the target language model comprises criteria based on an acoustic score for target words and a language score for target words.

14. The method of claim 13, further comprising applying a weighting parameter to the criteria for the garbage language model, and applying one minus the weighting parameter to the criteria for the target language model.

15. The method of claim 14, further comprising providing a user option to tune the weighting parameter, and responding to tuning of the weighting parameter by raising or lowering the criteria for the garbage language model.

16. The method of claim 1, wherein the garbage language model is assembled based on a restricted selection of relatively common words from a dictation grammar.

17. The method of claim 1, wherein the target language model and the garbage language model are constructed as context-free grammars.

18. The method of claim 1, wherein the garbage language model comprises an N-gram filler that is constructed as a context-free grammar.

19. A method comprising:

training a target language model specific to a limited domain of target texts;

training a garbage language model based on a general dictation grammar;

providing the trained target language model and the trained garbage language model for use in an automatic reading tutoring system that is configured to;

display portions of the target texts;

receive acoustic inputs through a configuration consistent with a user reading the displayed portions of the target texts;

model the acoustic inputs with reference to the target language model and the garbage language model to indicate whether the acoustic inputs comprise proper readings or miscues relative to the target texts; and

provide feedback via a user output based on whether the acoustic inputs are indicated to comprise proper readings or miscues relative to the target texts.

20. A medium comprising instructions that are readable and executable by a computing system, wherein the instructions configure the computing system to execute an automatic reading tutoring application comprising a series of iterations of tutoring a user in reading a subject language, wherein each of the iterations comprises:

providing a text sample comprising target words via a user-perceptible output;

calculating a language model score for the target words;

receiving an acoustic signal via a user input after the text sample is provided;

calculating an acoustic score for the target words with reference to the acoustic signal;

evaluating whether the acoustic signal comprises a miscue with reference to the text sample based on a weighted comparison of the acoustic score and language model score of the target words with an acoustic score and language model score of a set of garbage words; and

providing user-perceptible feedback based on the evaluation of whether the acoustic signal comprises a miscue.