Intelligent tutoring feedback

• US 20060069561A1
• Filed: 09/10/2004
• Published: 03/30/2006
• Est. Priority Date: 09/10/2004
• Status: Active Grant

• Current Assignee: Scientific Learning Corp. (Carnegie Learning, Inc.)
• Original Assignee: Scientific Learning Corp. (Carnegie Learning, Inc.)
• Inventors: Adams, Marilyn Jager, Beattie, Valerie L., Barrow, Michael
• Granted Patent: US 8,109,765 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Methods and related computer program products, systems, and devices for providing intelligent feedback to a user based on audio input associated with a user reading a passage are disclosed. The method can include assessing a level of fluency of a user'"'"'s reading of the sequence of words using speech recognition technology to compare the audio input with an expected sequence of words and providing feedback to the user related to the level of fluency for a word.

Claims

1. A computer-based method comprising:

receiving audio input associated with a user reading a sequence of words displayed on a graphical user interface;

assessing a level of fluency and pronunciation accuracy of a user'"'"'s reading of the sequence of words using speech recognition technology to compare the audio input with an expected sequence of words; and

providing feedback to the user related to the level of fluency and pronunciation accuracy for a word, wherein the feedback includes;

providing immediate feedback if the level of fluency and pronunciation accuracy for a word does not meet a first set of criteria;

providing deferred feedback to the user if the level of fluency and pronunciation accuracy for a word meets the first set of criteria, but does not meet a second set of fluency and pronunciation accuracy criteria;

providing deferred feedback to the user if immediate feedback was given for the word, depending on the word'"'"'s category and type of immediate feedback given.

2. The method of claim 1 wherein providing immediate feedback includes providing an intervention if a specified time period since identifying the previous word in the sequence of words in the audio has elapsed and the speech recognition process has not identified the word in the audio.

3. The method of claim 1 wherein providing deferred feedback to the user includes:

representing the user'"'"'s pronunciation of a word by an acoustic match confidence level; and

providing feedback if the acoustic match confidence level is below a specified target level.

4. The method of claim 1 wherein providing deferred feedback to the user includes providing feedback based on the elapsed time before, during, and/or after the audio identified as a word.

5. The method of claim 1 wherein the criteria for providing immediate and deferred feedback for a word are dependent on a word category associated with the word.

6. The method of claim 1 wherein the immediate feedback is a visual intervention which includes visual indications provided on the graphical user interface, and which does not include audio indications.

7. The method of claim 1 wherein the immediate feedback is an audio intervention which includes an audio indication.

8. The method of claim 6 wherein the visual indication includes highlighting the word on the user interface.

9. The method of claim 6 wherein the visual indications include:

coloring the text prior to the word in the passage in a first color; and

coloring the word and text subsequent to the word in the sentence in a second color, the first color being different from the second color.

10. The method of claim 7 wherein the audio indication includes a pronunciation of the word.

11. The method of claim 1 wherein a visual intervention is provided based on the user'"'"'s first attempt to read a word and an audio intervention is provided based on the user'"'"'s second attempt to read the word.

12. The method of claim 11 further comprising:

providing the user a third opportunity to read the word subsequent to the audio intervention; and

providing a visual indication on the graphical user interface to indicate that the user should continue reading with the subsequent word if the fluency and pronunciation accuracy criteria are not met.

13. The method of claim 1 wherein the deferred feedback includes coloring the sequence of words read by the user using different colors on the graphical user interface.

14. The method of claim 13 further comprising:

providing a color based indication for words which met both sets of fluency and pronunciation accuracy criteria using a first color;

providing a color based indication for words which received an audio intervention using a second color; and

providing a color based indication for words which did not receive an audio intervention and did not meet the second set of fluency and pronunciation accuracy criteria using a third color.

15. The method of claim 14 further comprising providing a color based indication for words which received a visual intervention using the third color.

16. The method of claim 1 wherein providing deferred feedback includes placing words on a review list.

17. The method of claim 16 wherein words which received an audio intervention are placed on a review list.

18. The method of claim 16 wherein words which received a visual intervention are placed on a review list.

19. The method of claim 1 further comprising assessing the fluency and pronunciation accuracy of the user'"'"'s reading on a word-by-word basis, including:

generating user statistics and performance measurements for both progress reporting and dynamic adjustment of the tutoring software settings for that user;

generation of lists of words which the user has mastered;

measuring the user'"'"'s base or “

trouble-free”

reading rate.

20. A computer program product residing on a computer readable medium comprising instructions for causing an electrical device to:

receive audio input associated with a user reading a sequence of words displayed on a graphical user interface;

assess a level of fluency and pronunciation accuracy of a user'"'"'s reading of the sequence of words using speech recognition technology to compare the audio input with an expected sequence of words; and

provide feedback to the user related to the level of fluency and pronunciation accuracy for a word, wherein the feedback includes;

provide immediate feedback if the level of fluency and pronunciation accuracy for a word does not meet a first set of criteria;

provide deferred feedback to the user if the level of fluency and pronunciation accuracy for a word meets the first set of criteria, but does not meet a second set of fluency and pronunciation accuracy criteria;

provide deferred feedback to the user if immediate feedback was given for the word, depending on the word'"'"'s category and type of immediate feedback given.

21. The computer program product of claim 20 further comprising instructions for causing an electrical device to:

represent the user'"'"'s pronunciation of a word by an acoustic match confidence level; and

provide feedback if the acoustic match confidence level is below a specified target level.

22. The computer program product of claim 20 wherein the immediate feedback is a visual intervention which includes visual indications provided on the graphical user interface, and which does not include audio indications.

23. The computer program product of claim 20 further comprising instructions for causing an electrical device to:

color the text prior to the word in the passage in a first color; and

color the word and text subsequent to the word in the sentence in a second color, the first color being different from the second color.

24. The computer program product of claim 20 wherein the visual intervention is provided based on the user'"'"'s first attempt to read a word and the audio intervention is provided based on the user'"'"'s second attempt to read the word.

25. The computer program product of claim 20 further comprising instructions for causing an electrical device to:

provide a color based indication for words which met both sets of fluency and pronunciation accuracy criteria using a first color;

provide a color based indication for words which received an audio intervention using a second color; and

provide a color based indication for words which did not receive an audio intervention and did not meet the second set of fluency and pronunciation accuracy criteria using a third color.

26. The computer program product of claim 20 wherein words which received an audio intervention are placed on a review list.

27. The computer program product of claim 20 wherein words which received a visual intervention are placed on a review list

28. A device configured to:

receive audio input associated with a user reading a sequence of words displayed on a graphical user interface;

assess a level of fluency and pronunciation accuracy of a user'"'"'s reading of the sequence of words using speech recognition technology to compare the audio input with an expected sequence of words; and

provide feedback to the user related to the level of fluency and pronunciation accuracy for a word, wherein the feedback includes;

provide immediate feedback if the level of fluency and pronunciation accuracy for a word does not meet a first set of criteria;

provide deferred feedback to the user if the level of fluency and pronunciation accuracy for a word meets the first set of criteria, but does not meet a second set of fluency and pronunciation accuracy criteria;

provide deferred feedback to the user if immediate feedback was given for the word, depending on the word'"'"'s category and type of immediate feedback given.

29. The device of claim 28 further configured to:

represent the user'"'"'s pronunciation of a word by an acoustic match confidence level; and

provide feedback if the acoustic match confidence level is below a specified target level.

30. The device of claim 28 further configured to:

color the text prior to the word in the passage in a first color; and

color the word and text subsequent to the word in the sentence in a second color, the first color being different from the second color.

31. The device of claim 28 further configured to:

provide a color based indication for words which met both sets of fluency and pronunciation accuracy criteria using a first color;

provide a color based indication for words which received an audio intervention using a second color; and

provide a color based indication for words which did not receive an audio intervention and did not meet the second set of fluency and pronunciation accuracy criteria using a third color.

32. The device of claim 28 further configured to place words on a review list based on an audio or visual intervention.

33. The device of claim 28 further configured to generate review exercises for the user based on the review list.