Reading and pronunciation tutor

• US 5,920,838 A
• Filed: 06/02/1997
• Issued: 07/06/1999
• Est. Priority Date: 06/02/1997
• Status: Expired due to Term

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Mostow, Jack, Aist, Gregory S.
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US08/863,505
• Time in Patent Office: 764 Days
• Field of Search: 704/270, 704/255, 704/246, 704/256, 704/266, 704/272, 704/251
• US Class Current: 704/255
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

A computer implemented reading tutor comprises a player for outputting a response. An input block implementing a plurality of functions such as silence detection, speech recognition, etc. captures the read material. A tutoring function compares the output of the speech recognizer to the text which was supposed to have been read and generates a response, as needed, based on information in a knowledge base and an optional student model. The response is output to the user through the player. A quality control function evaluates the captured read material and stores the captured material in the knowledge base under certain conditions. An auto-enhancement function uses information available to the tutor to create additional resources such as identifying rhyming words, words with common roots, etc., which can be used as responses.

Claims

1. A computer implemented reading tutor, comprising:

means for outputting a response;

means for capturing and recognizing read material;

means for evaluating the captured read material and for generating the response, said response being input to said means for outputting; and

quality control means,

said means for evaluating the captured read material additionally causing said captured read material to be stored as a response in response to said quality control means.

2. The computer implemented reading tutor of claim 1 wherein said quality control means applies a confidence threshold based on a characteristic of a speaker.

3. The computer implemented reading tutor of claim 1 wherein said quality control means applies a confidence threshold based on whether the device is in an authoring mode or a tutoring mode.

4. The computer implemented reading tutor of claim 1 wherein said quality control means compares a rate of reading the captured read material to a predetermined rate of reading.

6. The computer implemented reading tutor of claim 2 wherein said quality control means applies a confidence threshold based on a characteristic of a speaker.

7. The computer implemented reading tutor of claim 2 wherein said quality control means applies a confidence threshold based on whether the device is in an authoring mode or a tutoring mode.

8. The computer implemented reading tutor of claim 2 wherein said quality control means compares a rate of reading the captured read material to a predetermined rate of reading.

9. The computer implemented reading tutor of claim 2 wherein the information in said knowledge base includes the text of read material, at least certain of said text being associated with its own key, the information in said knowledge base additionally including a plurality of resources, said resources being associated with text through said keys.

5. In combination, a computer implemented reading tutor and a knowledge base, comprising:

a knowledge base;

means for outputting a response;

means for capturing and recognizing read material;

means for evaluating the captured read material and for generating the response based on information in said knowledge base, said response being input to said means for outputting; and

quality control means,

said means for evaluating the captured read material additionally causing said captured read material to be stored in said knowledge base as a response in response to said quality control means.

10. A reading tutor system, comprising:

a knowledge base;

a first circuit for capturing and recognizing read material from speech;

a second circuit for comparing said captured material with the text of the read material and for generating a response based on said comparison and information in said knowledge base;

a third circuit for outputting said response;

a fourth circuit for evaluating said captured material for possible storage in said knowledge base; and

a fifth circuit for storing certain of said captured material in said knowledge base based on said evaluation.

11. The reading tutor system of claim 10 wherein said fourth circuit is responsive to one of an authoring and tutoring modes.

12. The reading tutor system of claim 10 wherein the information in said knowledge base includes the text of read material, at least certain of said text being associated with its own key, the information in said knowledge base additionally including a plurality of resources, said resources being associated with text through said keys.

13. A reading tutor system, comprising:

a processor;

at least one input device and one output device in communication with said processor; and

a memory, coupled to said processor, and storing a set of ordered data and a set of instructions which, when executed by said processor, cause said processor to perform the steps of;

capturing and recognizing read material from speech;

comparing said captured material with the text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said captured material for possible storage in said knowledge base; and

storing certain of said captured material in said knowledge base based on said evaluating step.

14. The system of claim 13 wherein the information in said knowledge base includes the text of read material, at least certain of said text being associated with its own key, the information in said knowledge base additionally including a plurality of resources, said resources being associated with text through said keys.

15. The system of claim 14 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of expanding said knowledge base.

16. The system of claim 15 wherein said memory stores instructions which, when executed by said processor, cause said processor to perform said step of expanding said knowledge base by performing the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

17. The system of claim 16 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of associating said added resources with keys.

18. The system of claim 13 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of generating a student model.

19. A tutoring module, comprising:

a first sequence of instructions for receiving read material from an input module;

a second sequence of instructions for evaluating said read material;

a third sequence of instructions for generating a response to said read material in response to said second sequence of instructions and information in a knowledge base; and

a fourth sequence of instructions for automatically adding said read material to said knowledge base as a response when predetermined conditions are met.

20. The module of claim 19, further comprising a fifth sequence of instructions for generating a student model, and wherein said third sequence of instructions is additionally responsive to said student model.

21. The module of claim 19, further comprising a fifth sequence of instructions for controlling a player module.

22. The module of claim 19, further comprising a fifth sequence of instructions for enabling information to be manually added to said knowledge base.

23. A computer-readable medium having stored thereon instructions which, when executed by a processor, cause said processor to perform the steps of:

capturing and recognizing read material from a speech signal;

comparing said captured material with the text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said captured material for possible storage in said knowledge base; and

storing certain of said captured material in said knowledge base based on said evaluation step.

24. The medium of claim 23 having stored thereon instructions which, when executed by said processor, cause said processor to perform the additional step of expanding said knowledge base.

25. The medium of claim 24 having stored thereon instructions which, when executed by said processor, cause said processor to perform said step of expanding said knowledge base by performing the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

26. The medium of claim 24 having stored thereon additional instructions which, when executed by said processor, cause said processor to perform the additional step of associating a key with text stored in said knowledge base.

27. The medium of claim 26 having stored thereon additional instructions which, when executed by said processor, cause said processor to perform the additional step of associating said added resources with keys.

28. The medium of claim 23 having stored thereon additional instructions which, when executed by processor, cause said processor to perform the additional step of generating a student model.

29. A computer-implemented method for responding to speech, comprising the steps of:

capturing read material from the speech;

recognizing said read material;

comparing said recognized material with text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said recognized material for storage in said knowledge base; and

storing certain of said recognized material in said knowledge base based on said evaluation step.

30. The method of claim 29 further comprising the step of expanding said knowledge base.

31. The method of claim 30 wherein said step of expanding said knowledge base includes the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

32. The method of claim 31 further comprising the step of associating a key with each of the resources added to said knowledge base.

33. The method of claim 29 further comprising the step of generating a student model, and wherein said step of generating a response is further responsive to said student model.

34. A reading tutor system, comprising:

a processor;

at least one input device and one output device in communication with said processor; and

a memory, coupled to said processor, and storing a set of ordered data and a set of instructions which, when executed by said processor, cause said processor to perform the steps of;

capturing and recognizing read material from speech;

comparing said captured material with the text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said captured material for possible storage in said knowledge base, said evaluating including;

marking portions of said material that have no recorded spoken pronunciations;

selecting portions of said material, including certain of said marked portions of said material;

capturing spoken pronunciations of said selected portions;

verifying that said spoken pronunciations match said selected portions;

rejecting said spoken pronunciations that do not match said selected portions;

unmarking said marked portions of said material; and

storing certain of said captured material in said knowledge base based on said evaluating step, said storing including storing said spoken pronunciations that match said selected portions in said knowledge base.

35. The system of claim 34 wherein the information in said knowledge base includes the text of read material, at least certain of said text being associated with its own key, the information in said knowledge base additionally including a plurality of resources, said resources being associated with text through said keys.

36. The system of claim 35 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of expanding said knowledge base.

37. The system of claim 36 wherein said memory stores instructions which, when executed by said processor, cause said processor to perform said step of expanding said knowledge base by performing the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

38. The system of claim 37 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of associating said added resources with keys.

39. The system of claim 34 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional step of generating a student model.

40. The system of claim 34 wherein said memory stores an additional set of instructions which, when executed by said processor, cause said processor to perform the additional steps of:

playing certain of said recorded spoken pronunciations; and

receiving a response denoting the acceptability of said played spoken pronunciations.

41. A computer-readable medium having stored thereon instructions which, when executed by a processor, cause said processor to perform the steps of:

capturing and recognizing read material from a speech signal;

comparing said captured material with the text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said captured material for possible storage in said knowledge base, said evaluating including;

marking portions of said material that have no recorded spoken pronunciations;

selecting portions of said material, including certain of said marked portions of said material;

capturing spoken pronunciations of said selected portions;

verifying that said spoken pronunciations match said selected portions;

rejecting said spoken pronunciations that do not match said selected portions;

unmarking said marked portions of said material; and

storing certain of said captured material in said knowledge base based on said evaluation step, said storing including storing said spoken pronunciations that match said selected portions in said knowledge base.

42. The medium of claim 41 having stored thereon instructions which, when executed by said processor, cause said processor to perform the additional step of expanding said knowledge base.

43. The medium of claim 42 having stored thereon instructions which, when executed by said processor, cause said processor to perform said step of expanding said knowledge base by performing the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

44. The medium of claim 42 having stored thereon additional instructions which, when executed by said processor, cause said processor to perform the additional step of associating a key with text stored in said knowledge base.

45. The medium of claim 44 having stored thereon additional instructions which, when executed by said processor, cause said processor to perform the additional step of associating said added resources with keys.

46. The medium of claim 41 having stored thereon additional instructions which, when executed by processor, cause said processor to perform the additional step of generating a student model.

47. The medium of claim 41 having stored thereon additional instructions which, when executed by said processor, cause said processor to perform the additional steps of:

playing certain of said recorded spoken pronunciations; and

receiving a response denoting the acceptability of said played spoken pronunciations.

48. A computer-implemented method for responding to speech, comprising the steps of:

capturing read material from the speech;

recognizing said read material;

comparing said recognized material with text of the read material;

generating a response based on said comparing step and information in a knowledge base;

outputting said response;

evaluating said recognized material for storage in said knowledge base, said evaluating including;

marking portions of said material that have no recorded spoken pronunciations;

selecting portions of said material, including certain of said marked portions of said material;

capturing spoken pronunciations of said selected portions;

verifying that said spoken pronunciations match said selected portions;

rejecting said spoken pronunciations that do not match said selected portions;

umarking said marked portions of said material; and

storing certain of said recognized material in said knowledge base based on said evaluation step, said storing including storing said spoken pronunciations that match said selected portions in said knowledge base.

49. The method of claim 48 further comprising the step of expanding said knowledge base.

50. The method of claim 49 wherein said step of expanding said knowledge base includes the steps of:

adding text to said knowledge base; and

adding resources to said knowledge base.

51. The method of claim 50 further comprising the step of associating a key with each of the resources added to said knowledge base.

52. The method of claim 48 further comprising the step of generating a student model, and wherein said step of generating a response is further responsive to said student model.

53. The method of claim 48 further comprising the steps of:

playing certain of said recorded spoken pronunciations; and

receiving a response denoting the acceptability of said played spoken pronunciations.