Language capability assessment and training apparatus and techniques

• US 20070122792A1
• Filed: 11/09/2005
• Published: 05/31/2007
• Est. Priority Date: 11/09/2005
• Status: Active Grant

• Current Assignee: SDL Inc (RWS Holdings Plc)
• Original Assignee: SDL Inc (RWS Holdings Plc)
• Inventors: Marcu, Daniel, Knight, Kevin, Galley, Michel
• Granted Patent: US 10,319,252 B2
• Time in Patent Office: Days
• US Class Current: 434/353
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

A learning system for a text-to-text application such as a machine translation system. The system has questions, and a matrix of correct answers to those questions. Any of the many different correct answers within the matrix can be considered as perfectly correct answers to the question. The system operates by displaying a question, which may be a phrase to be translated, and obtaining an answer to the question from the user. The answer is compared against the matrix and scored. Feedback may also be provided to the user.

Claims

1. A method comprising:

accepting input from a user representing the user'"'"'s answer to a text to text based question, where there are many different correct answers, each of which is completely correct; and

accessing a database which includes said many different correct answers in a packed format where at least a plurality of said answers use common information to represent parts of said answers; and

determining an error between said input and any of said correct answers.

2. A method as in claim 1, wherein said determining an error comprises scoring the answers according to a kind of said error.

3. A method as in claim 1, wherein said scoring the answers comprises counting an error only once even when multiple parts are affected by said error.

4. A method as in claim 1, wherein said text to text based question comprises a question about translation between a first language and a second language.

5. A method as in claim 1, wherein said text to text based question comprises pronunciation of speech.

6. A method as in claim 1, further comprising determining, from said error, feedback to help the user in understanding more about said error.

7. A method as in claim 1, further comprising producing an answer matrix which includes each of said many different correct answers, prior to said accessing.

8. A method as in claim 7, wherein said producing an answer matrix comprises forming a user interface which facilitates entry of the many different answers.

9. A method as in claim 8, wherein said user interface includes an associated tool that tests certain ones of said many different answers.

10. A method as in claim 1, wherein said packed format is a recursive translation network.

11. A method as in claim 1, further comprising analyzing said answer by maintaining a list including most prominent single hypotheses through the packed database, and following the most promising hypothesis.

12. A method as in claim 3, wherein said errors include word insertion errors, Word deletion errors, word substitution errors, word permutation errors, word stemming errors, and paraphrasing errors.

17. A method as in claim 11, wherein said determining comprises following a most promising path through the matrix of correct answers.

13. A method comprising:

displaying text in a first language to a user;

accepting input of the text in a second language from the users as a translation;

comparing said translation to a matrix of prestored answers;

determining either an exact match for a closest match between the translation and any answer in the matrix; and

providing a score indicative of either the closest match or the exact match.

14. A method as in claim 13, further comprising providing feedback to the user indicating information about errors made in said translation.

15. A method as in claim 14, wherein said providing a score comprises scoring each of a plurality of errors as being a single error even when said error affects more than one word.

16. A method as in claim 15, wherein said scoring comprises continued considering a single error for each of the word insertion, word deletion, word substitution, word permutation, word stemming, and paraphrasing.

32. An apparatus as in claim 14, wherein said score scores each of a plurality of errors as being a single error even when said error affects more than one word.

18. An apparatus comprising:

an interface part operating to accept input from a user representing the user'"'"'s answer to a text to text based question, where there are many different correct answers, each of which is completely correct;

a database which includes said many different correct answers in a packed format, where at least a plurality of said answers rely on common information for parts of said answers; and

a machine, which operates to determine an error between said input and said many different correct answers by identifying a closest correct answer to said input, and identifying an error between said closest answer and said input.

19. An apparatus as in claim 18, wherein said machine scores the answers according to a kind of said error.

20. An apparatus method as in claim 18, wherein said machine scores the answers comprises counting an error only once even when multiple parts are affected by said error.

21. An apparatus as in claim 18, wherein said text to text based question comprises a question about translation between a first language and a second language.

22. An apparatus as in claim 18, wherein said text to text based question comprises pronunciation of speech.

23. An apparatus as in claim 18, wherein said machine determines, from said error, feedback to help the user in understanding more about said error.

24. An apparatus as in claim 18, wherein said machine produces an answer matrix which includes each of said many different correct answers, prior to said accessing.

25. An apparatus as in claim 24, wherein said machine operates to form a answer matrix user interface that allows entry of details of said answer matrix, and which facilitates entry of the many different answers.

26. An apparatus as in claim 25, wherein said answer matrix user interface includes an associated tool that tests certain ones of said many different answers.

27. An apparatus as in claim 18, wherein said packed format is a recursive translation network.

28. An apparatus as in claim 18, wherein said machine operates by analyzing said answer by maintaining a list including a most prominent single hypotheses through the packed database, and following the most promising hypothesis.

29. An apparatus as in claim 20, wherein said errors include word insertion errors, Word deletion errors, word substitution errors, word permutation errors, word stemming errors, and paraphrasing errors.

30. An apparatus comprising:

a machine that includes a memory storing a matrix of prestored answers to questions, said machine operating to produce signals indicative of a user interface that operates to display text in a first language to a user, and accepts input of the text in a second language from the user as a translation, said machine determining either an exact match for a closest match between the translation and any answer in the matrix, and providing a score indicative of a relationship between said input of said text, and any of said answers.

31. An apparatus as in claim 30, wherein said machine further provides feedback to the user indicating information about errors made in said translation.