Systems and Methods for Determining Text Complexity

• US 20140234810A1
• Filed: 02/14/2014
• Published: 08/21/2014
• Est. Priority Date: 02/15/2013
• Status: Active Grant

• Current Assignee: Educational Testing Service
• Original Assignee: Educational Testing Service
• Inventors: Flor, Michael, Beigman Klebanov, Beata
• Granted Patent: US 10,134,297 B2
• Time in Patent Office: Days
• US Class Current: 434/169
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G09B 17/003 electrically operated apparatus or devices
• CPC Class Codes: G09B 17/04 for increasing the rate of reading; Reading rate control
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Computer-implemented systems and methods are provided for determining a document'"'"'s complexity. For example, a computer performing the complexity analysis can receive a document. The computer can determine the content words within the document and determine an association measure for each group of content words. An association profile can be created for the document using the association measures. The computer can use the association profile to determine the complexity of the document. The complexity of the document may correspond to the document'"'"'s suitable reading level or, if the document is an essay, an essay score.

Claims

1. A computer-implemented method of determining a complexity of a document, comprising:

receiving the document;

identifying content words within the document;

determining an association measure for each group of content words;

creating an association profile for the document using the association measure for each group of content words;

determining the complexity of the document using the association profile.

2. The method of claim 1, wherein the document is a media file including spoken words.

3. The method of claim 1, wherein each of the content words have a type selected from the group consisting of noun, verb, adjective, and adverb.

4. The method of claim 1, wherein the content words are unique from one another.

5. The method of claim 1, wherein a group of content words includes exactly two content words.

6. The method of claim 1, further comprising:

retrieving co-occurrence data for each group of content words;

wherein determining the association measure for each group of content words includes processing the co-occurrence data associated with the group of content words.

7. The method of claim 6, wherein co-occurrence data is retrieved from a remote source.

8. The method of claim 1, wherein determining an association measure includes retrieving the association measure from a remote source.

9. The method of claim 1, wherein the association measure is based on a formula selected from the group consisting of PMI, NPMI, PNPMI, and RAS.

10. The method of claim 1, wherein the association profile includes association measures for substantially all pairs of content words in the document.

11. The method of claim 1, wherein the association profile is represented by a histogram.

12. The method of claim 1, wherein the association profile is normalized.

13. The method of claim 1, wherein the complexity of the document corresponds to a reading level of the document.

14. The method of claim 1, wherein determining the complexity of the document includes computing an average of the association measures included in the association profile.

15. The method of claim 14, wherein the average is inversely correlated to the complexity of the document.

16. The method of claim 1, wherein determining the complexity of the document includes using a readability index.

17. The method of claim 1, wherein the document is poetry and the association measure is based on a RAS formula.

18. The method of claim 1, wherein each group of content words includes words occurring anywhere in the document.

19. The method of claim 1, wherein each group of content words includes only words occurring in a paragraph of the document.

20. The method of claim 1, wherein each group of content words includes only words occurring in a sentence of the document.

21. The method of claim 1, wherein each group of content words includes only words occurring within a predetermined window of words in the document.

22. The method of claim 1, wherein the document is an essay and the complexity of the document is represented by an essay score.

23. The method of claim 22, further comprising:

determining a proportion of content word pairs in the document with association measures that fall within one or more predetermined ranges of association measures;

wherein determining the complexity of the document includes using the proportion.

24. A computer-implemented method of determining a complexity of a document, comprising:

receiving the document;

identifying content words within each segment of the document;

determining pairs of content words, wherein a pair of content words includes words from different but adjoining segments;

determining an association measure for each pair of content words;

creating an association profile for the document using the association measure for each pair of content words;

determining the complexity of the document using the association profile.

25. The method of claim 24, wherein a segment of the document is of a type selected from the group consisting of sentence and paragraph.