Techniques for grammar rule composition and testing

• US 9,298,697 B2
• Filed: 01/26/2010
• Issued: 03/29/2016
• Est. Priority Date: 01/26/2010
• Status: Active Grant

• Current Assignee: University of Phoenix Incorporated
• Original Assignee: Apollo Education Group, Inc.
• Inventors: Allen, Andrew T., Myers, Ken R., Chan, John M., Bomer, Michelle N., Yackshaw, Catherine
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): AMIN, MUSTAFA A
• Application Number:
  US12/693,870
• Publication Number:
  US 20110185284A1
• Time in Patent Office: 2,254 Days
• Field of Search: 715/257
• US Class Current: 1/1
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique

Abstract

Techniques are provided for composing and testing grammar rules that are configured to provide feedback on violations of target grammar guidelines in the text of documents. Tests may be performed to determine whether a new grammar rule produces false positives, is redundant or conflicts with other rules, or is expensive to apply to a document. A user may test the rule, in isolation or with other rules, on a batch of documents. A display of a summary of feedback that is generated in the batch of documents by the rules is provided. The user may also apply a rule to documents submitted for feedback as a hidden rule. Any violations of a hidden rule are recorded, but feedback on the violations is not provided in submitted documents. Furthermore, a user may indicate a level for a rule, which may control the set of documents to which the rule is applied.

Claims

1. A computer-executed method comprising:

displaying a user interface with a control for receiving grammar rule definitions;

receiving, through the control, a first input that defines a particular grammar rule;

receiving a second input that selects a batch of documents;

wherein the batch of documents comprises at least two documents from a pool of documents;

wherein the pool of documents comprises both;

the at least two documents, and one or more other documents that are not included in the batch of documents;

based at least in part on the second input, detecting violations of the particular grammar rule within text contained in the batch of documents;

based at least in part on the second input, causing display of information, in a particular user interface, relating to the violations of the particular grammar rule;

wherein the displayed information indicates violations of the particular grammar rule in multiple documents within the batch of documents;

wherein the particular user interface includes a particular control associated with each rule violation, for indicating whether the rule violation is actually a rule violation without changing the contents of the document in which the rule violation was identified;

wherein the method is performed by one or more computing devices.

2. The computer-executed method of claim 1, further comprising:

causing display of the user interface with a second control for requesting detection of violations of the particular grammar rule within text contained in the batch of documents;

receiving, through the second control, a request to detect violations of the particular grammar rule within text contained in the batch of documents; and

only in response to the request, detecting violations of the particular grammar rule within text contained in the batch of documents.

3. The computer-executed method of claim 1, wherein detecting violations of the particular grammar rule comprises detecting violations of only the particular grammar rule.

4. The computer-executed method of claim 1, wherein the method further comprises receiving input that identifies one or more additional grammar rules;

wherein detecting violations comprises detecting violations of a plurality of grammar rules comprising the particular grammar rule and the one or more additional grammar rules;

the method further comprising causing display of information relating to the violations of each of the one or more additional grammar rules.

5. The computer-executed method of claim 4, wherein receiving input that identifies one or more additional grammar rules comprises:

causing display of information that lists a first set of grammar rules;

receiving user input that selects a second set of grammar rules;

wherein said second set of grammar rules is a subset of said first set of grammar rules; and

wherein the one or more additional grammar rules belong to said second set of grammar rules.

6. The computer-executed method of claim 4, wherein the method further comprises:

identifying a subset of the plurality of grammar rules as high-correlation grammar rules based on locations of violations of the subset of the plurality of grammar rules within the batch of documents; and

causing display of information that identifies the high-correlation grammar rules.

7. The computer-executed method of claim 1, further comprising:

causing display of information that indicates the violations of the particular grammar rule to a panel of multiple users;

receiving, from each user of the panel of multiple users, data that specifies whether a violation of the violations represents a false positive;

aggregating the data received from the panel of multiple users to produce aggregated feedback; and

causing display of information comprising the aggregated feedback.

8. The computer-executed method of claim 1, wherein the information relating to the violations of the particular grammar rule includes at least one of:

(a) one or more character sequences from the text contained in the batch of documents that violates the particular grammar rule;

(b) one or more comments associated with a violation of the particular grammar rule;

(c) one or more characters in the text contained in the batch of documents that occur proximate to one or more character sequences that violate the particular grammar rule;

or

(d) one or more attributes of the particular grammar rule.

9. The computer-executed method of claim 1, wherein the second input that selects the batch of documents identifies one or more attributes of the at least two documents of the batch of documents, wherein the one or more other documents do not have the one or more attributes.

10. The computer-executed method of claim 1, wherein the first input defines a pattern that, if present in text, would violate the particular grammar rule, wherein the pattern is based at least in part on at least two different parts in the text.

11. The computer-executed method of claim 1, wherein the first input defines a pattern that, if present in text, would violate the particular grammar rule, wherein the pattern is based at least in part on at least two different natural language words in the text.

12. The computer-executed method of claim 1, wherein the second input that identifies the batch of documents identifies how many documents, of the pool of documents, to include in the batch of documents.

13. The computer-executed method of claim 1, wherein the second input that identifies the batch of documents identifies a path expression that identifies a location of the batch of documents.

14. The computer-executed method of claim 1, wherein the information relating to the violations of the particular grammar rule includes a total run time for detecting violations of the particular grammar rule.

15. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more processors, cause:

displaying a user interface with a control for receiving grammar rule definitions;

receiving, through the control, a first input that defines a particular grammar rule;

receiving a second input that selects a batch of documents;

wherein the batch of documents comprises at least two documents from a pool of documents;

wherein the pool of documents comprises both;

the at least two documents, and

one or more other documents that are not included in the batch of documents;

based at least in part on the second input, detecting violations of the particular grammar rule within text contained in the batch of documents;

based at least in part on the second input, causing display of information, in a particular user interface, relating to the violations of the particular grammar rule;

wherein the displayed information indicates violations of the particular grammar rule in multiple documents within the batch of documents;

wherein the particular user interface includes a particular control associated with each rule violation, for indicating whether the rule violation is actually a rule violation without changing the contents of the document in which the rule violation was identified.

16. The one or more non-transitory computer-readable media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:

causing display of the user interface with a second control for requesting detection of violations of the particular grammar rule within text contained in the batch of documents;

receiving, through the second control, a request to detect violations of the particular grammar rule within text contained in the batch of documents; and

only in response to the request, detecting violations of the particular grammar rule within text contained in the batch of documents.

17. The one or more non-transitory computer-readable media of claim 15, wherein detecting violations of the particular grammar rule comprises detecting violations of only the particular grammar rule.

18. The one or more non-transitory computer-readable media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:

receiving input that identifies one or more additional grammar rules;

wherein detecting violations comprises detecting violations of a plurality of grammar rules comprising the particular grammar rule and the one or more additional grammar rules; and

causing display of information relating to the violations of each of the one or more additional grammar rules.

19. The one or more non-transitory computer-readable media of claim 18, wherein receiving input that identifies one or more additional grammar rules comprises:

causing display of information that lists a first set of grammar rules;

receiving user input that selects a second set of grammar rules;

wherein said second set of grammar rules is a subset of said first set of grammar rules; and

wherein the one or more additional grammar rules belong to said second set of grammar rules.

20. The one or more non-transitory computer-readable media of claim 18, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:

identifying a subset of the plurality of grammar rules as high-correlation grammar rules based on locations of violations of the subset of the plurality of grammar rules within the batch of documents; and

causing display of information that identifies the high-correlation grammar rules.

21. The one or more non-transitory computer-readable media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:

causing display of information that indicates the violations of the particular grammar rule to a panel of multiple users;

receiving, from each user of the panel of multiple users, data that specifies whether a violation of the violations represents a false positive;

aggregating the data received from the panel of multiple users to produce aggregated feedback; and

causing display of information comprising the aggregated feedback.

22. The one or more non-transitory computer-readable media of claim 15, wherein the information relating to the violations of the particular grammar rule includes at least one of:

(a) one or more character sequences from the text contained in the batch of documents that violates the particular grammar rule;

(b) one or more comments associated with a violation of the particular grammar rule;

(c) one or more characters in the text contained in the batch of documents that occur proximate to one or more character sequences that violate the particular grammar rule;

or

(d) one or more attributes of the particular grammar rule.

23. The one or more non-transitory computer-readable media of claim 15, wherein the second input that selects the batch of documents identifies one or more attributes of the at least two documents of the batch of documents, wherein the one or more other documents do not have the one or more attributes.

24. The one or more non-transitory computer-readable media of claim 15, wherein the first input defines a pattern that, if present in text, would violate the particular grammar rule, wherein the pattern is based at least in part on at least two different parts in the text.

25. The one or more non-transitory computer-readable media of claim 15, wherein the first input defines a pattern that, if present in text, would violate the particular grammar rule, wherein the pattern is based at least in part on at least two different natural language words in the text.

26. The one or more non-transitory computer-readable media of claim 15, wherein the second input that identifies the batch of documents identifies how many documents, of the pool of documents, to include in the batch of documents.

27. The one or more non-transitory computer-readable media of claim 15, wherein the second input that identifies the batch of documents identifies a path expression that identifies a location of the batch of documents.

28. The one or more non-transitory computer-readable media of claim 15, wherein the information relating to the violations of the particular grammar rule includes a total run time for detecting violations of the particular grammar rule.