Method for evaluating standardized test results

• US 6,960,088 B1
• Filed: 08/28/2000
• Issued: 11/01/2005
• Est. Priority Date: 08/28/2000
• Status: Active Grant

• Current Assignee: Eliot R. Long
• Original Assignee: Eliot R. Long
• Inventors: Long, Eliot R.
• Primary Examiner(s): Banks, Derris H.
• Assistant Examiner(s): Sotomayor, John
• Application Number:
  US09/649,484
• Time in Patent Office: 1,891 Days
• Field of Search: 434/322, 434/323, 434/350, 434/118, 434/219
• US Class Current: 434/322
• CPC Class Codes: G09B 3/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

A method for evaluating standardized test results includes providing a set of class profiles, each indicative of performance of a respective class for a selected test on a question-by-questions basis. One or more normative profiles are generated from these class profiles, and each normative profile is indicative of expected class performance for the test on a question-by-question basis. Individual ones of the class profiles that deviate excessively from the normative profiles on a question-by-question basis are then identified and used as an indication that the test was improperly administered for the respective class.

Claims

1. A method for evaluating group-administered test results, said method comprising:

(a) providing a set of class profiles by test administrator, each class profile indicative of the question by question performance of a respective class on a selected group-administered test comprising a plurality of questions which are directed to measure the same characteristic;

(b) generating with a computer a normative profile indicate of normative class performance on the selected test, where the members of each class included in the normative profile are subject to the direction of the same test administrator on the selected test;

(c) comparing each of the class profiles with the normative profile on a question-by-question basis; and

(d) generating a measure of the question by question variation of each of the class profiles from the normative profile.

2. The method of claim 1 wherein act (c) comprises identifying selected ones of the class profiles that are characterized by a low correlation with the normative profile.

3. The method of claim 1 wherein act (c) comprises comparing (1) at least two portions of each class profile that differ in question difficulty against (2) corresponding portions of the normative profile.

4. The method of claim 1 wherein act (c) comprises comparing (1) a difference between at least a portion of each of the class profiles and a corresponding portion of the normative profile on a question-by-question basis;

and (2) a measure of expected variability of class performance for the corresponding portion of the class profiles.

5. The method of claim 1 wherein act (c) comprises comparing (1) a measure of difference between a first portion of each of the class profiles and a corresponding first portion of the normative profile and (2) a measure of difference between a second portion of each of the class profiles and a corresponding second portion of the normative profile.

6. The method of claim 1 wherein act (c) comprises comparing the frequency of successive equal measures of class success in answering the questions on a question-by-question basis.

13. The method of claim 1 or 7 wherein act (a) comprises:

(a1) providing a respective individual profile for each of a plurality of individuals that took the group-administered test, each individual associated with a respective one of the classes, each individual profile indicative of accuracy of the respective individual in answering the plurality of questions on a question-by-question basis; and

(a2) determining each class profile from the individual profiles of the individuals in the respective class, each class profile indicative of accuracy of the individuals in the respective class in answering the questions on a question-by-question basis.

14. The method of claim 1 or 7 wherein the class profiles of (a) are each characterized by a skill level, and wherein the normative profile of (b) is indicative of expected class performance for a class of said skill level.

15. The method of claim 1 or 7 wherein the normative profile of (b) is indicative of expected class performance for classes of a wide range of skill levels.

7. A method for evaluating group-administered test results, said method comprising:

(a) providing a set of class profiles by test administrator, each class profile indicative of the question by question performance of a respective class on a selected group-administered test comprising a plurality of questions which are directed to measure the same characteristic;

(b) generating with a computer a normative profile indicative of normative class performance on the selected test where the members of each class included in the normative profile are subject to the direction of the same test administrator on the selected test;

(c) generating a measure of the question by question variation of each of the class profiles from the normative profile; and

(d) identifying individual ones of the class profiles that deviate excessively from the normative profile on a question-by-question basis.

8. The method of claim 7 wherein act (d) comprises identifying selected ones of the class profiles that are characterized by a low correlation with the normative profile.

9. The method of claim 7 wherein act (d) comprises comparing (1) at least two portions of each class profile that differ in question difficulty against (2) corresponding portions of the normative profile.

10. The method of claim 7 wherein act (d) comprises comparing (1) a difference between at least a portion of each of the class profiles and a corresponding portion of the normative profile on a question-by-question basis;

and (2) a measure of expected variability of class performance for the corresponding portion of the class profiles.

11. The method of claim 7 wherein act (d) comprises comparing (1) a measure of difference between a first portion of each of the class profiles and a corresponding first portion of the normative profile and (2) a measure of difference between a second portion of each of the class profiles and a corresponding second portion of the normative profile.

12. The method of claim 7 wherein act (d) comprises comparing the frequency of successive equal measures of class success in answering the questions on a question-by-question basis.

16. A method for evaluating standardized test results, said method comprising:

(a) providing a set of sub-group profiles, each sub-group comprising a plurality of individuals whose test question responses are exposed to a common element affecting test administration and each sub-group profile indicative of the question by question performance of a respective sub-group on a selected standardized test comprising a plurality of questions which are directed to measure the same characteristic;

(b) generating with a computer a normative profile indicative of normative sub-group performance on the selected test; and

(c) evaluating administration of the selected test based on a comparison of each of the sub-group profiles from the normative profile on a question-by-question basis.

17. The method of claim 16 wherein each sub-group comprises individuals to whom the selected test was administrated by a respective test administrator in a respective testing session.

18. The method of claim 16 or 17 wherein act (c) comprises identifying selected ones of the sub-group profiles that are characterized by a low correlation with the normative profile.

19. The method of claim 16 or 17 wherein act (c) comprises comparing (1) at least two portions of each sub-group profile that differ in question difficulty against (2) corresponding portions of the normative profile.

20. The method of claim 16 or 17 wherein act (c) comprising comparing (1) a difference between at least a portion of each of the sub-group profiles and a corresponding portion of the normative profile on a question-by-question basis;

and (2) a measure of expected variability of sub-group performance for the corresponding portion of the sub-group profiles.

21. The method of claim 16 or 17 wherein act (c) comprises comparing (1) a measure of difference between a first portion of each of the sub-group profiles and a corresponding first portion of the normative profile and (2) a measure of difference between a second portion of each of the sub-group profiles and a corresponding second portion of the normative profile.

22. The method of claim 16 or 17 wherein act (c) comprises comparing the frequency of successive equal measures of class success in answering the questions on a question-by-question basis.

23. The method of claim 16 or 17 wherein act (a) comprises:

(a1) providing a respective individual profile for each of a plurality of individuals that took the standardized test each individual associated with a respective one of the sub-groups, each individual profile indicative of accuracy of the respective individual in answering the plurality of questions on a question-by-question basis; and

(a2) determining each sub-group profile from the individual profiles of the individuals in the respective sub-group, each sub group profile indicative of accuracy of the individuals in the respective sub-group in answering the questions on a question-by-question basis.

24. The method claim 16 or 17 wherein the sub-group profiles of (a) are each character by a skill level, and wherein the normative profile of (b) is indicative of expected sub-group performance for a sub-group of said skill level.

25. The method of claim 16 or 17 wherein the normative profile of (b) is indicative of expected sub-group performance for sub-groups of a wide range of skill levels.