SYSTEM AND METHOD FOR IMPROVING THE QUALITY OF COMPUTER GENERATED EXAMS

• US 20100203492A1
• Filed: 04/24/2009
• Published: 08/12/2010
• Est. Priority Date: 10/25/2006
• Status: Active Grant

• Current Assignee: Reliant Exams AS
• Original Assignee: Reliant Exams AS
• Inventors: Mikkelsen, Eirik, Nibe, Kjell Bjornar
• Granted Patent: US 10,504,376 B2
• Time in Patent Office: Days
• US Class Current: 434/362
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

System and method for generating exams, questionnaires or similar, including a set of questions to be answered, the system comprising a data base including a number of questions in at least one topic, each question being associated with a data set related to answers given to the questions in previous uses of the question sets, the system further comprising selection means for randomly selecting a question within one or more predetermined topics from said data base, and an evaluation means for evaluating the selected questions relative to predetermined requirements to the selected questions and their corresponding answers in said data set, and discarding questions not fulfilling said requirements.

Claims

1. System for generating exams, questionnaires or similar, including a set of questions to be answered,

the system comprising a data base including a number of questions in at least one topic, each question being associated with a data set related to answers given to the questions in previous uses of the question sets,

said database is at least one data storage device,

the system further comprising selection means for randomly selecting a question within one or more predetermined topics from said data base,

and an evaluation means for evaluating the selected questions relative to predetermined requirements to the selected questions and their corresponding answers in said data set, and discarding questions not fulfilling said requirements.

2. System according to claim 1,

wherein the data set comprises information regarding the quality of the answers to each question in a predetermined scale.

3. System according to claim 2,

wherein the quality information also comprises the deviation in said quality of the answers to the question and the quality of the complete data set in at least one previous use.

4. System according to claim 1,

wherein the information also comprises the ratio of correct answers to each question.

5. System according to claim 1,

wherein the evaluation means is adapted to reject questions not fulfilling the predetermined requirements,

said requirements being related to at least one of the following:

correlation between the quality of the responses given to the questions and the general quality of the response to all the questions in which the answer is provided, so as to ensure that the questions distinguish the generally good candidates,

deviation in the quality being above a predetermined value, so as to ensure that the question is suitable for distinguishing between the candidates,

the number of correct answers per failure in the question fits with a predetermined distribution, so as to ensure a controlled distribution of the difficulty of the questions in the exam to be generated.

6. System according to claim 1,

wherein the system is arranged to use the Kuder-Richardsons measure (KR 20) or the Cronbachs Alpha measure to measure the overall test-quality of the question item set comprising previously used question items.

7. System according to claim 1,

wherein the system is arranged to use the Spearman-Brown prediction formula to measure the quality of a set of items of known and unknown quality:

8. Method for generating exams, questionnaires or similar, the method comprising the steps of:

storing a number of questions in at least one topic a data base, each question being associated with a data set related to answers given to the questions in previous exams, said database being at least one data storage device,

randomly selecting a question within one or more predetermined topics from said data base,

evaluating the selected questions relative to predetermined requirements corresponding to said data set and discarding questions not fulfilling said requirements.

9. Method according to claim 8,

wherein the data set comprises information regarding the quality of the responses to each question in a predetermined scale.

10. Method according to claim 9,

wherein the quality information also comprises the deviation in said quality and the quality of the complete exam of which each answer was a part.

11. Method according to claim 8,

wherein the information also comprises the ratio of correct answers to each question.

12. Method according to claim 8,

wherein the evaluation means is adapted to reject questions not fulfilling the predetermined requirements,

said requirements being related to at least one of the following:

correlation between the quality of the responses given to the questions and the general quality of the exam in which the answer is provided, so as to ensure that the questions distinguish the generally good candidates,

deviation in the quality being above a predetermined value, so as to ensure that the question is suitable for distinguishing between the candidates,

the number of correct answers per failure in the question fits with a predetermined distribution, so as to ensure a controlled distribution of the difficulty of the questions in the exam to be generated.

13. Method according to claim 8,

wherein the method uses the Kuder-Richardsons measure (KR 20) or the Cronbachs Alpha measure to measure the overall test-quality of the question item set comprising previously used question items.

14. Method according to claim 8,

wherein the method uses the Spearman-Brown prediction formula to measure the quality of a set of items of known and unknown quality: