MEASURING LANGUAGE LEARNING USING STANDARDIZED SCORE SCALES AND ADAPTIVE ASSESSMENT ENGINES

• US 20180130375A1
• Filed: 11/08/2016
• Published: 05/10/2018
• Est. Priority Date: 11/08/2016
• Status: Active Grant

• Current Assignee: Pearson Education Incorporated (Pearson plc)
• Original Assignee: Pearson Education Incorporated (Pearson plc)
• Inventors: Booth, David, Seddon, Paul
• Granted Patent: US 10,319,255 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G09B 5/08 providing for individual presentation of information to a plurality of student stations
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation
• CPC Class Codes: G09B 7/06 of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers
• CPC Class Codes: G09B 7/08 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying further information

Abstract

The present invention allows a language learning platform to calculate a user'"'"'s overall language skill. This may be performed as a standalone test or before, during and/or after a language course. The language learning platform may perform a Pre-CAT process, a CAT process, a stage 2—fixed form process and a scoring and reporting process. The Pre-CAT process is designed to quickly get an initial estimate of the user'"'"'s overall language skill. The CAT processes improves on the accuracy of the Pre-CAT process, preferably until the conditional standard error of measurement for the user'"'"'s overall language skill is less than a predetermined number. The stage 2—fixed form process may use the user'"'"'s overall language skill to select speaking, listening and writing assessment items of an appropriate difficulty for the user. The language learning platform may score the responses of the user, present the measured user'"'"'s overall language skill to the user and save the user'"'"'s overall language skill in a database for use in future tests.

Claims

1. A system for adaptively assessing a language learning of a user configured to:

receive a user identification from the user using a user interface displayed on a client device;

search in a first database for a history of the user associated with the user identification;

estimate an overall language skill of the user based on the history in the first database of the user, a difficulty level of a product being used by the user and/or responses to an initial plurality of assessment items displayed on the user interface to the user;

retrieve a random item pattern comprising a sequence of question types, wherein at least one question type is a multiple choice type of question and at least one question type is a fill in the blank type of question;

sequentially for each item in the random item pattern;

select a first next question type in the random item pattern,

select a first assessment item from a second database of a plurality of calibrated assessment items, wherein the first assessment item matches the first next question type in the random item pattern and matches the overall language skill of the user,

modify the user interface to present the first assessment item to the user,

generate a first score for a first response from the user to the first assessment item, and

update the overall language skill of the user and a conditional standard error of measurement for the overall language skill of the user based on the first score;

select one or more fixed forms for speaking, listening and writing based on the overall language skill of the user;

modify the user interface to present the one or more fixed forms for speaking, listening and writing to the user;

generate a third score for a third response to the one or more fixed forms for speaking, listening and writing;

update the overall language skill of the user and the conditional standard error of measurement of the overall language skill of the user based on the third score for the third response to the one or more fixed forms for speaking, listening and writing;

modify the user interface to notify the user of the overall language skill of the user; and

record the overall language skill to the first database.

2. The system of claim 1, further configured to:

before the estimate the overall language skill of the user, retrieve a difficulty level of the product being used by the user.

3. The system of claim 1, wherein the overall language skill is expressed on a logarithmic scale.

4. The system of claim 1, wherein the second database comprises the plurality of calibrated assessment items, wherein each assessment item in the plurality of calibrated assessment items is associated with a known difficulty level of the assessment item and a question type of the assessment item.

5. The system of claim 1, wherein the first database comprises a record of every assessment item presented to the user.

6. The system of claim 5, wherein the system does not present an assessment item in the plurality of calibrated assessment items to the user that was previously presented to the user.

7. The system of claim 1, further configured to:

before the selection of one or more fixed forms for speaking, listening and writing, repeat the steps, at least once, of;

select a second next question type corresponding to a second language skill of the user,

select a second assessment item from the second database of calibrated assessment items, wherein the second assessment item matches the selected second next question type and matches the second language skill of the user,

modify the user interface to present the second assessment item to the user,

generate a second score for a second response from the user to the second assessment item,

update the overall language skill of the user and the conditional standard error of measurement for the overall language skill of the user based on the second score, and

compare the conditional standard error of measurement for the overall language skill of the user to a predetermined threshold,

until the conditional standard error of measurement for the overall language skill of the user is less than the predetermined threshold.

8. A system for adaptively assessing a language learning of a user configured to:

retrieve a random item pattern comprising a sequence of question types, wherein at least one question type is a multiple choice type of question and at least one question type is a fill in the blank type of question; and

repeat the steps, at least once, of;

sequentially select a first next question type in the random item pattern,

select a first assessment item from a database of a plurality of calibrated assessment items, wherein the first assessment item matches the first next question type in the random item pattern and matches the overall language skill of the user,

modify the user interface to present the first assessment item to the user,

generate a first score for a response from the user to the first assessment item, and

update an overall language skill of the user and a conditional standard error of measurement for the overall language skill of the user based on the first score,

until the conditional standard error of measurement for the overall language skill of the user is less than a predetermined threshold.

9. The system of claim 8, wherein the overall language skill is expressed on a logarithmic scale.

10. The system of claim 8, wherein the comprises the plurality of calibrated assessment items, wherein each assessment item in the plurality of calibrated assessment items is associated with a known difficulty level of the assessment item and a question type of the assessment item.

11. The system of claim 8, wherein the database comprises a record of every assessment item presented to the user.

12. The system of claim 11, wherein the system does not present an assessment item in the plurality of calibrated assessment items to the user that was previously presented to the user.

13. The system of claim 8, further configured to:

before the selection of one or more fixed forms for speaking, listening and writing, repeat the steps, at least once, of;

select a second next question type corresponding to a second language skill of the user,

select a second assessment item from the second database of calibrated assessment items, wherein the second assessment item matches the selected second next question type and matches the second language skill of the user,

modify the user interface to present the second assessment item to the user,

generate a second score for a second response from the user to the second assessment item,

update the overall language skill of the user and the conditional standard error of measurement for the overall language skill of the user based on the second score, and

compare the conditional standard error of measurement for the overall language skill of the user to a predetermined threshold,

until the conditional standard error of measurement for the overall language skill of the user is less than the predetermined threshold.

14. The system of claim 8, further configured to cause the processor of the hardware server to:

repeat the steps, at least once, of;

select a second next question type corresponding to a second language skill of the user,

select a second assessment item from the database of calibrated assessment items, wherein the second assessment item matches the selected second next question type and matches the second language skill of the user,

modify the user interface to present the second assessment item to the user,

generate a second score for a second response from the user to the second assessment item, and

update the overall language skill of the user and the conditional standard error of measurement for the overall language skill of the user based on the second score,

until the conditional standard error of measurement for the overall language skill of the user is less than a predetermined threshold.

15. A method, comprising the steps of:

displaying by a hardware server a user interface on a client device to a user;

searching by the hardware server in a first database for a history of the user;

estimating by the hardware server an overall language skill of the user based on the history of the user in the first database, a difficulty level of a product being used by the user and/or an initial response to an initial assessment item displayed on the user interface to the user;

generating by the hardware server a random item pattern comprising a sequence of question types, wherein at least one question type is a multiple choice type of question and at least one question type is a fill in the blank type of question; and

sequentially for each item in the random item pattern;

selecting by the hardware server a first next question type in the random item pattern,

selecting by the hardware server a first assessment item from a second database of a plurality of calibrated assessment items, wherein the first assessment item matches the first next question type in the random item pattern and matches the overall language skill of the user,

modifying the user interface to present by the hardware server the first assessment item on the client device to the user,

generating by the hardware server a first score for a first response from the user to the first assessment item, and

updating by the hardware server an overall language skill of the user and a conditional standard error of measurement for the overall language skill of the user based on the first score.

16. The method of claim 15, wherein the overall language skill is expressed on a logarithmic scale.

17. The method of claim 15, wherein the second database comprises the plurality of calibrated assessment items, wherein each assessment item in the plurality of calibrated assessment items is associated with a known difficulty level of the assessment item and a question type of the assessment item.

18. The system of claim 15, wherein the first database comprises a record of every assessment item presented to the user.

19. The system of claim 18, wherein the system does not present an assessment item in the plurality of calibrated assessment items that was previously presented to the user.

20. The method of claim 15, further comprising the steps of:

repeating the steps, at least once, of;

selecting by the hardware server a second next question type corresponding to a second language skill of the user,

selecting by the hardware server a second assessment item from the second database of calibrated assessment items, wherein the second assessment item matches the selected second next question type and matches the second language skill of the user,

modifying by the hardware server the user interface to present the second assessment item to the user,

generating by the hardware server a second score for a second response from the user to the second assessment item,

updating by the hardware server the overall language skill of the user and the conditional standard error of measurement for the overall language skill of the user based on the second score, and

comparing by the hardware server the conditional standard error of measurement for the overall language skill of the user to a predetermined threshold,

until the conditional standard error of measurement for the overall language skill of the user is less than the predetermined threshold.