Instruction based on competency assessment and prediction

• US 9,542,853 B1
• Filed: 01/17/2014
• Issued: 01/10/2017
• Est. Priority Date: 12/10/2007
• Status: Active Grant

• Current Assignee: Accella Learning, LLC
• Original Assignee: Accella Learning, LLC
• Inventors: Bergeron, Bryan P.
• Primary Examiner(s): EGLOFF, PETER RICHARD
• Application Number:
  US14/157,878
• Time in Patent Office: 1,089 Days
• Field of Search: 434/322, 434/323, 434/365
• US Class Current: 1/1
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G09B 23/30 Anatomical models G09B23/281 - G09B23/288 take precedence
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G09B 7/04 characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation

Abstract

An intelligent tutoring system can be implemented using filters, predictive modeling, and a knowledge warehouse, which allows content to be dynamically selected for individualized presentation to a learner. Such an intelligent tutoring system can be implemented using a variety of techniques, including polynomial vector equations in modeling, extensible markup language storage formats for presentation data, and a knowledge warehouse with information stored in data cubes.

Claims

1. A non-transitory computer readable medium having stored thereon a set of computer executable instructions operable to configure a computer to perform a set of acts comprising:

a) receiving a set of user data from a database, said user data comprising a plurality of values corresponding to a plurality of competencies;

b) storing said set of received user data in a local memory of said computer;

c) applying the set of received user data stored in said local memory to a predictive model;

d) determining a set of content for presentation to a user, wherein;

i) the set of content is determined based on matching material for presentation to the user with a predicted skill level for the user at a time of presentation as indicated by the predictive model; and

ii) the set of content is tagged as implicating a first competency from the plurality of competencies;

e) incorporating, into the database from which the set of user data was received, a set of evaluation data received from said user in response to an evaluation comprising at least one question tagged as implicating the first competency from the plurality of competencies;

wherein;

A) determining the set of content for presentation to the user based on matching material for presentation to the user with the predicted skill level for the user comprises;

I) determining, using the predictive model, an estimated skill level for the user in the first competency at the time of presentation;

II) determining, based on the estimated skill level for the user in the first competency at the time of presentation, whether the user should be treated as requiring instruction in the first competency; and

III) determining that the set of content is suitable for providing instruction in the first competency based on the set of content being tagged as implicating the first competency;

B) the set of content is tagged as implicating both the first competency from the plurality of competencies and a second competency from the plurality of competencies;

C) determining the set of content for presentation to the user based on matching material for presentation to the user with the predicted skill level for the user comprises;

I) determining an estimated skill level for the user in the second competency at the time of presentation;

II) determining, based on the estimated skill level for the user in the second competency at the time of presentation, whether the user should be treated as requiring instruction in the second competency; and

III) determining that the set of content is suitable for providing instruction both in the first competency based on the set of content being tagged as implicating the first competency, and in the second competency based on the set of content being tagged as implicating the second competency;

D) the predictive model is specialized for predicting skill levels for the user in the first competency;

E) the estimated skill level for the user in the second competency is determined using a second predictive model specialized for predicting skill levels for the user in the second competency

F) the predictive model is specialized for predicting skill levels for the user in the first competency through the use of a parameter which mediates the influence of time on an expected change in the user'"'"'s skill level which is specific to the first competency;

G) the second predictive model is specialized for predicting skill levels for the user in the second competency through the use of a parameter which mediates the influence of time on an expected change in the user'"'"'s skill level which is specific to the second competency and which is different from the parameter used to specialize the first predictive model;

H) the set of content for presentation to the user comprises a simulated patient case; and

I) the set of acts the instructions are operable to configure the computer to perform comprises providing the simulated patient case to a mannequin-based simulator.

2. The non-transitory computer readable medium of claim 1, wherein the predictive model is a model of the form:

E(Δ

L)∝

e<sup>−

(t/s)</sup>

wherein;

a) E(Δ

L) is an expected change in the user'"'"'s skill level;

b) t is a duration between a time associated with a skill level from the set of user data received from the database and the time of presentation for which the predicted skill level is indicated; and

c) s is a retention factor specific to the user and to the first competency.

3. The non-transitory computer readable medium of claim 2, wherein the duration between the time associated with the skill level from the set of user data received from the database and the time of presentation for which the predicted skill level is indicated is more than one day.

4. A method comprising:

a) receiving a set of user data from a database, said user data comprising a plurality of values corresponding to a plurality of competencies;

b) storing said set of received user data in a local memory of a computer;

c) applying the set of received user data stored in said local memory to a predictive model;

d) determining a set of content for presentation to a user, wherein;

i) the set of content is determined based on matching material for presentation to the user with a predicted skill level for the user at a time of presentation as indicated by the predictive model;

ii) the set of content is tagged as implicating a first competency from the plurality of competencies; and

iii) determining the set of content for presentation to the user is performed by a processor executing instructions stored on a computer readable medium;

e) incorporating, into the database from which the set of user data was received, a set of evaluation data received from said user in response to an evaluation comprising at least one question tagged as implicating the first competency from the plurality of competencies;

wherein;

A) the set of content for presentation to the user comprises a simulated patient case;

B) the method comprises providing the simulated patient case to a simulator;

C) the evaluation comprising at least one question tagged as implicating the first competency comprises a request for a diagnosis based on interaction between the user and the simulator;

D) the method comprises presenting the set of content to the user by configuring the simulator to generate findings appropriate to the simulated patient case; and

E) the simulator is a mannequin-based simulator.

5. The method of claim 4, comprising determining a continuing education requirement for the user, wherein the continuing education requirement is specific to the user and requires differing levels of training for different competencies based on data stored in the database from which the set of user data was received.

6. The method of claim 4, wherein determining the set of content based on matching material for presentation to the user with the predicted skill level for the user at the time of presentation as indicated by the predictive model comprises determining an estimated skill level for the user in the first competency by calculating a weighted average of a plurality of scores for the user in the first competency taken from the set of user data received from database, wherein the scores from the plurality of scores are weighted based on relative significance for assessments used to generate those scores.

7. The method of claim 6, wherein:

a) the plurality of scores for the user in the first competency taken from the set of user data received from the database comprises;

i) a first score associated with a first time; and

ii) a second score associated with a second time;

b) the first time and the second time are separated by a duration of more than one day.

8. The method of claim 6, wherein:

a) the set of user data received from the database comprises a plurality of arrays, each array from the plurality of arrays comprising a vector for each competency from the plurality of competencies and associated with a time, wherein the plurality of arrays comprises;

i) a first array associated with the first time; and

ii) a second array associated with the second time;

b) the first score is comprised within a competency vector from the first array comprising;

i) the first score;

ii) an identification of the first competency; and

iii) a weight for the first score;

c) the second score is comprised within a competency vector from the second array comprising;

i) the second score;

ii) an identification of the first competency; and

iii) a weight for the second score.

9. A machine comprising:

a) a database;

b) a computer; and

c) a simulator;

wherein;

A) the computer is configured by data stored on a non-transitory computer readable medium to perform a set of acts comprising;

I) receiving a set of user data from the database, said user data comprising a plurality of values corresponding to a plurality of competencies;

II) storing said set of received user data in a local memory;

III) applying the set of received user data stored in said local memory to a predictive model;

IV) determining a set of content for presentation to a user, wherein;

a) the set of content is determined based on matching material for presentation to the user with a predicted skill level for the user at a time of presentation as indicated by the predictive model; and

b) the set of content is tagged as implicating a first competency from the plurality of competencies;

V) incorporating, into the database, a set of evaluation data received from said user in response to an evaluation comprising at least one question tagged as implicating the first competency from the plurality of competencies;

B) determining the set of content for presentation to the user based on matching material for presentation to the user with the predicted skill level for the user comprises;

I) determining, using the predictive model, an estimated skill level for the user in the first competency at the time of presentation;

II) determining, based on the estimated skill level for the user in the first competency at the time of presentation, whether the user should be treated as requiring instruction in the first competency;

III) determining that the set of content is suitable for providing instruction in the first competency based on the set of content being tagged as implicating the first competency;

C) the set of content is tagged as implicating both the first competency from the plurality of competencies and a second competency from the plurality of competencies; and

D) determining the set of content for presentation to the user based on matching material for presentation to the user with the predicted skill level for the user comprises;

I) determining an estimated skill level for the user in the second competency at the time of presentation;

II) determining, based on the estimated skill level for the user in the second competency at the time of presentation, whether the user should be treated as requiring instruction in the second competency;

III) determining that the set of content is suitable for providing instruction both in the first competency based on the set of content being tagged as implicating the first competency, and in the second competency based on the set of content being tagged as implicating the second competency;

E) the predictive model is specialized for predicting skill levels for the user in the first competency;

F) the estimated skill level for the user in the second competency is determined using a second predictive model specialized for predicting skill levels for the user in the second competency;

G) the evaluation comprising at least one question tagged as implicating the first competency from the plurality of competencies also comprises at least one question tagged as implicating the second competency from the plurality of competencies;

H) incorporating the set of evaluation data received from the user comprises modifying the predictive model and the second predictive model by using one or more curve fitting algorithms to account for responses provided by the user to the questions tagged as implicating the first and second competencies;

I) the set of content for presentation to the user comprises a simulated patient case;

J) the set of acts the computer is configured to perform comprises providing the simulated patient case to the simulator; and

K) the simulator is a mannequin-based simulator.

10. The machine of claim 9, wherein:

a) the machine comprises a means for generating an estimated user level; and

b) the set of acts the computer is configured to perform comprises treating an estimated user level generated by the means for generating the estimated user level as the predicted skill level for the user.

11. The machine of claim 10, wherein the computer is the means for generating the estimated user level.