PROBABILISTIC DECISION MAKING SYSTEM AND METHODS OF USE

• US 20110016067A1
• Filed: 03/11/2009
• Published: 01/20/2011
• Est. Priority Date: 03/12/2008
• Status: Active Grant

• Current Assignee: Aptima Inc., Wright State University
• Original Assignee: Aptima Inc., Wright State University
• Inventors: Levchuk, Georgiy, Sheblinski, Wayne, Freeman, Jared
• Granted Patent: US 8,655,822 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 50/20 Education

Abstract

Embodiments of this invention comprise modeling a subject'"'"'s state and the influence of training scenarios, or actions, on that state to create a training policy. Both state and effects of actions are modeled as probabilistic using Partially Observable Markov Decision Process (POMDP) techniques. The POMDP is well suited to decision-theoretic planning under uncertainty. Utilizing this model and the resulting training policy with real world subjects creates a surprisingly effective decision aid for instructors to improve learning relative to a traditional scenario selection strategy. POMDP provides a more valid representation of trainee state and training effects, thus it is capable of producing more valid recommendations concerning how to structure training to subjects.

Claims

1. A computer based system for determining training treatments for a subject, said system comprising:

a memory to store at least one action comprising at least one training treatment;

a processor capable of executing machine instructions; and

the machine instructions including means for executing a POMDP model to create a training policy to determine the at least one training treatment to train a subject on a topic.

2. The system of claim 1 wherein the subject is a team and the training treatments are training treatments for team training.

3. The system of claim 1 wherein the means for executing a POMDP model further comprises the POMDP model having at least one state, at least one transition function, at least one reward function, at least one observation and at least one observation function.

4. The system of claim 3 wherein:

the at least one state comprises a representation of an expertise state of the subject;

the at least one transition function comprises a representation of the probability of an expected changed expertise state of the subject after training the subject on the treatment;

the at least one reward function comprises a representation of at least one objective and at least one cost of training the subject on the treatment;

the at least one observation comprises a representation of a measure of the subject; and

the at least one observation function comprises a representation of the probability of an expected observation of the subject after training the subject on the treatment.

5. The system of claim 4 wherein the representation of the state of expertise of the subject comprises a set of numbers representing the expertise state of the subject on the topic.

6. The system of claim 4 wherein the observation comprises a set of numbers representing the measures of the subject.

7. The system of claim 4 wherein:

the transition function representation of the expected changed expertise state of the subject further comprises a probability of moving from the expertise state to the expected changed expertise state conditioned on the training treatment given to a subject;

the reward function representation of the objective further comprises at least one number, where each number represents a benefit of subject attaining the expertise state given the training treatment; and

the observation function representation of the expected observation of the subject further comprises the probability of an observation given the subject'"'"'s expertise state and training treatment given to the subject.

8. The system of claim 4 wherein the step of utilizing a POMDP model further comprises creating a training policy by linking each state to at least one training treatment at a node and interconnecting each node to another node by at least one observation.

9. The system of claim 8 wherein the step of utilizing a POMDP model further comprises applying the training policy by obtaining the state of the subject, selecting the node having that state and determining the linked training treatment at that node as the training treatment to train the subject on the topic.

10. The system of claim 9 further comprising:

after the step of applying the training policy to determine the training treatment, training the subject on the training treatment;

obtaining the observation for the subject; and

applying the training policy to select the interconnected node and the changed state of the subject based on the observation, and determine a next training treatment to train the subject.

11. The system of claim 10 further comprising:

a process threshold; and

iterating the steps of training the subject, obtaining the observation for the subject and applying the training policy to select the training treatment until the process threshold is met.

12. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform the method steps comprising:

generating a decision making policy from a POMDP model;

the POMDP model comprising at least one state parameter, at least one observation parameter and at least one action parameter; and

the action parameter comprising training treatments.

13. The program storage device of claim 12 wherein the at least one state parameter comprises the state of expertise of a subject and the at least one observation parameter comprises a measure of the expertise of the subject.

14. The program storage device of claim 13 wherein the step of generating a decision making policy further comprises:

defining the at least one state parameter, the at least one action parameter and the at least one observation parameter;

defining a plurality of functions comprising at least one transition function, at least one observation function and at least one utility function; and

generating the decision making policy based on said parameters and said functions.

15. The program storage device of claim 14 further comprising;

determining a changed state of the subject after applying an action parameter;

comparing the changed state of the subject to a process threshold;

selecting the at least one action parameter from the decision making policy;

applying the at least one action parameter to the subject;

determining a new changed state of the subject;

comparing the new changed state of the subject to the process threshold; and

repeating the steps of selecting the at least one action parameter, applying the at least one action parameter, determining a new changed state and comparing the new changed state until the process threshold is met.

16. A computer based method for structuring training treatments for a subject on a topic, said method comprising:

defining at least one action comprising at least one training treatment;

utilizing a POMDP model to create a training policy to determine the at least one training treatment to train a subject on a topic.

17. The method of claim 16 wherein the subject is a team and the training treatments are training treatments for team training.

18. The method of claim 16 wherein the step of utilizing a POMDP model further comprises the POMDP model having at least one state, at least one transition function, at least one reward function, at least one observation and at least one observation function.

19. The method of claim 18 wherein:

the at least one state comprises a representation of an expertise state of the subject;

the at least one transition function comprises a representation of the probability of an expected changed expertise state of the subject after training the subject on the treatment;

the at least one reward function comprises a representation of at least one objective and at least one cost of training the subject on the treatment;

the at least one observation comprises a representation of a measure of the subject; and

the at least one observation function comprises a representation of the probability of an expected observation of the subject after training the subject on the treatment.

20. The method of claim 19 wherein the representation of the state of expertise of the subject comprises a set of numbers representing the expertise state of the subject on the topic.

21. The method of claim 19 wherein the observation comprises a set of numbers representing the measures of the subject.

22. The method of claim 19 wherein:

the transition function representation of the expected changed expertise state of the subject further comprises a probability of moving from the expertise state to the expected changed expertise state conditioned on the training treatment given to a subject;

the reward function representation of the objective further comprises at least one number, where each number represents a benefit of subject attaining the expertise state given the training treatment; and

the observation function representation of the expected observation of the subject further comprises the probability of an observation given the subject'"'"'s expertise state and training treatment given to the subject;

23. The method of claim 19 wherein the step of utilizing a POMDP model further comprises creating a training policy by linking each state to at least one training treatment at a node and interconnecting each node to another node by at least one observation.

24. The method of claim 23 wherein the step of utilizing a POMDP model further comprises applying the training policy by obtaining the state of the subject, selecting the node having that state and determining the linked training treatment at that node as the training treatment to train the subject on the topic.

25. The method of claim 24 further comprising:

after the step of applying the training policy to determine the training treatment, training the subject on the training treatment;

obtaining the observation for the subject; and

applying the training policy to select the interconnected node and the changed state of the subject based on the observation, and determine a next training treatment to train the subject.

26. The method of claim 25 further comprising:

a process threshold; and

iterating the steps of training the subject, obtaining the observation for the subject and applying the training policy to select the training treatment until the process threshold is met.