Method for remediation based on knowledge and/or functionality

• US 6,260,033 B1
• Filed: 11/16/1998
• Issued: 07/10/2001
• Est. Priority Date: 09/13/1996
• Status: Expired due to Fees

• Current Assignee: Curtis M. Tatsuoka
• Original Assignee: Curtis M. Tatsuoka
• Inventors: Tatsuoka, Curtis M.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US09/192,815
• Time in Patent Office: 967 Days
• Field of Search: 706/45, 706/50, 705/7, 434/322, 707/501
• US Class Current: 706/45
• CPC Class Codes: G01R 31/318371 Methodologies therefor, e.g. algorithms, procedures
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls

Abstract

The invention is a method for remediation comprising steps (a) through (g). Step (a) consists of compiling a collection of one or more topics where a topic is a set of facts, a set of values, or a combination of a set of facts and a set of values that characterize knowledge and/or functionality. The set of facts that characterize knowledge is any set of facts. The set of facts that characterize functionality is a set of facts relating to the functionality of a test subject. Step (b) consists of compiling a collection of one or more treatments for each topic where a treatment comprises materials intended to teach a test subject. Step (c) consists of specifying a plurality of question blocks for each of the one or more treatments of step (b) where a question block consists of one or more questions. Response distributions are assigned to the questions in the question blocks. Step (d) consists of selecting one or more topics from those in the collection of step (a) for remediation. Step (e) consists of selecting one or more treatments from those specified in step (b) for the topics selected in step (d). Step (f) consists of obtaining responses to one or more question blocks associated with the treatments selected in step (e) from a test subject after exposure to the one or more treatments of step (e). Step (g) consists of obtaining a measure of the effectiveness of the treatments of step (e).

Claims

1. A method for remediation comprising the steps:

(a) compiling a collection of one or more topics, a topic being a set of facts, a set of values, or a combination of a set of facts and a set of values that characterize knowledge and/or functionality, the set of facts that characterize knowledge being any set of facts, the set of facts that characterize functionality being a set of facts relating to the functionality of a test subject;

(b) compiling a collection of one or more treatments for each topic, a treatment comprising materials intended to teach a test subject;

(c) specifying a plurality of question blocks for each of the one or more treatments of step (b), a question block consisting of one or more questions, a response distribution being assigned to at least one of the questions in at least one of the question blocks;

(d) selecting one or more topics from those in the collection of step (a) for remediation;

(e) selecting one or more treatments from those specified in step (b) for the topics selected in step (d);

(f) obtaining responses to one or more question blocks associated with the treatments selected in step (e) from a test subject after exposure to the one or more treatments of step (e);

(g) obtaining a measure of the effectiveness of the treatments of step (e) and utilizing one or more of the response distributions assigned in step (c).

2. The method of claim 1 further comprising the steps:

(h) repeating method from step (e) for one or more active topics, an active topic being a topic for which one or more treatment stopping rules have not been satisfied;

otherwise;

(i) repeating method from step (d) unless a method termination rule is satisfied.

3. The method of claim 2 wherein in step (h) a treatment stopping rule is one of the group consisting of (1) based on a function of one or more responses to question blocks, (2) that one of one or more predetermined sets of responses to question blocks have been obtained, (3) that a predetermined number of responses to question blocks have been obtained, (4) that a weighted treatment loss function value exceeds a predetermined value after hypothetical or actual administration of one or more treatments, (5) that a weighted treatment loss function value exceeds a predetermined value after hypothetical or actual administration of one or more questions, (6) that a weighted treatment loss function value exceeds a predetermined value after hypothetical or actual administration of one or more topics, (7) the combination of one or more treatment stopping rules, (8) based on one or more responses, (9) based on one or more response function values, (10) that a predetermined number of treatment types have been administered, and (11) that a predetermined number of treatments have been administered.

4. The method of claim 3 wherein a treatment loss function incorporates one or more of the group consisting of (1) a cost of administered treatment types, (2) a cost of administered treatments, (3) a cost of administered questions, (4) a cost of administered topics, and (5) response function values.

5. The method of claim 2 wherein step (a) includes the step:

(a1) specifying one or more topic-specific domains, each topic-specific domain comprising a plurality of states, the relationship between states in a topic-specific domain being characterized by a partial ordering, a test subject'"'"'s relationship to a topic-specific domain being representable by an SPS (state probability set);

the treatment stopping rule in step (h) being based on a member of the group consisting of (1) a weighted loss function, (2) a weighted improvement measure, (3) an SPS corresponding to a topic-specific domain, (4) a weighted uncertainty measure of an SPS corresponding to a topic-specific domain, (5) a weighted distance measure between a first SPS and a second SPS, and (6) a weighted treatment loss function where the treatment loss function is a function of a state in a topic-specific domain.

6. The method of claim 1 wherein in step (i) a method termination rule consists of one or more treatment stopping rules.

7. The method of claim 1 wherein the treatments specified in step (b) can be classified as to treatment type, step (e) comprising the steps:

(e1) selecting one or more treatment types from a treatment-type pool for a topic selected in step (d), the number of treatment types in the treatment-type pool being limited to one if a treatment-type selection process (TSP) stopping rule is satisfied and;

(e2) selecting one or more treatments from each treatment type selected in step (e1).

8. The method of claim 7 wherein in step (g) the value of a treatment parameter is measure of effectiveness, a probability distribution being associated with the treatment parameter, the selection process of step (e1) utilizing the probability distributions associated with one or more treatment parameters.

9. The method of claim 7 wherein step (e1) is discontinued for a topic when a TSP stopping rule is satisfied.

10. The method of claim 9 wherein a TSP stopping rule is based on a member of the group consisting of (1) one or more responses to questions, (2) a weighted TSP loss function, (3) the number of treatments selected exceeding a predetermined number, (4) the number of responses to questions exceeding a predetermined number, (5) the number of topics selected exceeding a predetermined number, (6) one of a plurality of predetermined subsets of responses to questions are obtained, and (7) response function values.

11. The method of claim 10 wherein a TSP loss function incorporates one or more of the group consisting of (1) a cost of administered treatment types, (2) a cost of administered treatments, (3) a cost of administered questions, (4) a cost of administered topics, (5) response function values, (6) a determination rule, and (7) a treatment parameter value, a treatment parameter being a measure of effectiveness of a treatment type.

12. The method of claim 7 wherein the selection of treatments is from the one or more treatments of the most effective treatment type, the most effective treatment type being determined by means of a determination rule.

13. The method of claim 12 wherein a determination rule is based on one of the group consisting of (1) a function of probability distributions associated with one or more treatment parameters, a treatment parameter being a measure of effectiveness of a treatment in remediating a test subject, (2) one or more probability distributions associated with one or more hyperparameters, and (3) one or more hyperparameter values.

14. The method of claim 7 further comprising the step:

(j) defining a remediation strategy tree.

15. The method of claim 14 wherein one or more branches of a remediation strategy tree are removed if a treatment stopping rule or a TSP (treatment-type selection process) stopping rule is satisfied at one or more points along paths of the remediation strategy tree.

16. The method of claim 7 wherein a response distribution for a question is a function of one or more treatments or a treatment type.

17. The method of claim 7 wherein a probability distribution is associated with the treatment parameter, a treatment parameter being a measure of effectiveness, a TSP stopping rule being based on the probability distributions of one or more treatment parameters or the posterior variance of a probability distribution associated with a treatment parameter.

18. The method of claim 7 wherein a TSP stopping rule is based on one or more responses to question blocks.

19. The method of claim 7 wherein in step (e1) selection of treatment type is based on one or more response distributions for questions, the response distributions being functions of one or more treatments or a treatment type.

20. The method of claim 7 wherein in step (e1) selection of treatment type is based on a weighted objective function, the weighting being done with respect to one or more response distributions for questions, a response distribution being a function of one or more treatments or a treatment type.

21. The method of claim 7 wherein a TSP stopping rule is based on a function of one or more response distributions which are functions of one or more treatments or a treatment type.

22. The method of claim 7 wherein step (a) includes the step:

(a1) specifying one or more topic-specific domains, each topic-specific domain comprising a plurality of states, the relationship between states in a topic-specific domain being characterized by a partial ordering, a test subject'"'"'s relationship to a topic-specific domain being representable by an SPS (state probability set);

the TSP stopping rule of step (e1) being based on a member of the group consisting of (1) a weighted loss function, (2) a weighted improvement measure, (3) an SPS corresponding to a topic-specific domain, (4) a weighted uncertainty measure of an SPS corresponding to a topic-specific domain, and (5) a weighted distance measure between a first SPS and a second SPS.

23. The method of claim 7 wherein a treatment type may include one or more treatments for a topic.

24. The method of claim 7 wherein step (g) includes the step:

(g1) estimating the value of a treatment parameter associated with a treatment type utilizing one or more responses to question blocks, a treatment parameter being a measure of effectiveness.

25. The method of claim 7 wherein in step (e1) the selection of a treatment type is based on one of a group consisting of (1) a weighted response function, (2) a weighted reward function, and (3) a weighted treatment loss function.

26. The method of claim 25 wherein a treatment loss function incorporates one or more of the group consisting of (1) a cost of administered treatments, (2) a cost of administered questions, (3) a cost of administered topics, (4) cost of treatment types, and (5) response function values.

27. The method of claim 7 wherein in step (e1) a prior treatment type is selected if the prior treatment type resulted in a set of responses matching one of one or more predetermined sets of responses to question blocks.

28. The method of claim 7 wherein in step (e1) a treatment type is selected by a process selected from the group consisting of (1) a random process and (2) a process selected randomly from a plurality of treatment selection processes.

29. The method of claim 7 wherein in step (e1) a treatment type is tentatively selected using a predetermined treatment-type selection rule, a random decision being made either to confirm the selection of the treatment type or to select another treatment type.

30. The method of claim 7 wherein step (e1) includes the steps:

(e1-1) creating a plurality of remediation strategies, a remediation strategy being representable by one or more remediation strategy trees;

(e1-2) selecting a best remediation strategy based on a comparative evaluation of the remediation strategies utilizing one or more objective functions and;

(e1-3) selecting a treatment type from the best remediation strategy.

31. The method of claim 7 wherein in step (e1) the treatment type selection process occurs in one or more stages, each of one or more treatment types being selected a predetermined number of times during a first stage.

32. The method of claim 7 wherein step (e1) can be accomplished using any one of a plurality of treatment-type selection processes, a particular treatment-type selection process being selected based on one of the group consisting of (1) a hybridized rule and (2) a relative ranking of treatment-type selection processes.

33. The method of claim 7 wherein a treatment type is deleted from the treatment type pool for a topic if a treatment-type deletion rule is satisfied.

34. The method of claim 33 wherein a treatment-type deletion rule is based on one of the group consisting of (1) an estimated value of a treatment parameter, the treatment parameter being a measure of effectiveness of a treatment and (2) a probability distribution associated with a treatment parameter, the probability distribution being a representation of the effectiveness of a treatment-type.

35. The method of claim 1 wherein in step (g) the value of a treatment parameter is a measure of effectiveness.

36. The method of claim 35 wherein a probability distribution is associated with the treatment parameter.

37. The method of claim 36 wherein the probability distribution associated with a treatment parameter is updated based on one or more responses.

38. The method of claim 36 wherein step (g) includes the step:

(g1) updating a treatment parameter value utilizing one or more responses to question blocks.

39. The method of claim 36 wherein the probability distribution associated with a treatment parameter is one of the group consisting of (1) a function of one or more other treatment parameters, (2) a function of the probability distributions associated with one or more other treatment parameters, and (3) a function of one or more hyperparameters.

40. The method of claim 36 wherein the probability distribution associated with a treatment parameter is a function of one or more responses to question blocks.

42. The method of claim 35 wherein a treatment parameter takes on a discrete set of values.

43. The method of claim 1 wherein in step (a) a plurality of topics are first identified and then one or more topics are selected in a random manner from the plurality of topics identified for inclusion in the collection.

44. The method of claim 1 wherein in step (a) a plurality of topics are first identified and then one or more of the easiest of the plurality of topics are selected for inclusion in the collection.

45. The method of claim 1 wherein the response distribution of step (c) is a function of one or more treatment parameters, a treatment parameter being a measure of effectiveness.

46. The method of claim 1 wherein step (g) includes the step:

(g1) updating a probability distribution utilizing one or more responses to question blocks and an updating rule.

47. The method of claim 1 wherein in step (f) a plurality of question-block sequences are generated, the question-blocks to be used in obtaining responses from a test subject being selected from one of the plurality of sequences based on a question-block sequence selection rule, the question-block sequence selection rule being based on a comparative evaluation of the question-block sequences based on one or more objective functions.

48. The method of claim 1 wherein step (g) includes the step:

(g1) applying a transformation function to a response distribution.

49. The method of claim 1 wherein step (g) includes the steps:

(g1) assigning a probability distribution to a hyperparameter and;

(g2) updating the probability distribution of the hyperparameter based on responses to one or more question blocks.

50. The method of claim 1 wherein the response distribution of a question is a function of the number of treatments administered to the test subject.

51. The method of claim 1 wherein the response distribution of a question is a function of one or more prior responses to question blocks.

52. The method of claim 1 wherein step (a) includes the step:

(a1) specifying one or more topic-specific domains, each topic-specific domain comprising a plurality of states, the relationship between states in a topic-specific domain being characterized by a partial ordering, a test subject'"'"'s relationship to a topic-specific domain being representable by an SPS (state probability set).

53. The method of claim 52 wherein the probability distribution associated with a treatment parameter is a function of an SPS of the test subject.

54. The method of claim 52 wherein in step (a) the determination to include at least one topic in the collection of topics was made by a process involving an SPS of the test subject.

55. The method of claim 52 wherein in step (d) a topic is selected based on an SPS and one or more response distributions.

56. The method of claim 52 wherein in step (e1) the selection process is based on a weighted improvement measure, an improvement measure being a measure of the difference between a first and second knowledge representation associated respectively with a first and second state in a topic-specific domain, the relationship between states in a topic-specific domain being characterized by a partial ordering.

57. The method of claim 56 wherein an improvement measure is a function of the difference in the number of facts associated with the first and second states and/or a function of the difference in corresponding quality measure values.

58. The method of claim 56 wherein an improvement measure incorporates a cost component, the cost component including costs of treatment-types and/or costs of treatments and/or costs of questions and/or costs of topics.

59. The method of claim 52 wherein partitions of states in a topic-specific domain are determined by response distributions of questions which are a function of a state in a topic-specific domain, a partition being a subset of states for which the response distributions of a question are the same or the union of such subsets, step (a1) including the steps:

(a1-1) determining the intersection of partitions for a collection of questions and;

(a1-2) replacing a first topic-specific domain configuration of states with a second topic-specific domain configuration of states, the second topic-specific domain configuration being the intersection of partitions of the first topic-specific domain configuration, the partial ordering of the second topic-specific domain configuration being derived from the partial ordering of the first topic specific domain configuration.

60. The method of claim 52 wherein in step (e1) a treatment type is selected based on one of the group consisting of (1) an SPS and (2) one or more question response distributions.

61. The method of claim 52 wherein an SPS is associated with a topic-specific domain, the SPS being updated in accordance with one or more responses to question blocks.

62. The method of claim 52 wherein in step (c) a response distribution is a function of a state in a topic-specific domain.

63. The method of claim 62 wherein there is a one-to-one correspondence between a plurality of test items and a plurality of questions, a response distribution for a test item being the same as a response distribution for a corresponding question.

64. The method of claim 52 wherein step (g) includes the step:

(g1) classifying the test subject to a state in one or more topic-specific domains.

65. The method of claim 64 wherein in step (a) at least one topic in the collection of topics is the union of one or more subsets of states higher than the state to which the test subject is classified.

66. The method of claim 64 wherein the probability distribution associated with a treatment parameter is a function of the test subject'"'"'s state in a topic-specific domain.

67. The method of claim 64 wherein the classification is based on one of the group consisting of (1) one or more responses to question blocks and (2) an SPS.

68. The method of claim 64 wherein the test subject is classified to one of the group consisting of (1) the state associated with the smallest value of a weighted loss function, (2) the state of a second topic-specific domain that is equivalent to the state to which the test subject has been classified in a first topic-specific domain, and (3) the state associated with the highest value in an SPS.

69. The method of claim 64 wherein step (d) is accomplished utilizing the results of step (g1).

70. The method of claim 64 wherein in step (g) the value of a treatment parameter is a measure of effectiveness, a probability distribution associated with a treatment parameter being based on the results of step (g1).

71. The method of claim 52 wherein in step (f) the selection of the one or more question blocks is based on one of the group consisting of (1) one or more item objective functions, (2) a weighted loss function, (3) a partition measure, (4) an uncertainty measure of an SPS, (5) a distance measure between two SPSs, (6) a discrepancy measure, and (7) a function based on a precision function.

72. The method of claim 52 wherein the treatments specified in step (b) can be classified as to treatment type, step (e) comprising the steps:

(e1) selecting one or more treatment types for each topic selected in step (d) based on one or more item objective functions and;

(e2) selecting one or more treatments from each treatment type selected in step (e1).

73. The method of claim 72 wherein step (e1) is based on one of the group consisting of (1) a weighted response function, (2) a weighted reward function, (3) the highest probability of success for an associated question, (4) the highest average probability of success for a collection of associated questions and (5) a weighted treatment loss function, the treatment loss function being a function of a state in a topic-specific domain.

74. The method of claim 72 wherein treatment type selection is based on one or more responses to question blocks.

75. The method of claim 72 wherein at least one of the item objective functions is one of the group consisting of (1) a weighted loss function, (2) a partition measure, (3) an uncertainty measure of an SPS, (4) a distance measure between two SPSs, (5) a discrepancy measure, and (6) a function based on a precision function.

76. The method of claim 72 wherein a determination rule is based on one of a group consisting of (1) one or more question response distributions that are functions of a state in a topic-specific domain, and (2) an SPS with respect to a topic-specific domain with states that represent values of a treatment parameter, a treatment parameter being a measure of effectiveness in remediating a test subject.

77. The method of claim 52 wherein the treatments specified in step (b) can be classified as to treatment type, step (e) comprising the steps:

(e1) selecting one or more treatment types from a treatment-type pool for a topic selected in step (d), the number of treatment types in the treatment-type pool being limited to one if a TSP stopping rule is satisfied, the TSP stopping rule being based on one of the group consisting of(1) an SPS, (2) whether an SPS value of a state in a topic-specific domain is greater than a predetermined threshold, (3) a weighted item objective function, (4) a loss function, (5) a distance measure between two SPSs, (6) an uncertainty measure, and (7) a TSP loss function, the TSP loss function being a function of;

a state in a topic-specific domain; and

(e2) selecting one or more treatments from each treatment type selected in step (e1).

78. The method of claim 52 wherein states in a topic-specific domain provide information about the effectiveness.

79. The method of claim 52 wherein at least one topic-specific domain comprises states which represent values of a treatment parameter, a treatment parameter providing a measure of effectiveness.

80. The method of claim 79 wherein a first state is higher than or equal to a second state if the treatment parameter value for the first state is greater than or equal to the treatment parameter value for the second state.

81. The method of claim 79 wherein a plurality of topic-specific domains are specified, at least one of the topic-specific domains having states corresponding to treatment parameter values.

82. The method of claim 79 wherein a topic-specific domain is generated by taking the product of a topic specific domain and a topic-specific domain consisting of states associated with treatment parameter values.

83. The method of claim 79 wherein a treatment type is deleted from the treatment type pool for a topic if a treatment-type deletion rule is satisfied, a treatment-type deletion rule being based on an SPS of a topic-specific domain.

84. The method of claim 52 wherein in step (d) a topic is selected based on one or more SPSs associated with topic-specific domains.

85. The method of claim 52 wherein a topic-specific domain has a finite number of states.

86. The method of claim 52 wherein an SPS is assigned to a first topic-specific domain based on the SPS obtained for a second topic-specific domain.

87. The method of claim 1 wherein gradations of treatments are treated as separate treatments.

88. The method of claim 1 wherein a response distribution for a question is a function of an extraneous variable.

41. The method of 36 wherein step (g) includes the steps:

(g1) assigning a probability distribution to a hyperparameter and;

(g2) updating the probability distribution of the hyperparameter based on responses to one or more question blocks, the response distribution associated with a question used to update the probability distribution of the hyperparameter being different from the response distribution associated with the same question used to update the probability distribution of a treatment parameter.