COMPUTER SYSTEM AND METHOD FOR TRAINING CERTIFYING OR MONITORING HUMAN CLINICAL RATERS

• US 20080312958A1
• Filed: 12/28/2007
• Published: 12/18/2008
• Est. Priority Date: 10/26/2001
• Status: Active Grant

• Current Assignee: Signant Health Global LLC
• Original Assignee: UBC Specialty Clinical Services, LLC (Signant Health)
• Inventors: Perlis, Roy Howard, Sachs, Gary Steven, DeBonis, Daniel Vincent II
• Granted Patent: US 8,087,938 B2
• Time in Patent Office: Days
• US Class Current: 705/2
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 70/60 relating to pathologies

Abstract

A method and system are disclosed for computerized training, monitoring, certification or re-certification of human raters in clinical trials. The invention provides an interactive computerized interview that can be compared scores obtained by human raters to monitor and train clinical raters on a continuing basis in order to reduce rater drift and variance during clinical trials. Remediation can be automatically flagged or provided to improve a deviating rater'"'"'s performance. Computerized assessment of symptom severity is provided without the need for human clinical raters. The system and method enable screening of prospective patients for inclusion or exclusion from a clinical trial by automatically obtaining computerized ratings of symptom severity.

Claims

1-40. -40. (canceled)

41. A method for recertifying or monitoring performance of a clinical rater participating in a clinical trial comprising:

receiving one or more rater inputs reflecting the rater'"'"'s clinical evaluation of severity of a previously diagnosed condition in a subject;

performing a computerized assessment of the subject to quantify severity of the previously diagnosed condition in the subject through a computerized interview that comprises;

(a) presenting a first question to the subject and receiving a first input from the subject in response thereto;

(b) based on the first input, selecting a second question from a plurality of questions;

(c) presenting the selected second question-to the subject and receiving a second input from the subject in response thereto; and

(d) based on one or more inputs received from the subject, determining in accordance with a clinical rating scale a computerized severity score for the previously diagnosed condition in the subject; and

(d) determining whether the one or more rater inputs are in accordance with the computerized severity score.

42. The method according to claim 41, wherein the previously diagnosed condition is a psychiatric illness.

43. The method according to claim 41, wherein the clinical rating scale is in accordance with one of the following scales:

the Young Mania Rating Scale (YMRS);

the Montgomery-Asberg Depression Rating Scale (MADRS);

the Hamilton Depression Scale (HAMD);

the Schedule for Affective Disorders and Schizophrenia, Current symptom version mania Rating Scale (SADS-C);

an Additional Psychiatry Scale;

a Geriatric/Cognitive Scale;

an Other Cognitive Assessment;

a Non-CNS Scale;

or a member of the APA group of scales.

44. The method according to claim 41, wherein at least one unselected question is not presented to the subject.

45. The method according to claim 41, further comprising:

based on the second input, selecting a third question from the plurality of questions;

and presenting the selected third question to the subject and receiving a third input from the subject in response thereto.

46. The method according to claim 45, wherein selection of the third question is based on a plurality of inputs by the subject.

47. The method according to claim 41, wherein the questions are presented by visual, audio, or multimedia display.

48. The method according to claim 41, wherein the inputs are received directly from the subject by means of a keyboard, a pointing device, a touch-screen device, or voice response.

49. The method according to claim 41, wherein the computerized assessment is performed within 24 hours after a clinical interview at which the clinical evaluation is made.

50. The method according to claim 41, wherein the clinical interview and computerized assessment are performed during the same clinic visit.

51. The method according to claim 41, wherein the step of receiving comprises receiving a plurality of rater inputs reflecting a given rater'"'"'s clinical evaluation of severity of the previously diagnosed condition in a plurality of subjects at a plurality of clinical interviews;

and wherein the step of determining comprises determining whether the plurality of rater inputs are in accordance with computerized severity scores for the previously diagnosed condition in the subjects.

52. The method according to claim 41, wherein the step of receiving comprises receiving a plurality of rater inputs reflecting a given rater'"'"'s clinical evaluation of severity of a previously diagnosed condition in a single subject at a plurality of clinical interviews of that subject;

and wherein the step of determining comprises determining whether the plurality of rater inputs are in accordance with a plurality of computerized severity scores for the previously diagnosed condition in that subject determined in conjunction with the plurality of clinical interviews.

53. A method for improving performance of a clinical rater participating or being trained to participate in a clinical trial, comprising:

monitoring performance of the clinical rater in accordance with claim 41; and

if rater input substantially differs from the computerized severity score or scores, providing an indication that the clinical rater is in need of remedial training or providing remedial training to the rater.

54. The method of claim 53, wherein the remedial training includes one or more of the following:

providing explanatory material concerning how to evaluate severity of the previously diagnosed condition, presenting a remedial script, and presenting a remedial semi-scripted interview to the rater.

55. A computer programmed to perform the method according to claim 53.

56. A computer readable medium having stored thereon instructions for causing a computer to perform the method according to claim 53.

57. A system for performing the method according to claim 41, comprising:

an output device configured to present selected questions to the subject;

an input device configured to accept inputs from the subject; and

one or more computers configured to present selected questions to the subject by means of the output device, receive inputs from the subject by means of the input device, and determine the computerized severity score based on the inputs received from the subject in accordance with the rating scale.

58. The method according to claim 58, wherein individual raters are provided with non-synchronous sessions for training, certification or recertification.

59. The method according to claim 58, further comprising:

based on the comparison, displaying an indication of the reference rating and associated rationale.

60. A computerized method for assisting or training a rater to evaluate severity of symptoms in a subject participating in a clinical trial and having a previously diagnosed medical condition, comprising:

presenting a first output to the rater conveying a first suggestion concerning a question to be posed to the subject or a clinical observation to be made concerning the subject;

receiving a first input from the rater in response to the first output;

based on the first input, selecting a second output from a plurality of alternative second outputs concerning a question to be posed to the subject or a clinical observation to be made concerning the subject and presenting the second output to the rater.

61. The method according to claim 60, further comprising:

based on the inputs received from the rater, determining in accordance with a clinical rating scale a computerized severity score for the medical condition.

62. The method according to claim 60, wherein the rater is a health care provider evaluating the severity of symptoms in a patient or evaluating a response of a patient to treatment or placebo.

63. A computer programmed to perform the method according to claim 60.

64. A computer readable medium having stored thereon instructions for causing a computer to perform the method according to claim 60.

65. A system for performing the method according to claim 60, comprising:

an output device configured to present selected outputs to the rater;

an input device configured to accept inputs from the rater; and

one or more computers configured to present selected outputs to the health care provider by means of the output device and receive inputs from the rater by means of the input device.

66. A computerized method for implementing a clinical rating scale to evaluate a previously diagnosed condition in a subject presented with a series of questions, the method comprising:

establishing a bank of questions directed to measurement of an item in the clinical rating scale;

presenting to the subject a first question selected from the bank of questions and receiving a first input from the subject in response thereto;

presenting to the subject a second question selected from the bank of questions based on the first input and receiving a second input from the subject in response thereto; and

based on received inputs, generating a score for the item in the clinical rating scale, wherein a given response to at least one question results in different scores for the item dependent on the place of the at least one question within the series of questions.

67. The method of claim 66, wherein each input to a question results either in generating a score for the item in the clinical rating scale or in selection of another question from the bank of questions.

68. The method of claim 66, wherein:

the step of establishing comprises establishing a plurality of banks of questions, each bank directed to measurement of a distinct item in the clinical rating scale; and

the step of generating comprises generating scores for a plurality of items in the clinical rating scale.

69. A branched script for implementing the method of claim 66, the script having a root and a plurality of leaves such that each leaf represents a score and is connected to the root via a plurality of intermediate nodes, each intermediate node representing at least one question from the plurality of questions.

70. The branched script according to claim 69, wherein a plurality of nodes representing the same question are on a plurality of distinct branches.

71. The method according to claim 66, further comprising the step of determining whether at least one input to a question is an acceptable input and presenting the question to the subject again if that input is not acceptable.

72. The method according to claim 66, wherein the series of questions includes a multiple choice question such that the subject selects from a list of inputs for responding to the question.

73. The method according to claim 72, wherein two or more inputs in the list of inputs result in a common question being presented to the subject.

74. The method according to claim 66, further comprising:

automatically determining by previously established criteria whether or not the subject qualifies for inclusion in a clinical trial.

75. A computer readable medium having stored thereon instructions for causing a computer to perform the method according to claim 66.

76. A computer programmed to perform the method according to claim 66.