System and method for intelligent quality control of a process

• US 6,556,951 B1
• Filed: 06/16/2000
• Issued: 04/29/2003
• Est. Priority Date: 11/26/1997
• Status: Expired due to Fees

• Current Assignee: United States Department Of Health And Human Services
• Original Assignee: United States Department Of Health And Human Services
• Inventors: Remaley, Alan T., Deleo, James M.
• Primary Examiner(s): Bui, Bryan
• Application Number:
  US09/555,325
• Time in Patent Office: 1,047 Days
• Field of Search: 702/183, 702/185, 702/182, 702/186, 702/187, 702/188, 700/48-50, 706/15, 706/21, 706/45
• US Class Current: 702/183
• CPC Class Codes: G06F 11/2263 using neural networks
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method and system for detecting errors in a process such as laboratory analysis of patient specimens and generation of test results is described. The steps of the method include collecting data elements having a range of values from the process. The number of data elements having values within predetermined intervals of the range are then counted. The counts of the data elements are applied as inputs to nodes of a neural network, each count being applied to a node representing the predetermined interval corresponding to the count. Output is then generated from the neural network based on the inputs, the output indicative of whether an error in the process (such as bias error or a precision error) has occurred. If the technology is used with a laboratory instrument, the output is generated in real time and available immediately for automatic or manual correction of the instrument.

Claims

1. A computer-implemented method for detecting errors in a process, the method comprising the following steps:

collecting data elements from the process, the data elements having a range of values;

counting the number of data elements having values within predetermined intervals of the range;

applying the counts of data elements as inputs to nodes of a neural network, each count being applied to a node representing the predetermined interval corresponding to the count; and

generating an output from the neural network based on the inputs, the output indicative of whether an error in the process has occurred.

2. The method of claim 1 wherein the collecting step comprises collecting through a moving window a predetermined number of data elements, and the method includes repeating the above steps after moving the window to replace a data element in the window with another data element outside the window.

3. The method of claim 1 including scaling the data elements to values within a predetermined range before counting the data elements.

4. The method of claim 3 wherein the scaling step comprises scaling the data elements to values within the range of zero to one.

5. The method of claim 1 wherein the process is laboratory analysis of patient specimens, and the collecting step comprises collecting test results from patients, the test results generated by a laboratory instrument from the patient specimens.

6. The method of claim 1 wherein the neural network is trained to detect a bias error or a precision error in the data elements applied as inputs to its nodes.

7. A computer-readable medium on which is stored instructions for executing the steps of claim 1.

8. The method of claim 1 further comprising:

training the neural network using ROC methodology.

9. The method of claim 8 wherein the neural network is enhanced by incorporating prevalence and misclassification cost information.

10. A computer-implemented method for detecting in real time errors in a process, the method comprising the following steps:

collecting through a moving window a set of data elements from the process;

scaling the data elements via a plurality of scaling techniques to generate scaled data elements;

basing inputs to nodes of a neural network on the scaled data elements;

generating an output from the neural network based on the inputs, the output indicative of whether an error in the process has occurred; and

repeating the above steps after moving the window to replace a data element in the window with another data element outside the window.

11. A method for detecting in real time errors caused by a laboratory instrument in the analysis of patient data, the method comprising the following steps:

collecting through a moving window a number of test results from a number of patients, the test results generated by a laboratory instrument from analysis of patient specimens;

applying the test results as inputs to nodes of a neural network;

generating an output from the neural network based on the inputs during operation of the laboratory instrument, the output indicative of whether an error in the analysis of patient specimens has occurred; and

repeating the above steps after moving the window to replace a test result in the window with another test result outside the window.

12. The method of claim 11 wherein the collecting step comprises collecting through the moving window at least one test result from each of a predetermined number of patients.

13. The method of claim 11 wherein the laboratory analysis generates test results of different types from each patient specimen, and the collecting step includes collecting test results of a same type in corresponding moving windows.

14. The method of claim 11 wherein the applying step comprises:

counting the number of test results having values within predetermined intervals of a range of values; and

applying the counts of test results as inputs to the nodes of the neural network, each count being applied to a node representing the predetermined interval corresponding to the count.

15. The method of claim 11 including scaling the test results to values within a predetermined range before counting the test results.

16. A computer system for executing the steps of claim 11.

17. The method of claim 11 further comprising:

scaling the test results using a plurality of scaling techniques to generate scaled test results;

wherein the applying comprises applying the scaled test results as inputs to nodes of the neural network.

18. The method of claim 11 further comprising:

predicting a measured analyte value for a patient from other analyte values for the patent;

via the predicted measured analyte value, determining whether a spontaneous error has occurred.

19. The method of claim 11 further comprising:

outputting a degree of membership the window holds for two dichotomous data populations, wherein at least one of the data populations indicates erroneous, unacceptable data.

20. The method of claim 11 further comprising:

outputting the degree of membership the window holds in a particular error class.

21. A system for detecting errors in the analysis of patient data, comprising:

a central processing unit;

a memory device coupled to the central processing unit and including;

a first data structure for collecting through a moving window a number of test results from a number of patients;

a second data structure for applying the test results as inputs to nodes of a neural network;

a neural network for generating an output based on the inputs applied to its nodes, the output indicative of whether an error in the analysis of patient specimens has occurred; and

a third data structure for moving the window to replace a test result in the window with another test result outside the window,

the neural network generating additional outputs as the test results in the window change;

wherein the second data structure includes a counter for counting the number of test results having values within predetermined intervals of the range of values and applying the counts of test results as inputs to the nodes of the neural network, each count being applied to a node representing the predetermined interval corresponding to the count.

22. The system of claim 21 including a laboratory instrument for analyzing patient specimens and generating test results therefrom.

23. The system of claim 21 wherein the neural network is trained to detect a bias or precision error in the test results applied as inputs to its nodes.

24. The system of claim 21 wherein the neural network is a multilayer network trained with a back-propagation algorithm.

25. The system of claim 21 wherein the nodes of the neural network correspond to intervals of test result values.

26. The system of claim 21 wherein the second data structure includes means for scaling the test results to values within a predetermined range before counting the test results.

27. A method of detecting, in real-time, errors in data produced by a laboratory instrument in analysis of patient specimens, the method comprising:

collecting through a moving window a plurality of test results from a plurality of patients, the test results generated by the laboratory instrument from the analysis of the patient specimens;

scaling the test results via a plurality of scaling techniques to generate scaled test results;

counting a number of scaled test results occurring in a plurality of ranges to generate counts of the number of scaled test results occurring in the ranges;

applying at least the counts to one or more neural networks of a first type to generate an output indicative of whether a bias or precision error is present; and

via one or more neural networks of a second type, predicting a measured analyte value for a patient from other analyte values for the patent to generate an output indicative of whether a spontaneous error is present.

28. A computer-readable medium comprising computer-executable instructions for performing the method of claim 27.

29. A computer-implemented method for detecting, in real time, a spontaneous error in a set of a plurality of measurements of analytes for a patient, the method comprising:

collecting the set of the plurality of measurements of analytes for the patent;

predicting a measured analyte value out of the set from other analyte values in the set; and

via the predicted measured analyte value, determining whether a spontaneous error has occurred.

30. The computer-implemented method of claim 29 wherein:

the measurements are taken from a laboratory instrument; and

the determining is accomplished during operation of the instrument.

31. A computer-implemented method for detecting, in real time, an error in measurements by an instrument, the method comprising:

applying a window of data representing measurements by the instrument to a neural network, wherein the neural network is trained to output a degree of membership the window holds for two dichotomous data populations; and

outputting a degree of membership the window holds for at least one of the data populations, wherein the data population indicates erroneous, unacceptable data.

32. The method of claim 31 wherein the instrument is a laboratory instrument.

33. The method of claim 32 wherein the measurements indicate analyte values for a patient.