Apparatus, method, and computer program product for determining confidence measures and combined confidence measures for assessing the quality of microarrays

• US 20040059520A1
• Filed: 09/25/2002
• Published: 03/25/2004
• Est. Priority Date: 09/25/2002
• Status: Active Grant

• Current Assignee: Biodiscovery Inc. (BioNano Genomics, Inc.)
• Original Assignee: Biodiscovery Inc. (BioNano Genomics, Inc.)
• Inventors: Shams, Soheil, Petrov, Anton
• Granted Patent: US 7,512,496 B2
• Time in Patent Office: Days
• US Class Current: 702/19
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G16B 40/00 ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
• CPC Class Codes: G16B 40/20 Supervised data analysis

Abstract

An apparatus, method, and computer program product are provided for determining a confidence measure for the output of a process for assessing proteomic and genomic information samples. Typically, data is received as the results from a microarry experiment. The confidence measure is intended to determine whether the results of the experiment have been degraded due to experimental error. Several criteria are used by which quality is determined. The criteria are preferably combined into a combined quality measure in order to account for several possible sources of error.

Claims

1. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the system comprising:

a memory for storing digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

a processor coupled with the memory for accessing the digital data therefrom, and for determining a confidence measure for each of at least a portion of the signals for which digital data is stored in the memory;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has a perimeter, wherein each data point is contained in an image snip having borders, and wherein the confidence measure is an open perimeter confidence measure for the data point, determined based on the percentage of the perimeter of a data point occupied by the border of its respective snip.

2. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 1, wherein the processor is operative for determining at least two confidence measures and for combining confidence measures.

3. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 2, wherein the confidence measures are combined into combined confidence measure including at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

4. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 2, wherein the confidence measures are combined heuristically into a plurality of not-necessarily exclusive subsets, each including subset members, with each subset producing a subset confidence measure by heuristically weighting the subset members.

5. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 4, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

6. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 4, wherein each subset confidence measure for a particular data point is bounded at a predetermined threshold level, and wherein when the subset confidence measure for a particular subset for the particular data point reaches the predetermined threshold level, a flag is produced for that data point, and wherein all flags produced for a particular data point are combined into a logical decision schema in order to determine whether to flag the data point.

7. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 6, wherein the subset confidence measures include at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

8. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 2, wherein the confidence measures are combined using a machine-learning algorithm selected from a group consisting of supervised and unsupervised machine-learning algorithms to produce a combined confidence measure.

9. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 8, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

10. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 8, wherein the machine learning algorithm is trained using a training set created by manually flagging data points considered to be defective.

11. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 10, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

12. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 8, wherein the machine learning algorithm is trained using a training set of flagged data points created by means of a replicate-based outlier detection schema.

13. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 12, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

14. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the method comprising:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has a perimeter, wherein each data point is contained in an image snip having borders, and wherein the confidence measure is an open perimeter confidence measure for the data point, determined based on the percentage of the perimeter of a data point occupied by the border of its respective snip.

15. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 14, wherein in the determining step, at least two confidence measures are determined and wherein the method further comprises a step of combining confidence measures.

16. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 15, wherein in the combining step, the confidence measures are combined into combined confidence measure including at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

17. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 15, wherein the confidence measures are combined heuristically into a plurality of not-necessarily exclusive subsets, each including subset members, with each subset producing a subset confidence measure by heuristically weighting the subset members.

18. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 17, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

19. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 17, wherein each subset confidence measure for a particular data point is bounded at a predetermined threshold level, and wherein when the subset confidence measure for a particular subset for the particular data point reaches the predetermined threshold level, a flag is produced for that data point, and wherein all flags produced for a particular data point are combined into a logical decision schema in order to determine whether to flag the data point.

21. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 15, wherein the confidence measures are combined using a machine-learning algorithm selected from a group consisting of supervised and unsupervised machine-learning algorithms to produce a combined confidence measure.

22. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 21, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

23. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 21, wherein the machine learning algorithm is trained using a training set created by manually flagging data points considered to be defective.

24. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 23, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

25. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 21, wherein the machine learning algorithm is trained using a training set of flagged data points created by means of a replicate-based outlier detection schema.

26. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 25, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

20. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 20, wherein the subset confidence measures include at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

27. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the computer program product usable with a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving the set of signals, the computer program product comprising means, stored on a computer readable medium, for:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has a perimeter, wherein each data point is contained in an image snip having borders, and wherein the confidence measure is an open perimeter confidence measure for the data point, determined based on the percentage of the perimeter of a data point occupied by the border of its respective snip.

28. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 27, wherein in the determining step, at least two confidence measures are determined and wherein the method further comprises a step of combining confidence measures.

29. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 28, wherein in the combining step, the confidence measures are combined into combined confidence measure including at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

30. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 28, wherein the confidence measures are combined heuristically into a plurality of not-necessarily exclusive subsets, each including subset members, with each subset producing a subset confidence measure by heuristically weighting the subset members.

31. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 30, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a normalized position offset confidence measure, a normalized background contamination confidence measure, a normalized signal contamination confidence measure, a normalized ignored pixel percentage confidence measure, a normalized open perimeter confidence measure, and a normalized shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

32. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 30, wherein each subset confidence measure for a particular data point is bounded at a predetermined threshold level, and wherein when the subset confidence measure for a particular subset for the particular data point reaches the predetermined threshold level, a flag is produced for that data point, and wherein all flags produced for a particular data point are combined into a logical decision schema in order to determine whether to flag the data point.

33. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 32, wherein the subset confidence measures include at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

34. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 28, wherein the confidence measures are combined using a machine-learning algorithm selected from a group consisting of supervised and unsupervised machine-learning algorithms to produce a combined confidence measure.

35. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 34, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

36. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 34, wherein the machine learning algorithm is trained using a training set created by manually flagging data points considered to be defective.

37. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 36, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

38. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 34, wherein the machine learning algorithm is trained using a training set of flagged data points created by means of a replicate-based outlier detection schema.

39. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples as set forth in claim 38, wherein the combined confidence measure includes at least one confidence measure selected from a group consisting of a position offset confidence measure, a background contamination confidence measure, a signal contamination confidence measure, a ignored pixel percentage confidence measure, a open perimeter confidence measure, and a shape regularity confidence measure, and wherein the normalization of each of the confidence measure includes a weighting factor to determine the relative contribution of each confidence measure to the combined confidence measure.

40. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the system comprising:

a memory for storing digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

a processor coupled with the memory for accessing the digital data therefrom, and for determining a confidence measure for each of at least a portion of the signals for which digital data is stored in the memory;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples;

wherein each data point has an expected shape and an actual shape, and where a shape regularity confidence measure for the data point is determined based on a comparison of the actual shape with the expected shape.

41. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the method comprising:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has an expected shape and an actual shape, and where a shape regularity confidence measure for the data point is determined based on a comparison of the actual shape with the expected shape.

42. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the computer program product usable with a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving the set of signals, the computer program product comprising means, stored on a computer readable medium, for:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has an expected shape and an actual shape, and where a shape regularity confidence measure for the data point is determined based on a comparison of the actual shape with the expected shape.

43. An apparatus for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the system comprising:

a memory for storing digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

a processor coupled with the memory for accessing the digital data therefrom, and for determining a confidence measure for each of at least a portion of the signals for which digital data is stored in the memory;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples;

wherein each data point has a perimeter, and where an open perimeter confidence measure for the data point is determined based on a percentage of the perimeter that is open.

44. A method for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the method comprising:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has a perimeter, and where an open perimeter confidence measure for the data point is determined based on a percentage of the perimeter that is open.

45. A computer program product for determining a confidence measure for the output of a process for assessing proteomic and genomic data samples, with the output represented as a set of signals, the computer program product usable with a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving the set of signals, the computer program product comprising means, stored on a computer readable medium, for:

receiving digital data, in the form of data points representing data samples, generated from at least a portion of the signals in the set of signals;

determining a confidence measure for each of at least a portion of the signals;

the confidence measure being indicative of a confidence level in the quality of the output of the process for assessing proteomic and genomic data samples; and

wherein each data point has a perimeter, and where an open perimeter confidence measure for the data point is determined based on a percentage of the perimeter that is open.