Method and apparatus for performing and quantifying color changes induced by specific concentrations of biological analytes in an automatically calibrated environment

• US 9,311,520 B2
• Filed: 04/05/2013
• Issued: 04/12/2016
• Est. Priority Date: 08/08/2012
• Status: Active Grant

• Current Assignee: Healthy.io, Ltd.
• Original Assignee: Scanadu Incorporated
• Inventors: Burg, Bernard, Zizi, Martin, Rowe, Aaron Alexander, Smart, Anthony, De Brouwer, Walter
• Primary Examiner(s): Nur, Abdullahi
• Application Number:
  US14/419,939
• Publication Number:
  US 20150211987A1
• Time in Patent Office: 1,103 Days
• Field of Search: 356402-425
• US Class Current: 1/1
• CPC Class Codes: A61B 5/1172 using fingerprinting
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/46 Measurement of colour; Colour measuring devices, e.g. colorimeters measuring colour temperature G01J5/60
• CPC Class Codes: G01N 2021/1765 Method using an image detector and processing of image signal
• CPC Class Codes: G01N 2021/8488 the band presenting reference patches
• CPC Class Codes: G01N 2035/00108 Test strips, e.g. paper
• CPC Class Codes: G01N 21/27 using photo-electric detection G01N21/31 takes precedence; circuits for computing concentration logarithmic circuits G06G7/24; photometric circuits in general G01J
• CPC Class Codes: G01N 21/274 Calibration, base line adjustment, drift correction
• CPC Class Codes: G01N 21/78 producing a change of colour
• CPC Class Codes: G01N 21/8483 Investigating reagent band test-element handling not specific to a test method G01N33/4875; analytical elements specific to chemical analysis of biological material G01N33/52; autometer with reagent band G01N35/04
• CPC Class Codes: G01N 2201/12 Circuits of general importance; Signal processing
• CPC Class Codes: G01N 35/00029 provided with flat sample substrates, e.g. slides G01N35/028 takes precedence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/1324 by using geometrical optics, e.g. using prisms G06V40/1312 takes precedence

Abstract

Methods and electronic devices for performing color-based reaction testing of biological materials. The method includes capturing and interpreting digital images of an unexposed and later exposed paddle at various delay times within an automatically calibrated environment. The test paddle includes a unique identification mechanism (UID), a Reference Color Bar (RCB) providing samples of standardized colors for image color calibration, compensation and corrections, and several test-specific sequences of Chemical Test Pads (CTP). The method further includes locating the paddle in the image, extracting the UID and validating the paddle, extracting the RCB and locating the plurality of CTP in each image. The method further reduces image noise in the CTP and calibrates the image automatically according to lighting measurements performed on the RCB. To determine test results, the method further determines several distances between the CTP and its possible trajectory in the color space described by the Manufacturer Interpretation Color Chart.

Claims

1. A computer-implemented method for quantifying color change of at least one test medium on a diagnostic instrument, the method comprising:

capturing a digital image of at least a portion of the diagnostic instrument, which has been exposed to a biological sample, the diagnostic instrument comprising at least one color reference comprising a plurality of reference samples of different colors and a plurality of test medium respectively having a plurality of reagents, each reagent changing color in the presence of particular analytes in the biological sample;

identifying at least one of the plurality of reference samples for at least one of the plurality of test medium in the diagnostic instrument;

determining a dominant camera-captured color of the at least one reference sample and a dominant camera-captured color of the at least one test medium;

color correcting the dominant camera-captured color of the at least one test medium based on a correction factor derived at least in part from the dominant camera-captured color of the reference sample to determine a corrected test medium color; and

comparing the corrected test medium color to a set of possible test medium colors corresponding to predetermined analyte concentrations to determine a test result including an analyte concentration of the biological sample being tested.

2. The computer-implemented method of claim 1, wherein the diagnostic instrument comprises a plurality of test media arranged in a plurality of test specific sequences.

3. The method of claim 1, wherein the diagnostic instrument further comprises an identification label including or associated with identification information about the diagnostic instrument, and wherein the captured digital image of the diagnostic instrument includes at least a portion of the identification label.

4. The method of claim 3, wherein the identification label is at least one of the following:

a bar code;

a QR code, a near-field communication tag, or any combination thereof.

5. The method of claim 3, wherein the identification information includes at least one of the following:

the manufacture date of the diagnostic instrument;

the expiration date of the diagnostic instrument;

the analytes tested for by the instrument;

identifying information about the test subject;

patient condition information;

or any combination thereof.

6. The method of claim 3, further comprising validating the diagnostic instrument by analyzing the portion of the digital image including the identification label to determine the identification information, and determining whether the diagnostic instrument is in condition for use based on the identification information.

7. The method of claim 6, wherein validating the diagnostic instrument further comprises selecting at least one possible test medium color based on the identifying information.

8. The method of claim 3, wherein the identifying step further comprises:

scanning the digital image to determine the position of the identification label on the digital image;

determining geometric corrections for the digital image based at least in part on the position of the identification label, the geometric corrections comprising one or more angles of rotation of the diagnostic instrument about at least one of the following;

the x-axis, the y-axis, the z-axis, or any combination thereof; and

,

calculating the position of the reference sample and the at least one test medium on the digital image based at least partially on the position of the identification label and the geometric corrections.

9. The method of claim 1, wherein the identifying step further comprises:

identifying a region of the digital image containing the reference sample; and

removing background artifacts from the identified region by applying a variance operator to remove portions of the identified region having low variance and maintaining portions of the identified region having high variance.

10. The method of claim 9, wherein the identifying step further comprises:

identifying a region of the digital image including the reference sample and a region of the digital image including the at least one test medium;

identifying edge artifacts on a boundary of the identified regions; and

removing identified edge artifacts from the digital image.

11. The method of claim 1, wherein the determining step further comprises:

defining spatial guard bands around or just within a boundary of the reference sample and the at least one test medium to form enclosed areas; and

reducing color noise within the enclosed areas of the reference sample and the at least one test medium by averaging the color within the enclosed areas.

12. The method of claim 11, further comprising applying a median filter to at least one enclosed area to determine the dominant color therein.

13. The method of claim 1, wherein the color correcting step further comprises:

identifying a reference sample that is white in standard lighting conditions on the digital image;

determining the correction factor based at least partially on the average color of the white reference sample in the digital image; and

color correcting the color of the at least one test medium by applying the correction factor using a white balancing algorithm.

14. The method of claim 1, wherein the color correcting step further comprises:

comparing a luminescence of the reference sample to a predetermined expected luminescence to determine a luminescence correction factor;

comparing the dominant camera-captured color of the reference sample with an expected sample color to determine a color correction factor; and

correcting the camera-captured color of the at least one test medium based at least partially on the luminescence correction factor and the color correction factor.

15. The method of claim 1, further comprising rejecting the digital image if the corrected test medium color exceeds a predetermined value.

16. The method of claim 1, wherein the camera-captured reference sample and the camera-captured test medium color each comprise a plurality of color components defining at least one colorspace, and wherein the at least one colorspace is at least one of the following:

red-green-blue (RGB);

cyan-magenta-yellow-key (CMYK);

International Commission on Illumination (CIE) colorspace;

pantone colorspace;

Munsell colorspace;

or any combination thereof.

17. The method of claim 1, wherein the comparing step further comprises:

determining a nearest possible test medium color to the corrected test medium color; and

determining the analyte concentration corresponding to the nearest possible test medium color.

18. The method of claim 1, wherein the comparing step further comprises:

mapping the set in at least one colorspace to form at least one color trajectory;

calculating a shortest substantially perpendicular distance between the corrected test medium color and the at least one color trajectory to determine an intersection point; and

calculating the analyte concentration of the biological sample based at least partially on the position of the intersection point.

19. The method of claim 18, wherein the at least one color trajectory is derived by polynomial interpolation of the set of colors.

20. The method of claim 19, further comprising calculating an error probability based on shortest perpendicular distance between the correct test medium color and the at least one color trajectory.

21. The method of claim 1, further comprising displaying at least a portion of the test result on a visual display of a portable electronic device.

22. The method of claim 1, further comprising determining at least one of the following:

a patient condition;

a probable cause of the patient condition;

a suggested follow-up action;

or any combination thereof, based at least partially on the test result.

23. The method of claim 1, further comprising transmitting at least a portion of the test result to an external source comprising at least one of the following:

an external computer network, an external data collection device, the Internet, or any combination thereof.

24. The method of claim 1, wherein the capturing step further comprises exposing the diagnostic image to a flash for improving lighting conditions of the digital image.

25. The method of claim 1, wherein at least one of the following:

the capturing step;

the identifying step;

the determining step;

the color correcting step;

the comparing step;

or any combination thereof, is performed using a portable electronic device.

26. The method of claim 25, wherein the portable electronic device is at least one of the following:

a Smartphone, a tablet PC, a computer, or any combination thereof.

27. A system for reading diagnostic test results, the system comprising:

a diagnostic instrument comprising a paddle, a color reference mounted to the paddle having a plurality of color reference samples of different colors, and a plurality of test media mounted to the paddle, the plurality of test media respectively containing reagents which change color in the presence of particular analytes in a biological sample; and

a portable electronic device comprising a camera sensor for capturing a digital image of at least a portion of the diagnostic instrument and a processor configured to;

identify at least one of the plurality of color reference samples and at least one of the plurality of test media in the digital image of the diagnostic instrument;

determine a dominant camera-captured color of a reference sample and a dominant camera-captured color of at least one test medium;

color correct the dominant camera-captured color of the at least one test medium based on a correction factor derived at least in part from the dominant camera-captured color of the reference sample to determine a corrected test medium color; and

compare the corrected test medium color to a set of possible test medium colors corresponding to predetermined analyte concentrations to determine a test result including an analyte concentration of the biological sample being tested.

28. The system of claim 27, wherein the plurality of test media are arranged in a plurality of test specific sequences.

29. The system of claim 27, further comprising:

an identification label associated with the portable electronic device,

wherein the digital image includes at least a portion of the identification label, and

wherein the processor is further configured to identify the identification label and to validate the diagnostic instrument based on identification information included on or associated with the identification label.

30. The system of claim 27, further comprising:

a data transmitter associated with the portable electronic device for transmitting at least a portion of the test results to an external source.

31. The system of claim 27, further comprising:

a visual display for displaying instructions for using the system or for displaying the test result.

32. A portable electronic device for analyzing a digital image of a diagnostic instrument, the portable electronic device comprising:

at least one processor to execute program instructions;

at least one display device in communication with the at least one processor;

at least one camera sensor in communication with the at least one processor; and

at least one computer-readable medium in communication with the at least one processor, the at least one computer-readable medium comprising program instructions that, when executed by the at least one processor, cause the portable electronic device to;

capture a digital image of at least a portion of the diagnostic instrument, which has been exposed to a biological sample, with the camera sensor, the diagnostic instrument comprising at least one color reference comprising a plurality of reference samples of different colors and at least one test medium containing a reagent, which changes color in the presence of particular analytes in the biological sample;

identify, in the digital image, at least one of the reference samples for the at least one test medium of the diagnostic instrument;

determine, in the digital image, a dominant camera-captured color of the at least one reference sample and a dominant camera-captured color of the at least one test medium;

color correct the dominant camera-captured color of the at least one test medium based on a correction factor derived at least in part from the dominant camera-captured color of the at least one reference sample to determine a corrected test medium color; and

compare the corrected test medium color to a set of possible test medium colors corresponding to predetermined analyte concentrations to determine a test result including an analyte concentration of the biological sample being tested.

33. The portable electronic device of claim 32, wherein the program instructions further cause the portable electronic device to

display use instructions on the display of the portable electronic device, wherein the use instructions explain to a user how to expose the diagnostic instrument to the biological sample and how to capture the digital image of the diagnostic instrument.

34. The portable electronic device of claim 33, wherein

the use instructions are displayed in conjunction with a video signal of the diagnostic instrument provided by the camera sensor of the portable electronic device.

35. The portable electronic device of claim 32, further comprising:

a timer, and

wherein the program instructions further cause the portable electronic device to;

instruct a user to expose the diagnostic instrument to the biological sample;

measure a predetermined preferred exposure time with the timer; and

instruct the user to capture the digital image with the camera sensor after the predetermined preferred exposure time passes.

36. The portable electronic device of claim 35, wherein the program instructions further cause the portable electronic device to:

measure an actual exposure time with the timer, the actual exposure time being the time between instructing a user to expose the diagnostic instrument to the biological sample and capturing the digital image of the diagnostic instrument; and

record a corrected exposure time based on a difference between the predetermined preferred exposure time and an actual exposure time.

37. The portable electronic device of claim 36, wherein the program instructions further cause the portable electronic device to color correct the dominant camera-captured color of the at least one test medium based on the corrected exposure time.

38. The portable electronic device of claim 32, further comprising

a data entry device, and

wherein the program instructions further cause the portable electronic device to:

receive patient reported information from the data entry device; and

compare the test results with the patient recorded information to determine a possible patient condition.