Method and system for automated visual analysis of a dipstick using standard user equipment

• US 10,068,329 B2
• Filed: 02/23/2016
• Issued: 09/04/2018
• Est. Priority Date: 05/12/2014
• Status: Active Grant

• Current Assignee: Healthy.io, Ltd.
• Original Assignee: Healthy.io, Ltd.
• Inventors: Adiri, Yonatan, Omer, Ido, Mendelowitz, Shachar, Salomon, Roee
• Primary Examiner(s): Wang, Carol
• Application Number:
  US15/050,710
• Publication Number:
  US 20160171690A1
• Time in Patent Office: 924 Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model

Abstract

A method for analyzing a dipstick using a smartphone is provided herein. The method includes: capturing an image containing: a dipstick having colored test reagents, and a color board having a grid of grey patches and a plurality of reference color patches; deriving, based on the grid, local illumination parameters associated with the colored test reagents and the plurality of reference color patches; determining whether illumination parameters are sufficient for interpreting the colored test reagent; in a case the illumination parameters are insufficient, notifying a user that an interpretation of the colored test reagents is not possible, otherwise, applying one or more image enhancement operation to the captured image, based on an analysis of the grid of grey patches, to yield enhanced reference color patches; and interpreting the one or more colored test reagents, by projecting the colored test reagents onto a vector of the enhanced reference color patches.

Claims

1. A method comprising:

capturing, using a capturing device of a mobile device, a captured image containing;

a dipstick having colored test reagents, and a color board comprising a grid of grey patches and a plurality of sets of reference color patches, said reference color patches reflecting variations in one of;

illumination conditions, capturing capabilities of potential mobile devices, and response functions of capturing devices of the potential mobile devices, wherein each of the sets of reference color patches is tailored to span a color gamut of a respective one of said colored test reagents;

deriving, based on the plurality of sets of reference color patches and the grid of grey patches on the captured image, local illumination parameters associated with the colored test reagents and the plurality of sets of reference color patches;

normalizing local illumination of the captured image, based on the local illumination parameters; and

interpreting the colored test reagents, based on the normalized local illumination and the reference color patches, in the captured image.

2. The method according to claim 1, wherein in a case that the local illumination parameters are not within a set of predefined illumination boundary conditions, instructing the user how to improve the local illumination parameters.

3. The method according to claim 1, further comprising applying one or more image enhancement operations on the captured image, to yield an enhanced image, wherein the one or more image enhancement operations comprises detecting portions of specular reflections coming from the reference color patches or the colored test reagents and applying image processing algorithms that reduces the specular reflections.

4. The method according to claim 3, wherein the detecting of portions of specular reflections is carried out by comparing a distribution of pixels associated with a same one of the colored test reagents or the reference color patches, to a predefined threshold.

5. The method according to claim 1, further comprising applying one or more image enhancement operations on the captured image, to yield an enhanced image, wherein the one or more image enhancement operations comprises normalizing pixel colors of each of the colored test reagents, to yield a uniform color, based on a normalization factor calculated based on the local illumination parameters and image capturing and processing capabilities of the mobile device.

6. The method according to claim 1, further comprising applying one or more image enhancement operations on the captured image, to yield an enhanced image, wherein the one or more image enhancement operations comprises normalizing pixel colors of the colored test reagents and the reference color patches, to achieve a uniform illumination.

7. The method according to claim 1, wherein the colored test reagents and the reference color patches are located, based on a specified layout, in prearranged locations.

8. The method according to claim 1, wherein the color board is generated based on data derived from a series of trials, in which a plurality of illumination conditions are tested against a plurality of different mobile devices.

9. The method according to claim 8, wherein the color board is generated per the dipstick based on the colored test reagents of the dipstick.

10. The method according to claim 1, wherein at least one of shape, color, location, and texture of the reference color patches on the color board are selected in an optimization process, configured to increase a likelihood of a successful interpretation of the colored test reagents.

11. A system comprising:

a mobile device having specified image capturing and processing capabilities, configured to capture an image containing a dipstick having colored test reagents, and a color board comprising a grid of grey patches and a plurality of sets of reference color patches, said reference color patches reflecting variations in one of;

illumination conditions, capturing capabilities of potential mobile devices, and response functions of capturing devices of the potential mobile devices, wherein each of the sets of reference color patches is tailored to span a color gamut of a respective one of said colored test reagents; and

a computer processor configured to;

derive, based on the plurality of sets of reference color patches and the grid of grey patches on the captured image, local illumination parameters associated with the colored test reagents and the plurality of sets of reference color patches;

normalize local illumination of the captured image, based on the local illumination parameters; and

interpret the colored test reagents, based on the normalized local illumination and the reference color patches, in the captured image.

12. The system according to claim 11, wherein in a case that the local illumination parameters are not within a set of predefined illumination boundary conditions, instructing the user how to improve the local illumination parameters.

13. The system according to claim 11, wherein the computer processor is further configured to apply one or more image enhancement operations on the captured image, to yield an enhanced image, and wherein the one or more image enhancement operations comprises detecting portions of specular reflections coming from the reference color patches or the colored test reagents and applying image processing algorithms that reduces the specular reflections.

14. The system according to claim 13, wherein the detecting of portions of specular reflections is carried out by comparing a distribution of pixels associated with a same one of the colored test reagents or the reference color patches, to a predefined threshold.

15. The system according to claim 11, wherein the computer processor is further configured to apply one or more image enhancement operations on the captured image, to yield an enhanced image, and wherein the one or more image enhancement operations comprises normalizing pixel colors of each of the colored test reagents, to yield a uniform color, based on a normalization factor calculated based on the local illumination parameters and the image capturing and processing capabilities of the mobile device.

16. The system according to claim 11, wherein the computer processor is further configured to apply one or more image enhancement operations on the captured image, to yield an enhanced image, and wherein the one or more image enhancement operations comprises normalizing pixel colors of the colored test reagents and the reference color patches to achieve a uniform illumination.

17. The system according to claim 11, wherein the colored test reagents and the reference color patches are located, based on a specified layout, in prearranged locations.

18. The system according to claim 11, wherein the color board is generated based on data derived from a series of trials, in which a plurality of illumination conditions are tested against a plurality of different mobile devices.

19. The system according to claim 18, wherein the color board is generated per the dipstick based on the colored test reagents of the dipstick.

20. The system according to claim 11, wherein at least one of:

shape, color, location, and texture of the reference color patches on the color board are selected in an optimization process, configured to increase a likelihood of a successful interpretation of the colored test reagents.