METHOD AND SYSTEM FOR AUTOMATED VISUAL ANALYSIS OF A DIPSTICK USING STANDARD USER EQUIPMENT

• US 20180374215A1
• Filed: 09/03/2018
• Published: 12/27/2018
• Est. Priority Date: 05/12/2014
• Status: Active Grant

• Current Assignee: Healthy.io, Ltd.
• Original Assignee: Healthy.io, Ltd.
• Inventors: OMER, Ido, ADIRI, Yonatan, MENDELOWITZ, Shachar, SALOMON, Roee
• Granted Patent: US 10,559,081 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model

Abstract

A method for analyzing a dipstick using a smartphone is provided herein. The method includes: capturing an image containing: a dipstick having colored test reagents, and a color board having a grid of grey patches and a plurality of reference color patches; deriving, based on the grid, local illumination parameters associated with the colored test reagents and the plurality of reference color patches; determining whether illumination parameters are sufficient for interpreting the colored test reagent; in a case the illumination parameters are insufficient, notifying a user that an interpretation of the colored test reagents is not possible, otherwise, applying one or more image enhancement operation to the captured image, based on an analysis of the grid of grey patches, to yield enhanced reference color patches; and interpreting the one or more colored test reagents, by projecting the colored test reagents onto a vector of the enhanced reference color patches.

Claims

1. A method comprising:

capturing, using a capturing device of a standard user equipment (UE) having specified image capturing and processing capabilities, an image of a dipstick having one or more colored test reagents, positioned on a calibration array having a plurality of colored calibration elements, wherein said plurality of sets of colored calibration elements reflects a plurality of;

illumination conditions, capturing capabilities of potential UEs and response functions of capturing devices of the potential UEs;

deriving, based on the colored calibration elements, local illumination parameters associated with the captured image of the dipstick and the calibration array;

applying one or more image enhancement operation to the captured image, to yield an enhanced image, based on predefined mapping between the derived local illumination parameters and one or more required adjustments;

normalizing illumination of the captured image based on the local illumination parameters; and

interpreting the one or more colored test reagents, based on the normalized illumination and the colored calibration elements, in the enhanced image.

2. The method according to claim 1, further comprising, in a case that the illumination parameters are not within a set of predefined illumination boundary, instructing a user of the UE how to improve the illumination parameters.

3. The method according to claim 1, wherein the one or more image enhancement operations comprises detecting portions of specular reflections coming from the colored calibration elements or the colored test reagents, and applying image processing algorithms that reduces the specular reflections.

4. The method according to claim 1, wherein the one or more image enhancement operation comprises normalizing a color of each pixel of each reagent, to yield, a uniform color, based on a normalization factor calculated based on the derived illumination parameters and the specified image capturing and processing capabilities.

5. The method according to claim 1, wherein the one or more image enhancement operation comprises normalizing a color of each pixel of each reagent and calibration elements to achieve uniform illumination.

6. The method according to claim 1, further comprising, in a case that the illumination parameters are not within a predefined illumination boundary, indicating to a user that a proper interpreting of the dipstick by the arbitrary platform is not possible.

7. The method according to claim 1, wherein the one or more colored test reagents, and the colored calibration elements are located, based on a specified layout, in prearranged locations.

8. The method according to claim 1, wherein the calibration array is generated based on data derived in a series of trials and error, in which a plurality of illumination conditions were tested against a plurality of UEs.

9. The method according to claim 1, wherein at least one of:

shape, color, location, and texture of the calibration elements on the calibration array are selected in an optimization process, configured to increase likelihood of a successful interpretation of the colored test reagents.

10. The method according to claim 2, wherein the instruction to the user indicate a specified movement pattern of the UE vis à

vis the dipstick and the calibration array.

11. The method according to claim 3, wherein the detecting of portions of specular reflections is carried out by comparing pixels associated with a same color element or reagent, to a predefined threshold.

12. A system comprising:

a user equipment (UE) having specified image capturing and processing capabilities, configured to capture, using an image capturing device, an image of a dipstick having one or more colored test reagents, positioned on a calibration array having a plurality of colored calibration elements, wherein said plurality of sets of colored calibration elements reflects a plurality of;

illumination conditions, capturing capabilities of potential UEs and response functions of capturing devices of the potential UEs; and

a computer processor configured to;

derive, based on the colored calibration elements in the captured image, local illumination parameters associated with the captured image of the dipstick and the calibration array;

apply one or more image enhancement operation to the captured image, to yield an enhanced image, based on predefined mapping between the derived local illumination parameters and one or more required adjustments;

normalize illumination of the captured image based on the local illumination parameters; and

interpret the one or more colored test reagents, based on the normalized illumination parameters and the colored calibration elements, in the enhanced captured image.

13. The system according to claim 12, wherein the computer processor is further configured, in a case that the illumination parameters are not within a predefined illumination boundary to instruct a user of the arbitrary platform how to improve the illumination parameters.

14. The system according to claim 12, wherein the one or more image enhancement operation comprises detecting portions of specular reflections coming from the colored calibration elements or the colored test reagents, and applying image processing algorithms that reduces the specular reflections.

15. The system according to claim 12, wherein the one or more image enhancement operation comprises determining, for each pixel at the captured image associated with one of the colored test reagents, a uniform color, based on a normalization factor calculated based on the derived illumination parameters and the specified image capturing and processing capabilities.

16. The system according to claim 12, wherein the one or more image enhancement operation comprises normalizing the pixel colors of the reagent and calibration elements to achieve uniform illumination.

17. The system according to claim 12, wherein the computer processor is further configured, in a case that the illumination parameters are not within a predefined illumination boundary to indicate to a user that interpreting of the dipstick by the arbitrary platform is not possible.

18. The system according to claim 12, wherein the one or more colored test reagents, and the colored calibration elements are located, based on a specified layout, in prearranged locations.

19. The system according to claim 12, wherein the calibration array is generated based on data derived in a series of trial and error, in which a plurality of illumination conditions were tested.

20. A computer program product comprising:

a non-transitory computer readable storage medium having a computer readable program embodied therewith, the computer readable program comprising;

a computer readable program configured to instruct a user equipment (UE) having specified image capturing and processing capabilities, to capture an image of a dipstick having one or more colored test reagents, positioned on a calibration array having a plurality of colored calibration elements, wherein said plurality of sets of colored calibration elements reflects a plurality of;

illumination conditions, capturing capabilities of potential UEs and response functions of capturing devices of the potential UEs;

a computer readable program configured to derive, based on the colored calibration elements in the captured image, local illumination parameters associated with the captured image of the dipstick and the calibration array;

a computer readable program configured to determine whether the local illumination parameters are within a predefined illumination boundary which is sufficient for interpreting the one or more colored test reagents, given the specified image capturing and processing capabilities of the UE;

a computer readable program configured to apply one or more image enhancement operation to the captured image, to yield an enhanced image, based on predefined mapping between the derived local illumination parameters and one or more required adjustments;

a computer readable program configured to normalize illumination of the captured image based on the local illumination parameters; and

a computer readable program configured to interpret the one or more colored test reagents, based on the normalized illumination parameters and the colored calibration elements, in the enhanced image.