Image-based disease diagnostics using a mobile device

• US 10,146,909 B2
• Filed: 04/06/2017
• Issued: 12/04/2018
• Est. Priority Date: 04/06/2017
• Status: Active Grant

• Current Assignee: Pfizer Inc.
• Original Assignee: Diassess, Inc. (Pfizer Inc.)
• Inventors: Dimov, Ivan Krastev, Myers, III, Frank B., Waldeisen, John Robert, Mitra, Debkishore
• Primary Examiner(s): Sasaki, Shogo
• Application Number:
  US15/481,349
• Publication Number:
  US 20180293350A1
• Time in Patent Office: 607 Days
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: C12Q 1/701 Specific hybridization probes
• CPC Class Codes: G01N 33/56983 Viruses
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A diagnostic system performs disease diagnostic tests using at least an optical property modifying device and a mobile device. A user provides a biological sample from a patient to the optical property modifying device that reacts with a reagent in one or more reaction chambers of the device. The user captures one or more images of the one or more reaction chambers using an optical sensor of the mobile device. The diagnostic system can determine a quality level of the images based on factors such as skew, scale, focusing, shadowing, or white-balancing. Based on an analysis of the captured image, the diagnostic system can determine a test result of a disease diagnostic test for the patient. The diagnostic system may communicate the test result, as well as instructions for the disease diagnostic test, to the user via the mobile device.

Claims

1. A system for determining results for image-based disease diagnostic tests, the system comprising:

an optical property modifying device for a nucleic acid-based disease diagnostic test, including a plurality of reaction chambers on a surface of the optical property modifying device, and configured to react a biological sample with an optical property modifying reagent, wherein a reaction within the plurality of reaction chambers modifies an optical property of the optical property modifying reagent;

a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor of a mobile device of a user, cause the processor to;

receive at least one image of the optical property modifying device captured by an optical sensor of the mobile device;

for each image of the at least one image, determine whether the surface of the optical property modifying device is shown in the image based on at least one of a known geometric characteristic and a known color characteristic associated with the surface of the optical property modifying device;

select one or more images of the at least one image that are determined to show the surface;

determine a test result for the nucleic acid-based disease diagnostic test based at least in part on the one or more images and at least one of the reaction chambers; and

provide the test result for display on an electronic display of the mobile device.

2. The system of claim 1, wherein the non-transitory computer readable storage medium has further instructions that when executed by the processor cause the processor to:

receive a geographical location of the mobile device from a GPS processor of the mobile device; and

store the geographical location and a timestamp of the geographical location with the test result.

3. The system of claim 2, wherein determining the test result for the nucleic acid-based disease diagnostic test is further based on at least one of the geographical location and the timestamp.

4. The system of claim 1, wherein the known geometric characteristic includes at least one of:

a two-dimensional barcode, a QR code, a border of the optical property modifying device, or an alignment marker.

5. The system of claim 1, wherein the known geometric characteristic includes at least a QR code, and wherein mobile device is further configured to determine a type and an expiration date of the nucleic acid-based disease diagnostic test based on the QR code.

6. The system of claim 1, wherein the non-transitory computer readable storage medium has further instructions that when executed by the processor cause the processor to:

for each image of the at least one image, determine spatial data of the image based on the known geometric characteristic or the known color characteristic, the spatial data indicating at least one of a skew, scale, and rotation of the image; and

wherein selecting the one or more images of the at least one image is further based on the spatial data.

7. The system of claim 1, wherein the non-transitory computer readable storage medium has further instructions that when executed by the processor cause the processor to:

for each image of the at least one image, determine a quality level of the image based on at least one of a focus level, a contrast level, and an illumination level of the image; and

wherein selecting the one or more images of the at least one image is based on the quality levels.

8. The system of claim 1, wherein the non-transitory computer readable storage medium has further instructions that when executed by the processor cause the processor to cause display of a plurality of steps to the user, the steps instructing the user to:

provide the biological sample to the optical property modifying device, wherein the optical property of the optical property modifying reagent is modified in response to the user providing the biological sample to the optical property modifying device;

wait a predetermined duration of time; and

capture the at least one image using the mobile device.

9. The system of claim 8, wherein the optical property modifying device is for an influenza diagnostic test, wherein the biological sample is from a nostril of a patient, and wherein the predetermined duration of time is less than 60 minutes.

10. The system of claim 1, wherein determine the test result for the nucleic acid-based disease diagnostic test based at least in part on the one or more images and at least one of the reaction chambers comprises:

provide the one or more images from the mobile device to a computer server; and

receive the test result from the computer server by the mobile device.

11. The system of claim 1, wherein the electronic display is further configured to:

display a field of view of the optical sensor;

display an image of the at least one image being within the field of view;

display an alignment marker; and

display an indication in response to the image being aligned with the alignment marker.

12. The system of claim 1, wherein the non-transitory computer readable storage medium has further instructions that when executed by the processor cause the processor to:

receive, in response to displaying the test result to the user, a confirmation or a modification of the test result; and

provide the test result and the one or more images to a computer server.

13. A method comprising:

receiving at least one image of an optical property modifying device for a nucleic acid-based disease diagnostic test captured by an optical sensor of a mobile device of a user, the optical property modifying device including a plurality of reaction chambers on a surface of the optical property modifying device and configured to react a biological sample with an optical property modifying reagent, wherein a reaction within the plurality of reaction chambers modifies an optical property of the optical property modifying reagent;

for each image of the at least one image, determining whether at least one surface of the optical property modifying device is shown in the image based on at least one of a known geometric characteristic and a known color characteristic associated with the at least one surface of the optical property modifying device;

selecting one or more images of the at least one image that are determined to show the at least one surface;

determining a test result for the nucleic acid-based disease diagnostic test based at least in part on the one or more images and at least one of the reaction chambers; and

providing the test result for display on an electronic display of the mobile device.

14. The method of claim 13, further comprising:

receiving a geographical location of the mobile device from a GPS processor of the mobile device; and

storing the geographical location and a timestamp of the geographical location with the test result.

15. The method of claim 14, wherein determining the test result for the nucleic acid-based disease diagnostic test is further based on at least one of the geographical location and the timestamp.

16. The method of claim 14, further comprising:

providing the test result, the geographical location, and the timestamp to a computer server including epidemiological data based on aggregated test results from a plurality of mobile devices associated with a population of patients.

17. The method of claim 13, further comprising:

for each image of the at least one image, determining spatial data of the image based on the known geometric characteristic or the known color characteristic, the spatial data indicating at least one of a skew, scale, or rotation of the image; and

wherein selecting the one or more images of the at least one image is further based on the spatial data.

18. The method of claim 13, further comprising:

for each image of the at least one image, determining a quality level of the image based on at least one of a focus level, a contrast level, and an illumination level of the image; and

wherein selecting the one or more images of the at least one image is based on the quality levels.

19. The method of claim 13, further comprising displaying a plurality of steps to the user, the steps instructing the user to:

provide the biological sample to the optical property modifying device, wherein the optical property of the optical property modifying reagent is modified in response to the user providing the biological sample to the optical property modifying device;

wait a predetermined duration of time; and

capture the at least one image using the mobile device.

20. The method of claim 13, further comprising:

causing display of a field of view of the optical sensor on the electronic display;

causing display of an image of the at least one image being within the field of view on the electronic display;

causing display of an alignment marker on the electronic display; and

causing display of an indication in response to the image being aligned with the alignment marker on the electronic display.

21. The method of claim 13, further comprising:

providing, in response to the test result indicating that a patient has tested positive for a disease associated with the nucleic acid-based disease diagnostic test, a notification to the mobile device indicating that a health care provider will be contacted for the patient regarding the test result.

22. The method of claim 13, further comprising:

providing information associated with the test result to a pharmacy to allow the pharmacy to fulfill a prescription for a patient to treat a disease associated with the nucleic acid-based disease diagnostic test.

23. The method of claim 13, further comprising:

generating a message including sponsored content based on the test result; and

providing the message to the mobile device for display to the user.

24. The method of claim 13, further comprising:

in response to the test result indicating that a patient has tested positive for a disease associated with the nucleic acid-based disease diagnostic test, providing the test result to a health care provider located within a predetermined distance from the patient; and

providing, to the mobile device for display to the user, a recommendation from the health care provider to treat the disease.

25. The method of claim 13, further comprising:

providing the test result to a computer server of a third party including health information based on aggregated test results from a plurality of mobile devices associated with a population of patients, wherein the third party generates health statistics of the population of patients using the health information.

26. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

receive at least one image of an optical property modifying device for a nucleic acid-based disease diagnostic test captured by an optical sensor of a mobile device of a user, the optical property modifying device including a plurality of reaction chambers on a surface of the optical property modifying device and configured to react a biological sample with an optical property modifying reagent, wherein a reaction within the plurality of reaction chambers modifies an optical property of the optical property modifying reagent;

for each image of the at least one image, determining whether at least one surface of the optical property modifying device is shown in the image based on at least one of a known geometric characteristic and a known color characteristic associated with the at least one surface of the optical property modifying device;

select one or more images of the at least one image that are determined to show the at least one surface;

determine a test result for the nucleic acid-based disease diagnostic test based at least in part on the one or more images and at least one of the reaction chambers; and

provide the test result for display on an electronic display of the mobile device.

27. The non-transitory computer readable storage medium of claim 26, having further instructions that when executed by the processor cause the processor to:

receive a geographical location of the mobile device from a GPS processor of the mobile device; and

store the geographical location and a timestamp of the geographical location with the test result.

28. The non-transitory computer readable storage medium of claim 26, having further instructions that when executed by the processor cause the processor to:

for each image of the at least one image, determine a quality level of the image based on at least one of a focus level, a contrast level, and an illumination level of the image; and

wherein selecting the one or more images of the at least one image is further based on the quality levels.

29. The non-transitory computer readable storage medium of claim 26, having further instructions that when executed by the processor cause the processor to display a plurality of steps to the user, the steps instructing the user to:

provide the biological sample to the optical property modifying device, wherein the optical property of the optical property modifying reagent is modified in response to the user providing the biological sample to the optical property modifying device;

wait a predetermined duration of time; and

capture the at least one image using the mobile device.

30. The non-transitory computer readable storage medium of claim 26, having further instructions that when executed by the processor cause the processor to:

display a field of view of the optical sensor on the electronic display;

display an image of the at least one image being within the field of view on the electronic display;

display an alignment marker on the electronic display; and

display an indication in response to the image being aligned with the alignment marker on the electronic display.