Machine vision and spectroscopic pharmaceutical verification

• US 8,417,539 B2
• Filed: 12/02/2008
• Issued: 04/09/2013
• Est. Priority Date: 04/16/2003
• Status: Expired due to Fees

• Current Assignee: ASB International Services Limited
• Original Assignee: Optopo Inc.
• Inventors: Chapman, Kenneth Wayne, Cull, Evan C., Potuluri, Prasant, Brady, David J., Stranzl, John E. Jr.
• Primary Examiner(s): NGUYEN, HIEP VAN
• Application Number:
  US12/326,558
• Publication Number:
  US 20090080735A1
• Time in Patent Office: 1,589 Days
• Field of Search: 705/2
• US Class Current: 705/2
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G16H 20/13 delivered from dispensers
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

A pharmaceutical solid in a prescription vial is identified from an optical property of the pharmaceutical solid using light reflected from two different light sources. For each known pharmaceutical solid, an optical property of the known pharmaceutical solid is stored. The prescription vial is illuminated with a first light source and a first image is recorded. The prescription vial is then illuminated with a second light source and a second image is recorded. The first image and the second image are processed to find an optical property of the pharmaceutical solid. The optical property found is compared to the stored optical properties. The identity of the pharmaceutical solid is determined from the comparison. The first light source and the second light source are selected to remove artifacts of the prescription bottle or to enhance or suppress two-dimensional or three-dimensional effects on the surface of the pharmaceutical solid.

Claims

1. A method for identifying a pharmaceutical solid in a closed prescription vial, comprising:

storing a plurality of known optical properties corresponding to known pharmaceutical solids;

illuminating the closed prescription vial with a first light source that is substantially transmitted by artifacts of the closed prescription vial and recording a first image through the closed prescription vial using an imaging device, wherein artifacts of the closed prescription vial include lettering on the prescription vial or indentations in the closed prescription vial;

illuminating the closed prescription vial with a second light source that is substantially reflected by the artifacts and recording a second image through the closed prescription vial using the imaging device;

illuminating the closed prescription vial with a third light source and recording a third image through the closed prescription vial using an imaging device;

illuminating the closed prescription vial with a fourth light source and recording a fourth image through the closed prescription vial using the imaging device;

using the second image to remove the effects of the artifacts in the first image and processing the first image, the third image, and the fourth image to find an optical property of the pharmaceutical solid;

performing a first comparison of the optical property to one or more of the plurality of stored optical properties; and

determining, using a processor, an identity of the pharmaceutical solid from the first comparison.

2. The method of claim 1, wherein the third light source comprises a directional light source and the fourth light source comprises a diffuse light source.

3. The method of claim 1, wherein the third light source comprises infrared light source.

4. The method of claim 1, wherein the fourth light source comprises visible light source.

5. The method of claim 4, further comprising

storing a plurality of colors corresponding to the known pharmaceutical solids,

processing the fourth image to find a color of the pharmaceutical solid,

performing a second comparison of the color to one or more of the plurality of stored colors, and

determining the identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

6. The method of claim 5, wherein processing the fourth image to find the color comprises

selecting pixels from a portion of the fourth image that correspond to a portion of the prescription vial substantially free of artifacts of the prescription vial and

statistically correlating color information in each of the selected pixels to determine the color.

7. The method of claim 1, wherein the pharmaceutical solid comprises a pill.

8. The method of claim 1, wherein the pharmaceutical solid comprises a capsule.

9. The method of claim 1, further comprising using the third image and the fourth image to determine an amount of the pharmaceutical solid in the prescription vial.

10. The method of claim 1, wherein illuminating the prescription vial with a third light source comprises illuminating a bottom of the prescription vial with the third light source and wherein illuminating the prescription vial with a fourth light source comprises illuminating the bottom of the prescription vial with the fourth light source.

11. The method of claim 1, wherein processing the third image and the fourth image to find the optical property comprises identifying geometric patterns formed by edges in the third image and the fourth image that represent shapes of pharmaceutical solids.

12. The method of claim 11, further comprising identifying a geometric pattern in a first location in the third image and using grayscale statistics of pixels in the geometric pattern to create a score for the geometric pattern.

13. The method of claim 12, further comprising identifying the geometric pattern in a second location in the third image and using grayscale statistics of pixels in the geometric pattern in the second location to modify the score.

14. The method of claim 11, further comprising selecting a geometric pattern from the identified geometric patterns in a first location in the fourth image and using color statistics of pixels in the geometric pattern to create a score for the geometric pattern.

15. The method of claim 14, further comprising selecting the geometric pattern in a second location in the fourth image and using color statistics of pixels in the geometric pattern to modify the score.

16. The method of claim 11, further comprising analyzing the fourth image using a color space and for each channel of the color space identifying edges, accumulating the identified edges from the each channel, and combining the accumulated edges into the identified geometric patterns.

17. The method of claim 16, wherein combining the accumulated edges into the identified geometric patterns comprises adding edges from color channels that are substantially transmitted by artifacts of the prescription vial and subtracting edges from color channels that are substantially reflected by the artifacts.

18. The method of claim 17, wherein the artifacts comprise one or more of lettering on the prescription via, indentations in the prescription vial, and debris on the prescription vial.

19. The method of claim 1, further comprising

storing a plurality of spectral signatures corresponding to the known pharmaceutical solids,

measuring a spectral signature of the pharmaceutical solid using a multimode multiplex spectrometer,

performing a second comparison of the measured spectral signature to one or more of the plurality of stored spectral signatures, and

determining the identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

20. The method of claim 19, wherein determining the identity of the pharmaceutical solid from the first comparison in addition to the second comparison comprises selecting a known pharmaceutical solid that most closely matches the optical property and the measured spectral signature.

21. The method of claim 19, further comprising using the second comparison to produce one or more known pharmaceutical solids that match the measured spectral signature and using the one or more known pharmaceutical solids to select the one or more of the plurality of stored optical properties used in the first comparison.

22. The method of claim 19, further comprising using the first comparison to produce one or more known pharmaceutical solids that match the optical property and using the one or more known pharmaceutical solids to select the one or more of the plurality of stored spectral signatures used in the second comparison.

23. The method of claim 19, further comprising using the third image and the fourth image to determine a location to direct a laser of the multimode multiplex spectrometer.

24. The method of claim 1, further comprising

storing a plurality of geometric engravings corresponding to the known pharmaceutical solids,

processing the third image to find a geometric engraving on the pharmaceutical solid,

performing a second comparison of the geometric engraving to one or more of the plurality of stored geometric engravings, and

determining the identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

25. The method of claim 24, wherein the geometric engraving comprises an indentation for identification.

26. The method of claim 24, wherein the geometric engraving comprises an indentation for a function other than identification.

27. The method of claim 1, further comprising

storing a plurality of coplanar symbols corresponding to the known pharmaceutical solids,

processing the fourth image to find a coplanar symbol on the pharmaceutical solid,

performing a second comparison of the geometric engraving to one or more of the plurality of stored coplanar symbols, and

determining the identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

28. The method of claim 27, wherein the coplanar symbol comprises a printed symbol on the pharmaceutical solid.

29. The method of claim 1, wherein the first light source is the third light source and the second light source is the fourth light source.

30. A method for identifying a pharmaceutical solid in a closed prescription vial, comprising:

storing a plurality of known optical properties corresponding to known pharmaceutical solids;

illuminating the closed prescription vial with a first light source that is substantially transmitted by artifacts of the closed prescription vial and recording a first image through the closed prescription vial using an imaging device, wherein artifacts of the closed prescription vial include lettering on the prescription vial or indentations in the closed prescription vial;

illuminating the closed prescription vial with a second light source that is substantially reflected by the artifacts and recording a second image through the closed prescription vial using the imaging device;

using the second image to remove the effects of the artifacts in the first image and processing the first image to find an optical property of the pharmaceutical solid;

performing a first comparison of the optical property to one or more of the plurality of stored known optical properties; and

determining, using a processor, an identity of the pharmaceutical solid from the first comparison.

31. The method of claim 30, wherein the prescription vial comprises an amber vial.

32. The method of claim 30, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known shapes corresponding to known pharmaceutical solids and the optical property comprises a shape.

33. The method of claim 30, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known colors corresponding to known pharmaceutical solids and the optical property comprises a color.

34. The method of claim 30, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known geometric engravings corresponding to known pharmaceutical solids and the optical property comprises a geometric engraving.

35. The method of claim 30, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known coplanar symbols corresponding to known pharmaceutical solids and the optical property comprises a coplanar symbol.

36. The method of claim 30, further comprising

storing a plurality of spectral signatures corresponding to the known pharmaceutical solids,

measuring a spectral signature of the pharmaceutical solid using a multimode multiplex spectrometer,

performing a second comparison of the measured spectral signature to one or more of the plurality of stored spectral signatures, and

determining an identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

37. A method for identifying a pharmaceutical solid in a closed prescription vial, comprising:

storing a plurality of known shapes corresponding to known pharmaceutical solids;

selecting a first direction and a second direction;

illuminating the closed prescription vial with a first light source that is substantially transmitted by artifacts of the closed prescription vial and recording a first image through the closed prescription vial using an imaging device, wherein artifacts of the closed prescription vial include lettering on the prescription vial or indentations in the closed prescription vial;

illuminating the closed prescription vial with a second light source that is substantially reflected by the artifacts and recording a second image through the closed prescription vial using the imaging device;

illuminating the closed prescription vial with a third light from the first direction and recording a third image through the closed prescription vial using an imaging device;

illuminating the closed prescription vial with a fourth light from the second direction and recording a fourth image through the closed prescription vial using the imaging device;

using the second image to remove the effects of the artifacts in the first image and processing the first image, the third image, and the fourth image to find a shape of the pharmaceutical solid;

performing a first comparison of the shape to one or more of the plurality of known shapes; and

determining, using a processor, an identity of the pharmaceutical solid from the first comparison.

38. The method of claim 37, further comprising selecting the first direction and the second direction to maximize areas of maximum reflectivity and maximize areas of minimum reflectivity in the third image and the fourth image.

39. The method of claim 37, further comprising selecting the first direction and the second direction to minimize areas of maximum reflectivity and minimize areas of minimum reflectivity in the third image and the fourth image.

40. The method of claim 37, further comprising storing a plurality of known optical properties corresponding to known pharmaceutical solids and processing the third image and the fourth image to find an optical property of the pharmaceutical solid in addition to shape.

41. The method of claim 40, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known colors corresponding to known pharmaceutical solids and the optical property comprises a color.

42. The method of claim 40, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known geometric engravings corresponding to known pharmaceutical solids and the optical property comprises a geometric engraving.

43. The method of claim 40, wherein the plurality of known optical properties corresponding to known pharmaceutical solids comprises a plurality of known coplanar symbols corresponding to known pharmaceutical solids and the optical property comprises a coplanar symbol.

44. The method of claim 37, further comprising

storing a plurality of spectral signatures corresponding to the known pharmaceutical solids,

measuring a spectral signature of the pharmaceutical solid using a multimode multiplex spectrometer,

performing a second comparison of the measured spectral signature to one or more of the plurality of stored spectral signatures, and

determining an identity of the pharmaceutical solid from the second comparison in addition to the first comparison.

45. The method of claim 37, wherein the first light source is the third light source and the second light source is the fourth light source.

46. A method for identifying a pharmaceutical solid in a closed prescription vial, comprising:

storing a plurality of known optical properties corresponding to known pharmaceutical solids;

illuminating the closed prescription vial with a light source that includes a light frequency that is substantially reflected by artifacts of the closed prescription vial, wherein artifacts of the closed prescription vial include lettering on the prescription vial or indentations in the closed prescription vial;

recording an image through the closed prescription vial using an imaging device;

using edges recorded in the image from the light frequency to remove the effects of the artifacts in the image and processing the image to find an optical property of the pharmaceutical solid;

performing a first comparison of the optical property to one or more of the plurality of stored known optical properties; and

determining, using a processor, an identity of the pharmaceutical solid from the first comparison.

47. The method of claim 46, further comprising

storing a plurality of spectral signatures corresponding to the known pharmaceutical solids,

measuring a spectral signature of the pharmaceutical solid using a multimode multiplex spectrometer,

performing a second comparison of the measured spectral signature to one or more of the plurality of stored spectral signatures, and

determining an identity of the pharmaceutical solid from the second comparison in addition to the first comparison.