Image processing techniques for plunger depth measurement

• US 9,881,367 B1
• Filed: 08/09/2017
• Issued: 01/30/2018
• Est. Priority Date: 08/09/2017
• Status: Active Grant

• Current Assignee: Amgen Inc.
• Original Assignee: Amgen Inc.
• Inventors: Milne, Graham F., Pearson, Thomas Clark, Freund, Erwin
• Primary Examiner(s): Park, Soo
• Application Number:
  US15/672,494
• Time in Patent Office: 174 Days
• Field of Search: None
• CPC Class Codes: A61M 2209/02 Equipment for testing the apparatus
• CPC Class Codes: A61M 5/315 Pistons; Piston-rods; Guiding, blocking or restricting the movement of the rod or piston; Appliances on the rod for facilitating dosing ; Dosing mechanisms
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

An automated visual inspection (AVI) system is described to measure the depth of plungers positioned within vessels. The AVI system implements various types of image processing as well as other techniques to identify, for a vessel, a reference point and the uppermost edge of the plunger. Because different inconsistencies and/or artifacts may be introduced within the vessel image, different processing techniques may be executed on the region containing the reference point versus the plunger region. In doing so, the AVI system provides a robust means to accurately measure plunger depth that is highly resilient to the aforementioned artifacts and consistently measures plunger depth for a wide variety and types of vessels.

Claims

1. A method for performing plunger depth inspection for a vessel having a plunger, comprising:

acquiring, via an imager, an image of the vessel including a reference point region of interest (ROI) that encompasses an image of a reference point from which to measure plunger depth, and an initial plunger ROI that encompasses an image of the plunger;

generating, via a controller, a binarized reference point image based upon a blob analysis of the reference point ROI;

calculating, via the controller, a vessel edge within the binarized reference point image;

aligning, via the controller, a refined plunger ROI with the top of the plunger within the initial plunger ROI using a matching plunger image template from among a plurality of plunger image templates;

calculating, via the controller, a plunger edge within the refined plunger ROI; and

calculating, via the controller, a depth of the plunger within the vessel based upon the vessel edge and the plunger edge.

2. The method of claim 1, wherein the act of generating the binarized reference point image comprises:

applying intensity thresholding and a closing algorithm to the reference point ROI to generate the binarized reference point image.

3. The method of claim 1, wherein the act of generating the binarized reference point image comprises:

scanning horizontal rows within the binarized reference point image to identify falling-edge pixel transitions and rising-edge pixel transitions, the falling-edge pixel transitions representing a transition from a binary value of 1 to a binary value of 0, and the rising-edge pixel transitions representing a transition from a binary value of 0 to a binary value of 1;

identifying, from within each scanned horizontal row, pairs of falling-edge pixel transitions and rising-edge pixel transitions; and

adjusting, within each scanned horizontal row, values of pixels between the pairs of falling-edge pixel transitions and rising-edge pixel transitions from a binary value of 0 to a binary value of 1.

4. The method of claim 1, further comprising:

identifying, via the controller, the matching plunger image template using a greyscale value pyramid template matching algorithm.

5. The method of claim 1, wherein the act of calculating the plunger edge within the refined plunger ROI comprises:

dividing the top of the refined plunger ROI into a plurality of columns;

performing one-dimensional (1D) edge-detection on each of the plurality of columns to determine, for each column, a y-coordinate value associated with an upper edge of the plunger; and

calculating the uppermost plunger edge within the refined plunger ROI based upon the y-coordinate value associated with the upper edge of the plunger in each of the plurality of columns.

6. The method of claim 5, wherein the act of calculating the plunger edge within the refined plunger ROI further comprises:

binning the y-coordinate value in each of the plurality of columns in accordance with a histogram according to y-coordinate values;

removing bins having less than a threshold number of elements to generate a constrained set of y-coordinate values; and

calculating the uppermost plunger edge within the refined plunger ROI based upon the constrained set of y-coordinate values.

7. The method of claim 6, wherein the act of calculating the plunger edge within the refined plunger ROI further comprises:

applying a best-fit-line algorithm to the constrained set of y-coordinate values.

8. An automated visual inspection (AVI) system, comprising:

an imager configured to acquire an image of each of a plurality of syringes in a sequential manner to generate a plurality of syringe images, each of the plurality of syringes having a respective flange and plunger;

a stage configured to receive a tray containing the plurality of syringes and to incrementally move the tray to facilitate sequential imaging of the plurality of syringes by the imager; and

a controller configured to, for each sequentially acquired syringe image from among the plurality of syringe images;

perform an image analysis of a flange region of interest (ROI) that encompasses the flange within the syringe image to generate a binarized flange image;

identify a plunger ROI that is aligned with a top of the plunger within the syringe image using a matching plunger image template from among a plurality of plunger image templates;

calculate an uppermost flange edge within the binarized flange image;

calculate an uppermost plunger edge within the plunger ROI; and

calculate a depth of the plunger within the syringe based upon the uppermost flange edge and the uppermost plunger edge.

9. The AVI system of claim 8, wherein the image analysis of the flange ROI is a blob analysis.

10. The AVI system of claim 8, wherein the image analysis of the flange ROI to generate the binarized flange image includes a closing algorithm to remove artifacts and fill voids within the binarized flange image.

11. The AVI system of claim 8, wherein the image analysis of the flange ROI to generate the binarized flange image includes:

scanning horizontal rows within the binarized flange image to identify falling-edge pixel transitions and rising-edge pixel transitions, the falling-edge pixel transitions representing a transition from a binary value of 1 to a binary value of 0, and the rising-edge pixel transitions representing a transition from a binary value of 0 to a binary value of 1;

identifying, from within each scanned horizontal row, pairs of falling-edge pixel transitions and rising-edge pixel transitions; and

adjusting, within each scanned horizontal row, values of pixels between the pairs of falling-edge pixel transitions and rising-edge pixel transitions from a binary value of 0 to a binary value of 1.

12. The AVI system of claim 8, wherein the controller is further configured to identify the matching plunger image template using a greyscale value pyramid template matching algorithm.

13. The AVI system of any one of claim 8, wherein the act of calculating the uppermost plunger edge within the plunger ROI comprises:

dividing the top of the plunger ROI into a plurality of columns;

performing one-dimensional (1D) edge-detection on each of the plurality of columns to determine, for each column, a y-coordinate value associated with an upper edge of the plunger; and

calculating the uppermost plunger edge within the plunger ROI based upon the y-coordinate value associated with the upper edge of the plunger in each of the plurality of columns.

14. A tangible, non-transitory, computer-readable medium storing instructions on a computing device that, when executed by one or more processors associated with the computing device, cause the computing device to:

acquire an image of a syringe having a flange and a plunger;

calculate an uppermost flange edge within the image of the syringe;

match an image of the plunger within the image of the syringe to a matching plunger image template from among a plurality of plunger image templates;

align a plunger ROI with the top of the plunger within the image of the syringe using the matching plunger image template;

calculate an uppermost plunger edge associated with the image of the plunger within the plunger ROI; and

calculate a depth of the plunger within the syringe based upon the uppermost flange edge and the uppermost plunger edge.

15. The tangible, non-transitory, computer-readable medium of claim 14, wherein the image of the syringe includes a flange region of interest (ROI) that encompasses an image of the flange, and

wherein the instructions to calculate the uppermost flange edge within the image of the syringe include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to:

generate a binarized flange image based upon a blob analysis of the flange ROI, and

calculate the uppermost flange edge within the image as the uppermost flange edge within the binarized flange image.

16. The tangible, non-transitory, computer-readable medium of claim 15, wherein the instructions to calculate the uppermost flange edge within the image of the syringe further include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to execute a closing algorithm to remove artifacts and to fill voids within the binarized flange image.

17. The tangible, non-transitory, computer-readable medium of claim 14, wherein the instructions to match the image of the plunger to the matching plunger image template further include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to execute a greyscale value pyramid template matching algorithm.

18. The tangible, non-transitory, computer-readable medium of claim 14, wherein the instructions to calculate the uppermost plunger edge further include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to perform one-dimensional (1D) edge-detection on each of a plurality of evenly-divided columns within the plunger ROI to determine, for each column, a y-coordinate value associated with an upper edge of the plunger, and to calculate the uppermost plunger edge within the refined plunger ROI based upon the y-coordinate value associated with the upper edge of the plunger in each of the plurality of evenly-divided columns.

19. The tangible, non-transitory, computer-readable medium of claim 18, wherein the instructions to calculate the uppermost plunger edge further include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to calculate the uppermost plunger edge based upon a constrained set of y-coordinate values that are selected in accordance with a histogram grouping of the y-coordinate values from each of the plurality of evenly-divided columns.

20. The tangible, non-transitory, computer-readable medium of claim 19, wherein the instructions to calculate the uppermost plunger edge further include instructions that, when executed by the one or more processors associated with the computing device, cause the computing device to apply a best-fit-line algorithm to the constrained set of y-coordinate values.