Methods and systems for imaging and classifying targets as empty or non-empty

• US 7,596,275 B1
• Filed: 03/01/2005
• Issued: 09/29/2009
• Est. Priority Date: 03/01/2004
• Status: Active Grant

• Current Assignee: Leidos, Inc. (Leidos Holdings, Inc.)
• Original Assignee: Science Applications International Corporation
• Inventors: Winso, James Henry, Richardson, Rex D., Muenchau, Ernest E., Shen, Weicheng, Tuszynski, Jaroslaw W.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Akhavannik; Hadi
• Application Number:
  US11/067,925
• Time in Patent Office: 1,673 Days
• Field of Search: 382128-132, 382/224, 278/58.1, 250/358.1
• US Class Current: 382/224
• CPC Class Codes: G01V 5/22 Active interrogation, i.e. by irradiating objects or goods using external radiation sources, e.g. using gamma rays or cosmic rays
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0008 checking presence/absence

Abstract

The present invention relates to systems and methods for characterizing targets, e.g., trucks, pallets, trains, cars, and the like, using digital target imaging. Accordingly, embodiments of the present invention provide, at base: a) image segmentation and filtering to identify surrounding white space, interior walls, floor, and ceiling of the container; morphology routines to identify and classify objects found inside the container; and decision routines to report status of container as empty or non-empty with a confidence factor from 1-100.

Claims

1. A method for processing an empty-status of a target comprising:

directing radiation at a target with at least one radiation source;

detecting the directed radiation that passes through the target with at least one radiation detector;

generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, the processing including;

identifying a region of the target in the target image; and

determining an interior of the target from the identified target region, including;

identifying a first wall and a second wall of the target;

identifying a floor of the target; and

identifying a ceiling of the target, including;

determining whether a correlation in mean brightness intensity from one pixel row to an adjacent lower pixel row in the container region meets a predetermined correlation threshold when the one pixel row is shifted to right or left, beginning from a top of the container region; and

upon determining that the correlation in mean brightness intensity between one of the upper pixel rows in the container region when shifted to the left or right and an adjacent lower one of the pixels no longer meets the predetermined correlation, identifying the one upper pixel row as the target ceiling.

2. The method of claim 1, the generating the target image comprises:

detecting any pixel row in the target image that is undesirable based on at least a second predetermined criterion;

correcting the undesirable pixel row;

detecting any pixel column in the target image that is undesirable based on at least a third predetermined criterion; and

correcting the undesirable pixel column.

3. The method of claim 2, the correcting the undesirable pixel row comprises replacing the undesirable pixel row with a pixel row having brightness intensity averaged from a plurality of neighboring rows that are desirable based on the at least second predetermined criterion;

and

the correcting the undesirable pixel column comprises replacing the undesirable pixel column with pixel column having brightness intensity averaged from a plurality of neighboring rows that are desirable based on the at least third predetermined criterion.

4. The method of claim 2, the correcting the undesirable pixel row comprises removing the undesirable pixel row from the target image and splicing together the remainder pixel rows in the target image;

and

the correcting the undesirable pixel column comprises removing the undesirable pixel column from the target image and splicing together the remainder pixel columns in the target image.

5. The method of claim 1, the identifying the target region comprises:

first inspecting a mean-to-standard deviation ratio of brightness intensity of each pixel column in the target image; and

second inspecting a mean-to-standard deviation ratio of brightness intensity of each pixel row in the target image.

6. The method of claim 1, the identifying the target region comprises:

first comparing a mean value of brightness intensity of each pixel row in the target image against a predetermined mean brightness intensity row threshold; and

second comparing a mean value of brightness intensity of each pixel column in the target image against a predetermined mean brightness intensity column threshold.

7. The method of claim 1, the identifying the target region comprises:

comparing the brightness intensity of each pixel in the target image against a predetermined threshold value representing a brightness intensity of a pixel outside of the target; and

identifying the target region as a block of those pixels in the target image that have brightness intensity less than the predetermined threshold value.

8. The method of claim 1, the identifying the first and second walls of the target comprises:

identifying the first and second walls of the target based on the identifying the target region and predetermined morphology of the target.

9. The method of claim 7, the determining the target interior comprises:

identifying a first wall and a second wall of the target by;

plotting a mean brightness intensity for each pixel column in the pixel block on a curve; and

identifying the first and second walls of the target as local minima on the curve.

10. The method of claim 1, the identifying the target floor further comprises:

determining a plurality of regions in the target image that are below the identified container regions;

evaluating the plurality of regions for all possible combinations of floor thickness and floor angle that do not exceed predetermined floor thickness and angle to generate one or more floor candidates; and

selecting one of the one or more floor candidates as the target floor based on at least a fourth predetermined criterion.

11. A system for imaging a target comprising:

a radiation source configured to direct radiation at the target;

a detector configured to detect at least a portion of the radiation from the radiation source that passes through the target;

a computer readable medium on which is encoded program code, the program code comprising;

program code for generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

program code for processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, including;

program code for identifying a region of the target in the target image;

program code for determining an interior of the target from the identified target region, including;

program code for identifying a first wall and a second wall of the target;

program code for identifying a floor of the target;

program code for identifying a ceiling of the target, including;

program code for determining whether a correlation in mean brightness intensity from one pixel row to an adjacent lower pixel row in the container region meets a predetermined correlation threshold when the one pixel row is shifted to right or left, beginning from a top of the container region; and

upon determining that the correlation in mean brightness intensity between one of the upper pixel rows in the container region when shifted to the left or right and an adjacent lower one of the pixels no longer meets the predetermined correlation, program code identifying the one upper pixel row as the target ceiling.

12. The system of claim 11, the program code for generating the target image comprises:

program code for detecting any pixel row in the target image that is undesirable based on at least a second predetermined criterion;

program code for correcting the undesirable pixel row;

program code for detecting any pixel column in the target image that is undesirable based on at least a third predetermined criterion; and

program code for correcting the undesirable pixel column.

13. The system of claim 12, the program code for correcting the undesirable pixel row comprises program code for replacing the undesirable pixel row with a pixel row having brightness intensity averaged from a plurality of neighboring rows that are desirable based on the at least second predetermined criterion;

and

the program code for correcting the undesirable pixel column comprises program code for replacing the undesirable pixel column with a pixel column having brightness intensity averaged from a plurality of neighboring rows that are desirable based on the at least third predetermined criterion.

14. The system of claim 12, the program code for correcting the undesirable pixel row comprises program code for removing the undesirable pixel row from target image and splicing together the remainder pixel rows in the target image;

and

the program code for correcting the undesirable pixel column comprises program code for removing the undesirable pixel column from the target image and splicing together the remainder pixel columns in the target image.

15. The system of claim 11, the program code for identifying the target region comprises:

program code for first inspecting a mean-to-standard deviation ratio of brightness intensity of each pixel column in the target image; and

program code for second inspecting a mean-to-standard deviation ratio of brightness intensity of each pixel row in the target image.

16. The system of claim 11, the program code for identifying the target region comprises:

program code for first comparing a mean value of brightness intensity of each pixel row in the target image against a predetermined mean brightness intensity row threshold; and

program code for second comparing a mean value of brightness intensity of each pixel column in the target image against a predetermined mean brightness intensity column threshold.

17. The system of claim 11, the program code for identifying the target region comprises:

program code for comparing the brightness intensity of each pixel in the target image against a predetermined threshold value representing a brightness intensity of a pixel outside of the target; and

program code for identifying the target region as a block of those pixels in the target image that have brightness intensity less than the predetermined threshold value.

18. The system of claim 11, the program code for identifying the first and second walls of the target comprises:

program code for identifying the first and second walls of the target based on the identifying the target region and predetermined morphology of the target.

19. The system of claim 17, the program code for determining the target interior comprises:

program code identifying a first wall and second wall of the target by;

plotting a mean brightness intensity for each pixel column in the pixel block on a curve; and

identifying the first and second walls of the target as local minima on the curve.

20. The system of claim 11, the program code for identifying the target floor further comprises:

program code for determining a plurality of regions in the target image that are below the identified container regions;

program code for evaluating the plurality of regions for all possible combinations of floor thickness and floor angle that do not exceed predetermined floor thickness and angle to generate one or more floor candidates; and

program code for selecting one of the one or move floor candidates as the target floor based on at least a fourth predetermined criterion.

21. The system of claim 11, the program code for identifying the target ceiling comprises:

program code for determining vertical and diagonal edge pixels in the container region;

program code for calculating a mean edge density function for all pixel rows in the container region based on the determined vertical and diagonal edges;

program code for fitting a best ramp function to the mean edge density function; and

program code for identifying the target ceiling from the best ramp function.

22. A method for processing an empty-status of a target comprising:

directing radiation at a target with at least one radiation source;

detecting the directed radiation that passes through the target with at least one radiation detector;

generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, the processing including;

identifying a region of the target in the target image; and

determining an interior of the target from the identified target region including;

identifying a first wall and a second wall of the target;

identifying a floor of the target; and

identifying a ceiling of the target including;

performing a Fast Fourier Transformation (FFT) of the brightness intensity of each pixel row in the target region;

identifying a dominant frequency from the FFT of a predetermined number of upper rows in the target region;

identifying a disappearance of dominant frequency from one of the predetermined upper rows to another one of the predetermined upper rows; and

selecting the one of the predetermined upper rows as the target ceiling.

23. A method for processing an empty-status of a target comprising:

directing radiation at a target with at least one radiation source;

detecting the directed radiation that passes through the target with at least one radiation detector;

generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, the processing including;

identifying a region of the target in the target image; and

determining an interior of the target from the identified target region, including;

identifying a first wall and a second wall of the target;

identifying a floor of the target; and

identifying a ceiling of the target, including;

determining vertical and diagonal edge pixels in the container region;

calculating a mean edge density function for all pixel rows in the container region based on the determined vertical and diagonal edges;

fitting a best ramp function to the mean edge density function; and

identifying the target ceiling from the best ramp function.

24. A method for processing an empty-status of a target comprising:

directing radiation at a target with at least one radiation source;

detecting the directed radiation that passes through the target with at least one radiation detector;

generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, the processing including;

identifying a region of the target in the target image;

determining an interior of the target from the identified target region;

performing a first segmentation of the target interior into empty and non-empty regions;

reporting any problems with the processing of the target image;

determining whether the non-empty region exceeds a predetermined threshold percentage of the target;

upon the non-empty region exceeding the predetermined threshold percentage, classifying the target as non-empty;

upon the non-empty region not exceeding the predetermined threshold percentage additionally processing the target image to evaluate the non-empty region including;

removing any vertical patterns from the empty region of the target interior;

re-determining the target interior based at least on the removing the vertical patterns from the empty region of the target interior;

performing a second segmentation of the target interior into new empty and new non-empty regions based on the re-determining the target interior;

performing morphological operations to the new non-empty region;

identifying one or more objects in the non-empty region and assigning weighing factors to pixels representing the one or more identified objects based on the morphological operations;

declaring the target as empty or non-empty based on the first predetermined criterion; and

assigning a confidence factor based at least on the weighting factors to the pixels representing the one or more identified objects.

25. A system for imaging a target comprising:

a radiation source configured to direct radiation at the target;

a detector configured to detect at least a portion of the radiation from the radiation source that passes through the target;

a computer readable medium on which is encoded program code, the program code comprising;

program code for generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

program code for processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, including;

program code for identifying a region of the target in the target image;

program code for determining an interior of the target from the identified target region, including;

program code for identifying a first wall and a second wall of the target;

program code for identifying a floor of the target;

program code for identifying a ceiling of the target including;

program code for performing a Fast Fourier Transformation (FFT) of the brightness intensity of each pixel row in the target region;

program code for identifying a dominant frequency from the FFT of a predetermined number of upper rows in the target region;

program code for identifying a disappearance of dominant frequency from one of the predetermined upper rows to another one of the predetermined upper rows; and

program code for selecting the one of the predetermined upper rows as the target ceiling.

26. A system for imaging a target comprising:

a radiation source configured to direct radiation at the target;

a detector configured to detect at least a portion of the radiation from the radiation source that passes through the target;

a computer readable medium on which is encoded program code, the program code comprising;

program code for generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

program code for processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, including;

program code for identifying a region of the target in the target image;

program code for determining an interior of the target from the identified target region, including;

program code for identifying a first wall and a second wall of the target;

program code for identifying a floor of the target;

program code for identifying a ceiling of the target, including;

program code for determining vertical and diagonal edge pixels in the container region;

program code for calculating a mean edge density function for all pixel rows in the container region based on the determined vertical and diagonal edges;

program code for fitting a best ramp function to the mean edge density function; and

program code for identifying the target ceiling from the best ramp function.

27. A system for imaging a target comprising:

a radiation source configured to direct radiation at the target;

a detector configured to detect at least a portion of the radiation from the radiation source that passes through the target;

a computer readable medium on which is encoded program code, the program code comprising;

program code for generating an image of the target, the image including a plurality of pixels arranged in rows and columns, each of the plurality of pixels is assigned a brightness intensity based on the detecting; and

program code for processing the target image based on the brightness intensity of each of the plurality of pixels to declare the target as empty or non-empty in accordance with a first predetermined criterion, including;

program code for identifying a region of the target in the target image;

program code for determining an interior of the target from the identified target region, including;

program code for performing a first segmentation of the target interior into empty and non-empty regions; and

program code for reporting any problems with the processing of the target image, including;

program code for determining whether the non-empty region exceeds a predetermined threshold percentage of the target; and

program code for upon the non-empty region exceeding the predetermined threshold percentage, classifying the target as non-empty

upon the non-empty region not exceeding the predetermined threshold percentage, program code for additionally processing the target image to evaluate the non-empty region, including;

program code for removing any vertical patterns from the empty region of the target interior;

program code for re-determining the target interior based at least on the removing the vertical patterns from the empty region of the target interior; and

program code for performing a second segmentation of the target interior into new empty and new non-empty regions based on the re-determining the target interior

program code for performing morphological operations to the new non-empty region;

program code for identifying one or more objects in the non-empty region and assigning weighing factors to pixels representing the one or more identified objects based on the morphological operations;

program code for declaring the target as empty or non-empty based on the first predetermined criterion; and

program code for assigning a confidence factor based at least on the weighting factors to the pixels representing the one or more identified objects.