FLUOROSCOPIC INSPECTION METHOD, DEVICE AND STORAGE MEDIUM FOR AUTOMATIC CLASSIFICATION AND RECOGNITION OF CARGOES

• US 20150189239A1
• Filed: 12/23/2014
• Published: 07/02/2015
• Est. Priority Date: 12/27/2013
• Status: Active Grant

• Current Assignee: Nuctech Company Limited (Tsinghua Tongfang Co., Ltd.)
• Original Assignee: Nuctech Company Limited (Tsinghua Tongfang Co., Ltd.)
• Inventors: ZHAO, Ziran, LIU, Yaohong, GU, Jianping, LI, Qiang, ZHANG, Jian
• Granted Patent: US 10,122,973 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01V 5/22 Active interrogation, i.e. by irradiating objects or goods using external radiation sources, e.g. using gamma rays or cosmic rays
• CPC Class Codes: G06T 2207/10121 Fluoroscopy
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30112 Baggage; Luggage; Suitcase
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/05 Recognition of patterns representing particular kinds of hidden objects, e.g. weapons, explosives, drugs
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The present disclosure relates to a fluoroscopic inspection system for automatic classification and recognition of cargoes. The system includes: an image data acquiring unit, configured to perform scanning and imaging for a container by using an X-ray scanning device to acquire a scanned image; an image segmenting unit, configured to segment the scanned image into small regions each having similar gray scales and texture features; a feature extracting unit, configured to extract features of the small regions; a training unit, configured to generate a classifier according to annotated images; and a classification and recognition unit, configured to recognize the small regions by using the classifier according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes, and merge small regions to obtain large regions each representing a category.

Claims

1. A fluoroscopic inspection method for automatic classification and recognition of cargoes, comprising:

performing scanning and imaging for a container by using an X-ray scanning device to acquire a scanned image;

segmenting the scanned image into small regions each having similar gray scales and texture features;

extracting features of the small regions;

generating a classifier according to annotated images, and/or storing or updating an existing classifier; and

recognizing the small regions by using the classifier according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes, and merging small regions to obtain large regions each representing a category.

2. The method according to claim 1, wherein, the method further comprises:

estimating a quantity of each category of cargoes, and submitting a result to an inspecting terminal.

3. The method according to claim 1, wherein, the method comprises a training stage and/or a recognition stage.

4. The method according to claim 3, wherein, in the training stage, firstly, after scanned images of cargoes in containers are acquired, each of the scanned images is pre-segmented into small regions each having similar gray scales and texture features, and features of the small regions are extracted;

subsequently, each of the scanned images is annotated according to a Manifest corresponding to the scanned image, and small regions that pertain to the same category and are adjacent to or interconnected with each other are merged into a large region; and

finally, features of all the small regions in each of the large regions constitute a feature cluster, and the classifier is trained according to the feature clusters of the known categories, for subsequent recognition.

5. The method according to claim 4, wherein, in the training stage, a plurality of features are acquired by using a SIFT or MR8 operator, and the plurality of features are fused by using a feature fusion or decision fusion method to extract the features;

further, the extracted features are organized by Dictionary learning.

6. The method according to claim 4, wherein, in the training stage, a sub-category image database is acquired by manually annotating the collected scanned images.

7. The method according to claim 4, wherein, in the recognition stage, firstly, cargoes in a container are scanned, and a scanned image is pre-segmented to generate several small regions each being relatively consistent in terms of gray scale and texture;

subsequently, features of the small regions are extracted, and the small regions are recognized by using a classifier generated by means of training according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes; and

finally, a probabilistic graphic model is constructed by using the probabilities and correlations between adjacent small regions, and the small regions are merged to obtain large regions each representing a category, thereby completing cargo classification.

8. The method according to claim 7, wherein, in the recognition stage, the probabilistic graphical model is constructed by using a Hide Markov Mode, a Conditional Random Field, or a Structure Output Prediction;

or the small regions are directly merged according to similarities, and a comprehensive category is output to serve as a category of a region resulted from the merge.

9. The method according to claim 7, wherein, in the recognition stage, a table of possible maximum gray scales and minimum gray scales against different thicknesses for each category of cargoes is constructed;

and a minimum possible weight and a maximum possible weight of a cargo are obtained by reference to a gray scale of an image and the table of maximum gray scales and minimum gray scales.

10. The method according to claim 9, wherein, after the recognition stage, the method further comprises one of more of the following:

1) displaying different cargoes using different colors according to categories of the cargoes, informing an operators how many categories the cargoes cover, and which category each category of the cargoes pertains to;

2) displaying at a click position in a certain region a category of cargoes and a minimum possible weight and a maximum possible weight of the category of cargoes according to a click operation of an operator;

3) if a result is different from a manually input category, highlighting the difference with a color on the scanned image; and

4) automatically analyzing a Manifest to obtain a cargo category and corresponding weight information, and if a result is different from an automatic analysis result, highlighting the difference with a color on the image.

11. A fluoroscopic inspection device for automatic classification and recognition of cargoes, comprising:

a processor,

a memory, configured to store instructions executable by the processor,

wherein the processor is configured to;

perform scanning and imaging for a container by using an X-ray scanning device to acquire a scanned image;

segment the scanned image into small regions each having similar gray scales and texture features;

extract features of the small regions;

generate a classifier according to annotated images, and/or store or update an existing classifier; and

recognize the small regions by using the classifier according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes, and merge small regions to obtain large regions each representing a category.

12. The device according to claim 11, wherein, the device further comprises an inspecting terminal;

wherein the processor is further configured to estimate a quantity of each category of cargoes, and submit a result to the inspecting terminal.

13. The device of claim 11, wherein, the automatic classification and recognition performed by the processor of the device for the cargoes comprises a training stage and/or a recognition stage.

14. The device according to claim 13, wherein, in the training stage, firstly, after scanned images of cargoes in containers are acquired, each of the scanned images is pre-segmented into small regions each having similar gray scales and texture features, and features of the small regions are extracted;

subsequently, each of the scanned images is annotated according to a Manifest corresponding to the scanned image, and small regions that pertain to the same category and are adjacent to or interconnected with each other are merged into a large region; and

finally, features of all the small regions in each of the large regions constitute a feature cluster, and the classifier is trained according to the feature clusters of the known categories, for subsequent recognition.

15. The device according to claim 14, wherein, in the training stage, a plurality of features are acquired by using a SIFT or MR8 operator, and the plurality of features are fused by using a feature fusion or decision fusion method to extract the features;

further, the extracted features are organized by Dictionary learning.

16. The device according to claim 14, wherein, in the training stage, a sub-category image database is acquired by manually annotating the collected scanned images.

17. The device according to claim 14, wherein, in the recognition stage, firstly, cargoes in a container are scanned, and a scanned image is pre-segmented to generate several small regions each being relatively consistent in terms of gray scale and texture;

subsequently, features of the small regions are extracted, and the small regions are recognized by using a classifier generated by means of training according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes; and

finally, a probabilistic graphic model is constructed by using the probabilities and correlations between adjacent small regions, and the small regions are merged to obtain large regions each representing a category, thereby completing cargo classification.

18. The device according to claim 17, wherein, in the recognition stage, the probabilistic graphical model is constructed by using a Hide Markov Mode, a Conditional Random Field, or a Structure Output Prediction;

or the small regions are directly merged according to similarities, and a comprehensive category is output to serve as a category of a region resulted from the merge.

19. The device according to claim 17, wherein, in the recognition stage, a table of possible maximum gray scales and minimum gray scales against different thicknesses for each category of cargoes is constructed;

and a minimum possible weight and a maximum possible weight of a cargo are obtained by reference to a gray scale of an image and the table of maximum gray scales and minimum gray scales.

20. The device according to claim 19, wherein, the device comprises a user interface, and after the recognition stage, the user interface is configured to perform one or more of:

1) display different cargoes using different colors according to categories of the cargoes, inform an operators how many categories the cargoes cover, and which category each category of the cargoes pertains to;

2) display at a click position in a certain region a category of cargoes and a minimum possible weight and a maximum possible weight of the category of cargoes according to a click operation of an operator;

3) if a result is different from a manually input category, highlight the difference with a color on the scanned image; and

4) if the processor is further configured to automatically analysis a Manifest to obtain a cargo category and corresponding weight information, and if a result is different from an automatic analysis result, highlight the difference with a color on the image.

21. A non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of a device, the device is enabled to perform a method for automatic classification and recognition of cargoes, the method comprising:

performing scanning and imaging for a container by using an X-ray scanning device to acquire a scanned image;

segmenting the scanned image into small regions each having similar gray scales and texture features;

extracting features of the small regions;

generating a classifier according to annotated images, and/or storing or updating an existing classifier; and

recognizing the small regions by using the classifier according to the extracted features, to obtain a probability of each small region pertaining to a certain category of cargoes, and merging small regions to obtain large regions each representing a category.

22. The non-transitory computer readable storage medium according to claim 22, wherein, the method comprises a training stage and/or a recognition stage.