Object detection methods, display methods and apparatuses

• US 10,013,750 B2
• Filed: 12/23/2013
• Issued: 07/03/2018
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Nuctech Company Limited (Tsinghua Tongfang Co., Ltd.), Tsinghua University
• Original Assignee: Nuctech Company Limited (Tsinghua Tongfang Co., Ltd.), Tsinghua University
• Inventors: Zhang, Li, Chen, Zhiqiang, Zhao, Ziran, Li, Qiang, Gu, Jianping, Sun, Yunda
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US14/138,399
• Publication Number:
  US 20150332448A1
• Time in Patent Office: 1,653 Days
• CPC Class Codes: G01V 5/226 using tomography
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30112 Baggage; Luggage; Suitcase
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Disclosed are object detection method, display methods and apparatuses. The method includes obtaining slice data of inspected luggage in the CT system; generating 3D volume data of objects in the luggage from the slice data; for each object, determining a semantic description including at least a quantifier description of the object based on the 3D volume data; and upon reception of a user selection of an object, presenting the semantic description of the selected object while displaying a 3D image of the object. The above solutions can create a 3D model for objects in the inspected luggage in a relatively accurate manner, and thus provide better basis for subsequent shape feature extraction and security inspection, and reduce omission factor.

Claims

1. A method for inspecting luggage in a CT system, comprising:

obtaining slice data of the inspected luggage in the CT system;

generating 3D volume data of at least one object in the luggage from the slice data;

calculating first, second and third depth projection images of the object in three directions based on the 3D volume data, wherein projection direction of the third depth projection image is orthogonal to those of the first and second depth projection images;

calculating a metric of symmetry, and a duty ratio and aspect ratio for each of the first, second, and third depth projection images, and a metric of similarity between each two of the first, second, and third depth projection images;

generating a shape feature parameter of the object at least comprising on the metrics of symmetry, the metrics of similarity, the duty ratios and aspect ratios of the first to third depth projection images;

classifying the shape feature parameter with a classifier based on shape feature to obtain a quantifier description expressing the shape of the object; and

outputting a semantic description including at least the quantifier description of the object;

wherein said generating 3D volume data of at least one object in the luggage from the slice data comprises;

interpolating the slice data to generate 3D volume data of the luggage;

performing unsupervised segmentation on the 3D volume data of the luggage to obtain a plurality of segmental regions;

performing isosurface extraction on the plurality of segmental regions to obtain corresponding isosurfaces; and

performing 3D surface segmentation on the isosurfaces to form a 3D model for the objects in the luggage; and

wherein the unsupervised segmentation is performed by using a Statistical Region Merging SRM which is extended into 3D processing by joining an atomic number and density of the 3D volume data into a vector so as to represent a 3D voxel as two vectors and replacing a grey difference in a conventional SRM with a module of a difference vector between the two vectors, replacing a 2D gradient in the conventional SRM with a 3D gradient, and replacing an area of a pixel in a 2D region in the conventional SRM with a volume of a voxel.

2. The method of claim 1, further comprising:

calculating an angle between the projection directions of the first and second depth projection images, wherein the shape feature parameter further includes the angle.

3. The method of claim 1, further comprising:

calculating a volume value of the object based on the 3D volume data, wherein the shape feature parameter further includes the volume value.

4. The method of claim 1, wherein the projection directions of the first and second depth projection images are substantially orthogonal to each other, and approximate directions of maximal and minimal projection areas of the object, respectively.

5. The method of claim 1, wherein the semantic description further comprises at least one of weight, density, atomic number and position of the object.

6. The method of claim 1, wherein the quantifier description comprises at least one of bag, piece, block, bottle, can, bar, box, pack, and item.

7. An apparatus for inspecting luggage in a CT system, comprising a processor and a storage device on which instructions are stored, the processor being configured to perform the instructions to:

obtain slice data of the inspected luggage in the CT system;

obtain slice data of the inspected luggage in the CT system;

generate 3D volume data of at least one object in the luggage from the slice data;

calculate first, second and third depth projection images of the object in three directions based on the 3D volume data, wherein projection direction of the third depth projection image is orthogonal to those of the first and second depth projection images;

calculate a metric of symmetry, and a duty ratio and aspect ratio for each of the first, second, and third depth projection images, and a metric of similarity between each two of the first, second, and third depth projection images;

generate a shape feature parameter of the object at least comprising on the metrics of symmetry, the metrics of similarity, the duty ratios and aspect ratios of the first to third depth projection images;

classify the shape feature parameter with a classifier based on shape feature to obtain a quantifier description expressing the shape of the object; and

output a semantic description including at least the quantifier description of the object;

wherein the apparatus is further configured to;

interpolate the slice data to generate 3D volume data of the luggage;

perform unsupervised segmentation on the 3D volume data of the luggage to obtain a plurality of segmental regions;

perform isosurface extraction on the plurality of segmental regions to obtain corresponding isosurfaces; and

perform 3D surface segmentation on the isosurfaces to form a 3D model for the objects in the luggage; and

wherein the unsupervised segmentation is performed by using a Statistical Region Merging SRM which is extended into 3D processing by joining an atomic number and density of the 3D volume data into a vector so as to represent a 3D voxel as two vectors and replacing a grey difference in a conventional SRM with a module of a difference vector between the two vectors, replacing a 2D gradient in the conventional SRM with a 3D gradient, and replacing an area of a pixel in a 2D region in the conventional SRM with a volume of a voxel.

8. A method for displaying objects in a CT system, comprising:

obtaining slice data of inspected luggage in the CT system;

generating 3D volume data of objects in the luggage from the slice data;

calculating a metric of symmetry, and a duty ratio and aspect ratio for each of the first, second, and third depth projection images, and a metric of similarity between each two of the first, second, and third depth projection images;

generating a shape feature parameter of the object at least comprising the metrics of symmetry, the metrics of similarity, the duty ratios and aspect ratios of the first to third depth projection images;

classify the shape feature parameter with a classifier based on shape feature to obtain a quantifier description expressing the shape of the object; and

outputting a semantic description including at least the quantifier description of the object; and

upon reception of a user selection of an object, presenting the semantic description of the selected object while displaying a 3D image of the object;

wherein said generating 3D volume data of at least one object in the luggage from the slice data comprises;

interpolating the slice data to generate 3D volume data of the luggage;

performing unsupervised segmentation on the 3D volume data of the luggage to obtain a plurality of segmental regions;

performing isosurface extraction on the plurality of segmental regions to obtain corresponding isosurfaces; and

performing 3D surface segmentation on the isosurfaces to form a 3D model for the objects in the luggage; and

wherein the unsupervised segmentation is performed by using a Statistical Region Merging SRM which is extended into 3D processing by joining an atomic number and density of the 3D volume data into a vector so as to represent a 3D voxel as two vectors and replacing a grey difference in a conventional SRM with a module of a difference vector between the two vectors, replacing a 2D gradient in the conventional SRM with a 3D gradient, and replacing an area of a pixel in a 2D region in the conventional SRM with a volume of a voxel.

9. The method of claim 8, wherein the semantic description further comprises at least one of weight, density, atomic number and position of the object.

10. The method of claim 8, further comprising:

calculating an angle between the projection directions of the first and second depth projection images, wherein the shape feature parameter further includes the angle.

11. The method of claim 8, further comprising:

calculating a volume value of the object based on the 3D volume data, wherein the shape feature parameter further includes the volume value.

12. The method of claim 8, wherein the projection directions of the first and second depth projection images are substantially orthogonal to each other, and approximate directions of maximal and minimal projection areas of the object, respectively.

13. The method of claim 8, wherein the selected object is highlighted while the other objects are blocked.

14. An apparatus for displaying objects in a CT system, comprising a processor and a storage device on which instructions are stored, the processor being configured to perform the instructions to:

obtain slice data of inspected luggage in the CT system;

generate 3D volume data of objects in the luggage from the slice data;

calculate a metric of symmetry, and a duty ratio and aspect ratio for each of the first, second, and third depth projection images, and a metric of similarity between each two of the first, second, and third depth projection images;

generate a shape feature parameter of the object at least comprising the metrics of symmetry, the metrics of similarity, the duty ratios and aspect ratios of the first to third depth projection images;

classify the shape feature parameter with a classifier based on shape feature to obtain a quantifier description expressing the shape of the object; and

output a semantic description including at least the quantifier description of the object; and

upon reception of a user selection of an object, present the semantic description of the selected object while displaying a 3D image of the object;

wherein the apparatus is further configured to;

interpolate the slice data to generate 3D volume data of the luggage;

perform unsupervised segmentation on the 3D volume data of the luggage to obtain a plurality of segmental regions;

perform isosurface extraction on the plurality of segmental regions to obtain corresponding isosurfaces; and

perform 3D surface segmentation on the isosurfaces to form a 3D model for the objects in the luggage; and

wherein the unsupervised segmentation is performed by using a Statistical Region Merging SRM which is extended into 3D processing by joining an atomic number and density of the 3D volume data into a vector so as to represent a 3D voxel as two vectors and replacing a grey difference in a conventional SRM with a module of a difference vector between the two vectors, replacing a 2D gradient in the conventional SRM with a 3D gradient, and replacing an area of a pixel in a 2D region in the conventional SRM with a volume of a voxel.