METHOD AND DEVICE FOR RECOGNIZING DANGEROUSNESS OF OBJECT

• US 20160132716A1
• Filed: 10/29/2015
• Published: 05/12/2016
• Est. Priority Date: 11/12/2014
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: CHENG, Zhongwei, FAN, Shengyin
• Granted Patent: US 9,805,249 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Disclosed is an object dangerousness recognition method comprising steps of generating, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information; determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object, and then, generating a first dangerousness parameter according to the solid shape; determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object, and then, generating a second dangerousness parameter according to the surface feature; and generating, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object.

Claims

1. A method of recognizing object dangerousness, comprising:

a point cloud generation step of generating, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information;

a first parameter generation step of determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object corresponding to the heterogeneous point cloud, and then, generating a first dangerousness parameter according to the solid shape;

a second parameter generation step of determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object corresponding to the heterogeneous point cloud, and then, generating a second dangerousness parameter according to the surface feature; and

a comprehensive parameter generation step of generating, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object corresponding to the heterogeneous point cloud.

2. The method according to claim 1, wherein, the first parameter generation step includes:

a first step of generating, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural two-dimensional projections of the heterogeneous point cloud according to plural directions;

a second step of generating, based on property information of each of the plural two-dimensional projections, a projection-based dangerousness parameter of the corresponding two-dimensional projection, and then, generating a self-dangerousness parameter of the heterogeneous point cloud according to the projection-based dangerousness parameter of each of the plural two-dimensional projections;

a third step of comparing the property information of the plural two-dimensional projections so as to generate a comparison-based dangerousness parameter; and

a fourth step of generating, based on the self-dangerousness parameter and the comparison-based dangerousness parameter, the first dangerousness parameter.

3. The method according to claim 2, wherein, the first step includes:

calculating, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural side weights between any two points in the heterogeneous point cloud, each of the plural side weights representing a relation between the corresponding two points;

generating, based on the plural side weights, a side weight matrix of the heterogeneous point cloud;

calculating plural eigenvectors corresponding to at least a part of eigenvalues of the side weight matrix; and

generating, based on the plural eigenvectors, the plural two-dimensional projections.

4. The method according to claim 1, wherein, the second parameter generation step includes:

conducting, based on the depth information and the planar image information of each point in the heterogeneous point cloud, coordinate conversion with respect to the heterogeneous point cloud, wherein, the planar image information includes planar position information, and in the coordinate conversion, a point having a minimum depth among plural points having same planar position information in the heterogeneous point cloud is retained; and

determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud after the coordinate conversion, a texture and/or material of the object corresponding to the heterogeneous point cloud, and then, generating the second dangerousness parameter according to the texture and/or material.

5. The method according to claim 1, wherein, the second parameter generation step includes:

obtaining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, solid texture information of the object, and then, generating a texture-based dangerousness parameter according to the solid texture information;

acquiring, based on the planar image information of each point in the heterogeneous point cloud, material information of the object, and then, generating a material-based dangerousness parameter according to the material information; and

generating, based on the texture-based dangerousness parameter and the material-based dangerousness parameter, the second dangerousness parameter.

6. The method according to claim 1, wherein, the comprehensive parameter generation step includes:

generating, according to historical information related to recognition accuracies based on the first and second dangerousness parameters and/or surface information of the object, first and second weights of the first and second dangerousness parameters, respectively;

utilizing the first dangerousness parameter and the first weight so as to obtain a weighted first dangerousness parameter, and utilizing the second dangerousness parameter and the second weight so as to obtain a weighted second dangerousness parameter; and

generating, based on the weighted first and second dangerousness parameters, the comprehensive dangerousness parameter.

7. The method according to claim 1, wherein, the point cloud generation step includes:

determining whether the object exists in a foreground object region of the image; and

if it is determined that the object exists, then generating, based on the image, the heterogeneous point cloud of the object.

11. A non-transitory machine-readable medium having machine-executable instructions for execution by a processing system, wherein, the machine-executable instructions, when executed, cause the processing system to carry out the method according to claim 1.

8. A device for recognizing object dangerousness, comprising:

a point cloud generation part configured to generate, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information;

a first parameter generation part configured to determine, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object corresponding to the heterogeneous point cloud, and then, to generate a first dangerousness parameter according to the solid shape;

a second parameter generation part configured to determine, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object corresponding to the heterogeneous point cloud, and then, to generate a second dangerousness parameter according to the surface feature; and

a comprehensive parameter generation part configured to generate, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object corresponding to the heterogeneous point cloud.

9. The device according to claim 8, wherein, the first parameter generation part includes:

a first module configured to generate, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural two-dimensional projections of the heterogeneous point cloud according to plural directions;

a second module configured to generate, based on property information of each of the plural two-dimensional projections, a projection-based dangerousness parameter of the corresponding two-dimensional projection, and then, to generate a self-dangerousness parameter of the heterogeneous point cloud according to the projection-based dangerousness parameter of each of the plural two-dimensional projections;

a third module configured to compare the property information of the plural two-dimensional projections so as to generate a comparison-based dangerousness parameter; and

a fourth module configured to generate, based on the self-dangerousness parameter and the comparison-based dangerousness parameter, the first dangerousness parameter.

10. The device according to claim 8, wherein, the second parameter generation part includes:

a fifth module configured to obtain, based on the depth information and the planar image information of each point in the heterogeneous point cloud, solid texture information of the object, and then, to generate a texture-based dangerousness parameter according to the solid texture information;

a sixth module configured to acquire, based on the planar image information of each point in the heterogeneous point cloud, material information of the object, and then, to generate a material-based dangerousness parameter according to the material information; and

a seventh module configured to generate, based on the texture-based dangerousness parameter and the material-based dangerousness parameter, the second dangerousness parameter.