Method and device for recognizing dangerousness of object

• US 9,805,249 B2
• Filed: 10/29/2015
• Issued: 10/31/2017
• Est. Priority Date: 11/12/2014
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Cheng, Zhongwei, Fan, Shengyin
• Primary Examiner(s): Tsai, Tsung-Yin
• Application Number:
  US14/926,042
• Publication Number:
  US 20160132716A1
• Time in Patent Office: 733 Days
• Field of Search: None
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Disclosed is an object dangerousness recognition method comprising steps of generating, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information; determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object, and then, generating a first dangerousness parameter according to the solid shape; determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object, and then, generating a second dangerousness parameter according to the surface feature; and generating, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object.

Claims

1. A method of recognizing object dangerousness, comprising:

a point cloud generation step of generating, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information;

a first parameter generation step of determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object corresponding to the heterogeneous point cloud, and then, generating a first dangerousness parameter according to the solid shape;

a second parameter generation step of determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object corresponding to the heterogeneous point cloud, and then, generating a second dangerousness parameter according to the surface feature; and

a comprehensive parameter generation step of generating, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object corresponding to the heterogeneous point cloud, wherein, the comprehensive parameter generation step includes;

generating, according to historical information related to recognition accuracies based on the first and second dangerousness parameters and/or surface information of the object, first and second weights of the first and second dangerousness parameters, respectively;

utilizing the first dangerousness parameter and the first weight so as to obtain a weighted first dangerousness parameter, and utilizing the second dangerousness parameter and the second weight so as to obtain a weighted second dangerousness parameter; and

generating, based on the weighted first and second dangerousness parameters, the comprehensive dangerousness parameter.

2. The method according to claim 1, wherein, the first parameter generation step includes:

a first step of generating, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural two-dimensional projections of the heterogeneous point cloud according to plural directions;

a second step of generating, based on property information of each of the plural two-dimensional projections, a projection-based dangerousness parameter of the corresponding two-dimensional projection, and then, generating a self-dangerousness parameter of the heterogeneous point cloud according to the projection-based dangerousness parameter of each of the plural two-dimensional projections;

a third step of comparing the property information of the plural two-dimensional projections so as to generate a comparison-based dangerousness parameter; and

a fourth step of generating, based on the self-dangerousness parameter and the comparison-based dangerousness parameter, the first dangerousness parameter.

3. The method according to claim 2, wherein, the first step includes:

calculating, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural side weights between any two points in the heterogeneous point cloud, each of the plural side weights representing a relation between the corresponding two points;

generating, based on the plural side weights, a side weight matrix of the heterogeneous point cloud;

calculating plural eigenvectors corresponding to at least a part of eigenvalues of the side weight matrix; and

generating, based on the plural eigenvectors, the plural two-dimensional projections.

4. The method according to claim 1, wherein, the second parameter generation step includes:

conducting, based on the depth information and the planar image information of each point in the heterogeneous point cloud, coordinate conversion with respect to the heterogeneous point cloud, wherein, the planar image information includes planar position information, and in the coordinate conversion, a point having a minimum depth among plural points having same planar position information in the heterogeneous point cloud is retained; and

determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud after the coordinate conversion, a texture and/or material of the object corresponding to the heterogeneous point cloud, and then, generating the second dangerousness parameter according to the texture and/or material.

5. The method according to claim 1, wherein, the second parameter generation step includes:

obtaining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, solid texture information of the object, and then, generating a texture-based dangerousness parameter according to the solid texture information;

acquiring, based on the planar image information of each point in the heterogeneous point cloud, material information of the object, and then, generating a material-based dangerousness parameter according to the material information; and

generating, based on the texture-based dangerousness parameter and the material-based dangerousness parameter, the second dangerousness parameter.

6. The method according to claim 1, wherein, the point cloud generation step includes:

determining whether the object exists in a foreground object region of the image; and

if it is determined that the object exists, then generating, based on the image, the heterogeneous point cloud of the object.

10. A non-transitory machine-readable medium having machine-executable instructions for execution by a processing system, wherein, the machine-executable instructions, when executed, cause the processing system to carry out the method according to claim 1.

7. A device for recognizing object dangerousness, comprising:

memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to perform operations including,

generating, based on an image captured by a stereo camera, a heterogeneous point cloud of an object in the image, each point in the heterogeneous point cloud having depth information and planar image information;

determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a solid shape of the object corresponding to the heterogeneous point cloud, and then, to generate a first dangerousness parameter according to the solid shape;

determining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, a surface feature of the object corresponding to the heterogeneous point cloud, and then, to generate a second dangerousness parameter according to the surface feature; and

generating, based on the first and second dangerousness parameters, a comprehensive dangerousness parameter of the object corresponding to the heterogeneous point cloud, wherein, generating the comprehensive dangerousness parameter includes;

generating, according to historical information related to recognition accuracies based on the first and second dangerousness parameters and/or surface information of the object, first and second weights of the first and second dangerousness parameters, respectively;

utilizing the first dangerousness parameter and the first weight so as to obtain a weighted first dangerousness parameter, and utilizing the second dangerousness parameter and the second weight so as to obtain a weighted second dangerousness parameter; and

generating, based on the weighted first and second dangerousness parameters, the comprehensive dangerousness parameter.

8. The device according to claim 7, wherein, the

one or more processors are further configured to perform operations including,

generating, based on the depth information and the planar image information of each point in the heterogeneous point cloud, plural two-dimensional projections of the heterogeneous point cloud according to plural directions;

generating, based on property information of each of the plural two-dimensional projections, a projection-based dangerousness parameter of the corresponding two-dimensional projection, and then, to generate a self-dangerousness parameter of the heterogeneous point cloud according to the projection-based dangerousness parameter of each of the plural two-dimensional projections;

comparing the property information of the plural two-dimensional projections so as to generate a comparison-based dangerousness parameter; and

generating, based on the self-dangerousness parameter and the comparison-based dangerousness parameter, the first dangerousness parameter.

9. The device according to claim 8, wherein, the one or more processors are further configured to perform operations including,

obtaining, based on the depth information and the planar image information of each point in the heterogeneous point cloud, solid texture information of the object, and then, to generate a texture-based dangerousness parameter according to the solid texture information;

acquiring, based on the planar image information of each point in the heterogeneous point cloud, material information of the object, and then, to generate a material-based dangerousness parameter according to the material information; and

generating, based on the texture-based dangerousness parameter and the material-based dangerousness parameter, the second dangerousness parameter.