Method and apparatus for detecting vehicle contour based on point cloud data

• US 10,229,330 B2
• Filed: 08/24/2016
• Issued: 03/12/2019
• Est. Priority Date: 01/27/2016
• Status: Active Grant

• Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Original Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Inventors: Li, Bo, Zhang, Tianlei, Xia, Tian, Tao, Ji, Lin, Yuanqing
• Primary Examiner(s): Kholdebarin, Iman K
• Application Number:
  US15/246,394
• Publication Number:
  US 20170213093A1
• Time in Patent Office: 930 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

The present application discloses a method and apparatus for detecting a vehicle contour based on point cloud data. The method includes: acquiring to-be-trained point cloud data; generating label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud, the labeling used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour; training a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and acquiring to-be-detected point cloud data, and obtaining a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model. The implementation may achieve an accurate detection of the vehicle contour.

Claims

1. A method for detecting a vehicle contour based on point cloud data, comprising:

acquiring to-be-trained point cloud data;

generating label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud, the labeling used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour;

training a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and

acquiring to-be-detected point cloud data, and obtaining a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model,

wherein the training of a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model comprises,

initializing parameters of the fully convolutional neural network;

adjusting the parameters of the fully convolutional neural network based on a loss function, wherein the loss function is a deviation between the label data corresponding to a current output and the label data corresponding to the to-be-trained point cloud data, and the current output is an output of the fully convolutional neural network corresponding to current to-be-trained point cloud data and current parameters of the fully convolutional neural network;

outputting parameters corresponding to the loss function as parameters of the vehicle detection model if the loss function is at a minimum value; and

otherwise, returning to execute the step of adjusting the parameters of the fully convolutional neural network based on a loss function.

2. The method according to claim 1, wherein the acquiring of to-be-trained point cloud data comprises:

mapping to-be-trained three-dimensional data, collected by a point cloud data collecting device, to generate a to-be-trained depth map, the values of respective pixel points in the to-be-trained depth map being respective points in the to-be-trained point cloud data corresponding to the to-be-trained three-dimensional data.

3. The method according to claim 2, wherein

the to-be-trained three-dimensional data D (x, y, z) is a coordinate in a coordinate system taking the point cloud data collecting device as an origin, an upward vertical direction as a z axis, and any two axes perpendicular to each other in a horizontal plane as an x axis and a y axis;

the coordinate of each pixel point in the to-be-trained depth map is (r, c), and the value of each of the pixel points is (d, z);

wherein

r=└

θ

/Δ

θ

┘

c=└

ϕ

/Δ

ϕ

┘

d=√

{square root over (x²+y²)}

θ

=a tan 2(y,x)

ϕ

=arcsin(z/√

{square root over (x²+y²+z²)});

θ

is an angle between a line connecting the to-be-trained three-dimensional data D to the origin and a first plane, Δ

θ

is a horizontal resolution of the point cloud data collecting device, and the first plane is a plane where the y axis and the z axis locate; and

φ

is an angle between a line connecting the to-be-trained three-dimensional data D to the origin and a second plane, Δ

ϕ

is a vertical resolution of the point cloud data collecting device, and the second plane is a plane where the x axis and the y axis locate.

4. The method according to claim 3, wherein the generating of label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud data comprises:

transforming a point in the to-be-trained point cloud data to generate two-dimensional label data with 26 channels if the point is on the vehicle contour, wherein the value of a first channel in the 26 channels is 1, the value of a second channel in the 26 channels is 0, and the 3^rd-26^th channels in the 26 channels are respective rotation vectors of coordinates of vertexes of a three-dimensional cube where a vehicle is located with respect to a three-dimensional coordinate of the point on the vehicle contour in the to-be-trained point cloud data; and

transforming a point in the to-be-trained point cloud data to generate two-dimensional label data with 2 channels if the point is not on the vehicle contour, wherein the value of a first channel in the 2 channels is 0, and the value of a second channel in the 2 channels is 1.

5. The method according to claim 3, wherein

the rotation vector (x′

_p, y′

, z′

_p) of a coordinate of a vertexe of a three-dimensional cube where a vehicle is located with respect to a three-dimensional coordinate of the point on the vehicle contour in the to-be-trained point cloud data is;

6. The method according to claim 5, wherein the acquiring of to-be-detected point cloud data, and obtaining of a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model comprises:

mapping to-be-detected three-dimensional data, which is collected by the point cloud data collecting device, to generate a to-be-detected depth map, the values of pixel points in the to-be-detected depth map corresponding to respective to-be-detected points in the to-be-detected point cloud data corresponding to the to-be-detected three-dimensional data;

acquiring a predicted result of each of the to-be-detected points in the to-be-detected point cloud data based on the vehicle detection model, wherein the predicted result comprises a probability that the to-be-detected point is on the vehicle contour; and

determining that a to-be-detected point is on the vehicle contour if the probability that the to-be-detected point is on the vehicle contour is greater than a preset threshold.

7. The method according to claim 6, wherein

the predicted result further comprises location information of the to-be-detected point;

the acquiring of to-be-detected point cloud data, and obtaining of a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model further comprises;

acquiring first to-be-detected points whose probabilities are each greater than the preset threshold in the to-be-detected point cloud data, to form a set of the first to-be-detected points; and

generating spatial vehicle location information corresponding to the to-be-detected point cloud data based on the location information of the to-be-detected points in the set of the first to-be-detected points.

8. An apparatus for detecting a vehicle contour based on point cloud data, comprising:

at least one processor; and

a memory storing instructions, which when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising;

acquiring to-be-trained point cloud data;

generating label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud data, the labeling used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour;

training a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and

acquiring to-be-detected point cloud data, and obtaining a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model,

wherein the training of a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model comprises,

initializing parameters of the fully convolutional neural network;

adjusting the parameters of the fully convolutional neural network based on a loss function, wherein the loss function is a deviation between the label data corresponding to a current output and the label data corresponding to the to-be-trained point cloud data, and the current output is an output of the fully convolutional neural network corresponding to current to-be-trained point cloud data and current parameters of the fully convolutional neural network;

outputting parameters corresponding to the loss function as parameters of the vehicle detection model if the loss function is at a minimum value; and

otherwise, returning to execute the step of adjusting the parameters of the fully convolutional neural network based on a loss function.

9. The apparatus according to claim 8, wherein the acquiring of to-be-trained point cloud data comprises:

mapping to-be-trained three-dimensional data, collected by a point cloud data collecting device, to generate a to-be-trained depth map, the values of respective pixel points in the to-be-trained depth map being respective points in the to-be-trained point cloud data corresponding to the to-be-trained three-dimensional data.

10. The apparatus according to claim 9, wherein

the to-be-trained three-dimensional data D (x, y, z) is a coordinate in a coordinate system taking the point cloud data collecting device as an origin, an upward vertical direction as a z axis, and any two axes perpendicular to each other in a horizontal plane as an x axis and a y axis;

the coordinate of each pixel point in the to-be-trained depth map is (r, c), and the value of each of the pixel points is (d, z);

wherein,

r=└

θ

/Δ

θ

┘

c=└

ϕ

/Δ

ϕ

┘

d=√

{square root over (x²+y²)}

θ

=a tan 2(y,x)

ϕ

=arcsin(z/√

{square root over (x²+y²+z²)});

θ

is an angle between a line connecting the to-be-trained three-dimensional data D to the origin and a first plane, Δ

θ

is a horizontal resolution of the point cloud data collecting device, and the first plane is a plane where the y axis and the z axis locate; and

φ

is an angle between a line connecting the to-be-trained three-dimensional data D to the origin and a second plane, Δ

ϕ

is a vertical resolution of the point cloud data collecting device, and the second plane is a plane where the x axis and the y axis locate.

11. The apparatus according to claim 10, wherein the generating of label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud data comprises:

transforming a point in the to-be-trained point cloud data to generate two-dimensional label data with 26 channels if the point is on the vehicle contour, wherein the value of a first channel in the 26 channels is 1, the value of a second channel in the 26 channels is 0, and the 3^rd-26^th channels in the 26 channels are respective rotation vectors of coordinates of vertexes of a three-dimensional cube where a vehicle is located with respect to a three-dimensional coordinate of the point on the vehicle contour in the to-be-trained point cloud data; and

transforming a point in the to-be-trained point cloud data to generate two-dimensional label data with 2 channels if the point is not on the vehicle contour, wherein the value of a first channel in the 2 channels is 0, and the value of a second channel in the 2 channels is 1.

12. The apparatus according to claim 10, wherein

the rotation vector (x′

_p, y′

_p, z′

_p) of a coordinate of a vertexe of a three-dimensional cube where a vehicle is located with respect to a three-dimensional coordinate of the point on the vehicle contour in the to-be-trained point cloud data is;

13. The apparatus according to claim 12, wherein the acquiring of to-be-detected point cloud data, and obtaining of a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model comprises:

mapping to-be-detected three-dimensional data, which is collected by the point cloud data collecting device, to generate a to-be-detected depth map, the values of pixel points in the to-be-detected depth map corresponding to respective to-be-detected points in the to-be-detected point cloud data corresponding to the to-be-detected three-dimensional data;

acquiring a predicted result of each of the to-be-detected points in the to-be-detected point cloud data based on the vehicle detection model, wherein the predicted result comprises a probability that the to-be-detected point is on the vehicle contour; and

determining that a to-be-detected point is on the vehicle contour if the probability that the to-be-detected point is on the vehicle contour is greater than a preset threshold.

14. The apparatus according to claim 13, wherein

the predicted result further comprises location information of the to-be-detected point;

the acquiring of to-be-detected point cloud data, and obtaining of a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model further comprises;

acquiring first to-be-detected points whose probabilities are each greater than the preset threshold in the to-be-detected point cloud data, to form a set of the first to-be-detected points; and

generating spatial vehicle location information corresponding to the to-be-detected point cloud data based on the location information of the to-be-detected points in the set of the first to-be-detected points.

15. A non-transitory storage medium storing one or more programs, the one or more programs when executed by an apparatus, causing the apparatus to perform a method for detecting a vehicle contour based on point cloud data, comprising:

acquiring to-be-trained point cloud data;

generating label data corresponding to points in the to-be-trained point cloud data in response to labeling on the points in the to-be-trained point cloud, the labeling used to indicate whether each of the points in the to-be-trained point cloud data is on a vehicle contour;

training a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model; and

acquiring to-be-detected point cloud data, and obtaining a detection result corresponding to each to-be-detected point in the to-be-detected point cloud data based on the vehicle detection model,

wherein the training of a fully convolutional neural network model based on the points in the to-be-trained point cloud data and the label data corresponding to the points in the to-be-trained point cloud data, to obtain a vehicle detection model comprises;

initializing parameters of the fully convolutional neural network;

adjusting the parameters of the fully convolutional neural network based on a loss function, wherein the loss function is a deviation between the label data corresponding to a current output and the label data corresponding to the to-be-trained point cloud data, and the current output is an output of the fully convolutional neural network corresponding to current to-be-trained point cloud data and current parameters of the fully convolutional neural network;

outputting parameters corresponding to the loss function as parameters of the vehicle detection model if the loss function is at a minimum value; and

otherwise, returning to execute the step of adjusting the parameters of the fully convolutional neural network based on a loss function.