OBJECTION RECOGNITION IN A 3D SCENE

• US 20160154999A1
• Filed: 11/21/2015
• Published: 06/02/2016
• Est. Priority Date: 12/02/2014
• Status: Active Grant

• Current Assignee: Vivo Mobile Communication Co., Ltd. (Vivo Communication Technology Co. Ltd.)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Fan, Lixin, BABAHAJIANI, Pouria
• Granted Patent: US 9,846,946 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method comprising: obtaining a three-dimensional (3D) point cloud about at least one object of interest; detecting ground and/or building objects from 3D point cloud data using an unsupervised segmentation method; removing the ground and/or building objects from the 3D point cloud data; and detecting one or more vertical objects from the remaining 3D point cloud data using a supervised segmentation method.

Claims

1. A method, comprising:

obtaining a three-dimensional (3D) point cloud about at least one object of interest;

detecting at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

removing the at least one of ground and building objects from the 3D point cloud data; and

detecting one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method.

2. The method according to claim 1, further comprising:

dividing the 3D cloud point data into rectangular tiles in a horizontal plane; and

determining an estimation of a ground plane within the rectangular tiles.

3. The method according to claim 2, wherein determining the estimation of the ground plane within the rectangular tiles comprises:

dividing the rectangular tiles into a plurality of grid cells;

searching a minimal-z-value point within each grid cell;

searching points in the each grid cell having a z-value within a first predetermined threshold from the minimal-z-value of a grid cell;

collecting the points having the z-value within the first predetermined threshold from the minimal-z-value of the grid cell from each grid cell;

estimating the ground plane of a tile on the basis of the collected points; and

determining points locating within a second predetermined threshold from the estimated ground plane to comprise ground points of the tile.

4. The method according to claim 1, further comprising:

projecting 3D points to range image pixels in a horizontal plane;

defining values of the range image pixels as a function of number of 3D points projected to the range image pixels and a maximal z-value among the 3D points projected to the range image pixels;

defining a geodesic elongation value for objects detected in the range image pixels; and

distinguishing buildings from other objects on the basis of the geodesic elongation value.

5. The method according to claim 4, the method further comprising:

binarizing the values of the range image pixels;

applying morphological operations for merging neighboring points in the binarized values of the range image pixels; and

extracting contours for finding boundaries of the objects.

6. The method according to claim 1 further comprising applying a voxel based segmentation to the remaining 3D point cloud data, wherein the voxel based segmentation comprises:

performing voxilisation of the 3D point cloud data; and

merging of voxels into super-voxels and carrying out supervised classification based on discriminative features extracted from the super-voxels.

7. The method according to claim 6 further comprising:

merging the 3D points into voxels comprising a plurality of 3D points such that for a selected 3D point, all neighboring 3D points within a third predefined threshold from the selected 3D point are merged into a voxel without exceeding a maximum number of 3D points in the voxel.

8. The method according to claim 6, further comprising:

merging any number of the voxels into the super-voxels such that a criteria is fulfilled, the criteria comprising;

a minimal geometrical distance between two voxels is smaller than a fourth predefined threshold; and

an angle between normal vectors of the two voxels is smaller than a fifth predefined threshold.

9. The method according to claim 6 further comprising:

extracting features from the super-voxels for classifying the super-voxels into objects.

10. The method according to claim 9, wherein the features include one or more of:

geometrical shape;

height of a voxel above ground;

horizontal distance of the voxel to a center line of a street;

surface normal of the voxel;

voxel planarity;

density of 3D points in the voxel; and

intensity of the voxel.

11. An apparatus, comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

obtain a three-dimensional (3D) point cloud about at least one object of interest;

detect at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

remove the at least one of ground and building objects from the 3D point cloud data; and

detect one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method.

12. The apparatus according to claim 11, wherein the apparatus is further caused to:

divide the 3D cloud point data into rectangular tiles in a horizontal plane; and

determine an estimation of a ground plane within the rectangular tiles.

13. The apparatus according to claim 12, wherein to determine the estimation of the ground plane within the rectangular tiles, the apparatus is further caused to:

divide the rectangular tiles into a plurality of grid cells;

search a minimal-z-value point within each grid cell;

search points in the each grid cell having a z-value within a first predetermined threshold from the minimal-z-value of a grid cell;

collecting the points having the z-value within the first predetermined threshold from the minimal-z-value of the grid cell from the each grid cell;

estimating the ground plane of a tile on the basis of the collected points; and

determining points locating within a second predetermined threshold from the estimated ground plane to comprise ground points of the tile.

14. The apparatus according claim 11, wherein the apparatus is further caused to:

project 3D points to range image pixels in a horizontal plane;

define values of the range image pixels as a function of number of 3D points projected to the range image pixels and a maximal z-value among the 3D points projected to the range image pixels;

define a geodesic elongation value for objects detected in the range image pixels; and

distinguish buildings from other objects on the basis of the geodesic elongation value.

15. The apparatus according to claim 14, wherein the apparatus is further caused to:

binarize the values of the range image pixels;

apply morphological operations for merging neighboring points in the binarized values of the range image pixels; and

extract contours for finding boundaries of the objects.

16. The apparatus according to claims 11, wherein the apparatus is further caused to apply a voxel based segmentation to the remaining 3D point cloud data, and wherein to apply the voxel based segmentation, the apparatus is further caused to:

perform voxilisation of the 3D point cloud data;

merge voxels into super-voxels; and

carry out supervised classification based on discriminative features extracted from the super-voxels.

17. The apparatus according to claim 16, wherein the apparatus is further caused to merge the 3D points into voxels comprising a plurality of 3D points such that for a selected 3D point, all neighboring 3D points within a third predefined threshold from the selected 3D point are merged into a voxel without exceeding a maximum number of 3D points in the voxel.

18. The apparatus according to claim 16, wherein the apparatus is further caused to merging any number of the voxels into the super-voxels such that following criteria is fulfilled:

a minimal geometrical distance between two voxels is smaller than a fourth predefined threshold; and

an angle between normal vectors of the two voxels is smaller than a fifth predefined threshold.

19. The apparatus according to claim 16, wherein the apparatus is the apparatus is further caused to extract features from the super-voxels for classifying the super-voxels into objects.

20. The apparatus according to claim 19, wherein the features include one or more of:

geometrical shape;

height of a voxel above ground;

horizontal distance of the voxel to a center line of a street;

surface normal of the voxel;

voxel planarity;

density of 3D points in the voxel; and

intensity of the voxel.

21. A computer readable storage medium stored with code thereon for use by an apparatus, which when executed by a processor, causes the apparatus to perform:

obtaining a three-dimensional (3D) point cloud about at least one object of interest;

detecting at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

removing the at least one of ground and building objects from the 3D point cloud data; and

detecting one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method.