System and method of detecting objects in scene point cloud

• US 9,098,773 B2
• Filed: 06/27/2013
• Issued: 08/04/2015
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: Chevron USA Incorporated (Chevron Corp.), University of Southern California
• Original Assignee: Chevron USA Incorporated (Chevron Corp.), University of Southern California
• Inventors: Huang, Jing, You, Suya, Anvar, Amir, Fisher, Christopher Lee
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/929,280
• Publication Number:
  US 20150003723A1
• Time in Patent Office: 768 Days
• Field of Search: 382/154, 382/156, 382/157, 382/159, 382/190, 382/224, 382/305, 345/419, 345/653, 345/664, 345/679
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A system and method of detecting one or more objects in a three-dimensional point cloud scene are provided. The method includes receiving a three-dimensional point cloud scene, the three-dimensional point cloud scene comprising a plurality of points; classifying at least a portion of the plurality of points in the three-dimensional point cloud into two or more categories by applying a classifying-oriented three-dimensional local descriptor and learning-based classifier; extracting from the three-dimensional point cloud scene one or more clusters of points utilizing the two or more categories by applying at least one of segmenting and clustering; and matching the extracted clusters with objects within a library by applying a matching-oriented three-dimensional local descriptor.

Claims

1. A method of detecting one or more objects in a three-dimensional point cloud scene, the method being implemented by a computer system that includes one or more processors configured to execute computer modules, the method comprising:

receiving, by one or more processors, a three-dimensional point cloud scene, the three-dimensional point cloud scene comprising a plurality of points;

classifying, by the one or more processors, at least a portion of the plurality of points in the three-dimensional point cloud into two or more categories by applying a classifying-oriented three-dimensional local descriptor and learning-based classifier;

extracting, by the one or more processors, from the three-dimensional point cloud scene one or more clusters of points utilizing the two or more categories by applying at least one of segmenting and clustering; and

matching, by the one or more processors, the extracted clusters with objects within a library by applying a matching-oriented three-dimensional local descriptor.

2. The method according to claim 1, further comprising receiving, by the one or more processors, the objects within the library and training the learning-based classifier to distinguish various objects within the three-dimensional point cloud scene.

3. The method according to claim 1, wherein the learning-based classifier comprises a support vector machine.

4. The method according to claim 1, wherein the classifying-oriented three-dimensional local descriptor comprises a Fast Point Feature Histogram that uses a histogram to encode geometrical properties of a neighborhood of each point by generalizing a mean curvature around each point.

5. The method according to claim 1, wherein the classifying-oriented three-dimensional local descriptor comprises a Fast Point Feature Histogram, the learning-based classifier is a Support Vector Machine, and the Fast Point Feature Histogram feeds positive/negative clusters to the Support Vector Machine to train the Support Vector Machine to distinguish various objects within the three-dimensional point cloud scene.

6. The method according to claim 1, wherein the library includes one or more mesh models of objects, one or more point clouds of the objects, or both.

7. The method according to claim 6, wherein if the library contains one or more mesh models of the objects, converting, by the one or more processors, the one or more mesh models of the objects into point clouds of the objects.

8. The method according to claim 1, further comprising filtering, by the one or more processors, each of the extracted clusters to obtain filtered clusters with desired characteristics and then matching, by the one or more processors, the filtered clusters with the objects within the library by applying the matching-oriented three-dimensional local descriptor.

9. The method according to claim 1, wherein the two or more categories comprise at least two of the following:

a plane category, a pipe category, an edge category and a thin pipe category.

10. The method according to claim 1, wherein applying at least one of segmenting and clustering comprises extracting and fitting connected components of the same category and clustering remaining points based on Euclidean distance.

11. The method according to claim 1, wherein segmenting is applied using a min-cut approach.

12. The method according to claim 1, wherein segmenting is applied by iteratively selecting a random unvisited seed point and expanding the seed point to unvisited neighboring points within a given Euclidean distance using a classical Flood-Fill algorithm.

13. The method according to claim 1, wherein clustering is applied by subtracting relatively large clusters from the original point cloud to remove small areas of regular shapes that may lie on a relatively larger part, and clustering remaining points that are believed to be part points.

14. The method according to claim 1, wherein clustering is applied by evaluating a linearity of each cluster by an absolute value of a correlation coefficient in a least square fitting on three-dimensional points of three projections in three orthogonal planes.

15. The method according to claim 1, wherein matching comprises computing all nearest-neighbor correspondences, or pairs of features, with a Nearest Neighbor Distance Ratio (NNDR) value, applying a filtering procedure to look for correspondences that fit a distance constraint, estimating a transformation based on all correspondences, and refining through Gram-Schmidt Orthogonalization.

16. The method according to claim 1, wherein the matching procedure comprises a three-dimensional self-similarity descriptor.

17. A system of detecting one or more objects in a three-dimensional point cloud scene, the system comprising a processor configured to:

receive a three-dimensional point cloud scene, the three-dimensional point cloud scene comprising a plurality of points;

classify at least a portion of the plurality of points in the three-dimensional point cloud into two or more categories by applying a classifying-oriented three-dimensional local descriptor and learning-based classifier;

extract from the three-dimensional point cloud scene one or more clusters of points utilizing the two or more categories by applying at least one of segmenting and clustering; and

match the extracted clusters with objects within a library by applying a matching-oriented three-dimensional local descriptor.

18. The system according to claim 17, wherein the processor is configured to filter each of the extracted clusters to obtain filtered clusters with desired characteristics and then match the filtered clusters with the objects within the library by applying the matching-oriented three-dimensional local descriptor.

19. The system according to claim 17, wherein at least one of the classifying-oriented three-dimensional local descriptor and the matching-oriented three-dimensional local descriptor comprises a three-dimensional self-similarity descriptor.

20. The system according to claim 17, wherein the two or more categories comprise at least two of the following:

a plane category, a pipe category, an edge category and a thin pipe category.