Building system and building method for panorama point cloud

• US 10,694,103 B2
• Filed: 04/24/2018
• Issued: 06/23/2020
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chiang, Chih-Tsung, Chen, Yi-Cheng, Huang, Shih-Ching, Liu, Yi-Tong
• Primary Examiner(s): Hodges, Susan E.
• Application Number:
  US15/960,947
• Publication Number:
  US 20190327412A1
• Time in Patent Office: 791 Days
• Field of Search: 348 38
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A building method and a building system for panorama point cloud data are provided. The building method for panorama point cloud data includes the following steps. At least one reference geometric object located at an overlap region of two adjacent three-dimensional capturing devices is captured by the two adjacent three-dimensional capturing devices to obtain two reference point cloud sets. A reference feature plane is obtained from each of the reference point cloud sets. A coordinate transformation matrix is obtained according to the reference feature planes. A plurality of real-time point cloud sets are obtained by the three-dimensional capturing devices. The coordinate transformation of the real-time point cloud sets are performed according to the coordinate transformation matrix, and the real-time point cloud sets are combined to obtain the panorama point cloud data.

Claims

1. A building method for panorama point cloud data, comprising:

capturing at least one reference geometric object located at an overlap region of two adjacent three-dimensional capturing devices by the two adjacent three-dimensional capturing devices to obtain two reference point cloud sets;

obtaining a reference feature plane from each of the reference point cloud sets;

obtaining a coordinate transformation matrix according to the reference feature planes, wherein the coordinate transformation matrix is obtained by an iterative closest points algorithm having a K-D tree structure;

obtaining a plurality of real-time point cloud sets by the three-dimensional capturing devices; and

performing coordinate transformation of the real-time point cloud sets according to the coordinate transformation matrix, and combining the real-time point cloud sets to obtain the panorama point cloud data.

2. The building method for panorama point cloud data according to claim 1, wherein the step of obtaining the reference feature plane from each of the reference point cloud sets comprises:

filtering out noises from each of the reference point cloud sets;

generating foreground objects by splitting each of the reference point cloud sets; and

extracting a maximum plane from each of the foreground objects and using the maximum plane as the reference feature plane.

3. The building method for panorama point cloud data according to claim 2, wherein the step of generating the foreground object from each of the reference point cloud sets comprises:

dividing a plurality of image points into a plurality of groups according to the degree of feature similarity by a region growing algorithm; and

obtaining the foreground object from the groups.

4. The building method for panorama point cloud data according to claim 2, wherein the step of extracting the maximum plane from each of the foreground objects and using the maximum plane as the reference feature plane comprises:

transforming a plurality of image points of each of the foreground objects to a Hough space; and

searching a maximum value in the Hough space by a sliding window algorithm to obtain the maximum plane.

5. The building method for panorama point cloud data according to claim 1, wherein the reference geometric object is a cube, a flat object, or a sphere.

6. The building method for panorama point cloud data according to claim 1, wherein the quantity of the at least one reference geometric object is larger than 2.

7. The building method for panorama point cloud data according to claim 1, wherein an overlap angle of the three-dimensional capturing devices is larger than 10°

.

8. The building method for panorama point cloud data according to claim 1, wherein the shooting distance of each of the three-dimensional capturing devices is between 0.5˜

5 meters.

9. The building method for panorama point cloud data according to claim 1, wherein the three-dimensional capturing devices are time of flight (ToF) sensors or infrared sensors.

10. A building system for panorama point cloud data, comprising:

two three-dimensional capturing devices disposed adjacent to each other and configured to capture at least one reference geometric object located at an overlap region of the three-dimensional capturing devices to obtain two reference point cloud sets;

a feature processing unit configured to obtain a reference feature plane from each of the reference point cloud sets; and

a coordinate processing unit configured to obtain a coordinate transformation matrix according to the reference feature planes, wherein the three-dimensional capturing devices further perform capturing to obtain a plurality of real-time point cloud sets, and the coordinate processing unit performs coordinate transformation of the real-time point cloud sets according to the coordinate transformation matrix and combines the real-time point cloud sets to obtain the panorama point cloud data, and the coordinate transformation matrix is obtained by an iterative closest points algorithm having a K-D tree structure.

11. The building system for panorama point cloud data according to claim 10, wherein the feature processing unit comprises:

a noise filter configured to filter out noises from each of the reference point cloud sets;

a splitter configured to generate foreground objects by splitting each of the reference point cloud sets; and

an extractor configured to extract a maximum plane from each of the foreground objects and using the maximum plane as the reference feature plane.

12. The building system for panorama point cloud data according to claim 11, wherein the splitter divides a plurality of image points into a plurality of groups according to the degree of feature similarity by a region growing algorithm and further obtains the foreground objects from the groups.

13. The building system for panorama point cloud data according to claim 11, wherein the extractor transforms a plurality of image points of each of the foreground objects to a Hough space, and searches a maximum value in the Hough space by a sliding window algorithm to obtain the maximum plane.

14. The building system for panorama point cloud data according to claim 10, wherein the reference geometric object is a cube, a flat object, or a sphere.

15. The building system for panorama point cloud data according to claim 10, wherein the quantity of the at least one reference geometric object is larger than 2.

16. The building system for panorama point cloud data according to claim 10, wherein an overlap angle of the three-dimensional capturing devices is larger than 10°

.

17. The building system for panorama point cloud data according to claim 10, wherein the shooting distance of each of the three-dimensional capturing devices is between 0.5˜

5 meters.

18. The building system for panorama point cloud data according to claim 10, wherein the three-dimensional capturing devices are time of flight (ToF) sensors or infrared sensors.

19. A non-transitory computer readable storage device storing a plurality of instructions, when executed by at least one processing unit, causing said at least one processing unit to execute a building method for panorama point cloud data, wherein the building method for panorama point cloud data comprises:

capturing at least one reference geometric object located at an overlap region of two adjacent three-dimensional capturing devices by the two adjacent three-dimensional capturing devices to obtain two reference point cloud sets;

obtaining a reference feature plane from each of the reference point cloud sets;

obtaining a coordinate transformation matrix according to the reference feature planes, wherein the coordinate transformation matrix is obtained by an iterative closest points algorithm having a K-D tree structure;

obtaining a plurality of real-time point cloud sets by the three-dimensional capturing devices; and

performing coordinate transformation of the real-time point cloud sets according to the coordinate transformation matrix, and combining the real-time point cloud sets to obtain the panorama point cloud data.