REAL-TIME THREE-DIMENSIONAL REAL ENVIRONMENT RECONSTRUCTION APPARATUS AND METHOD

• US 20130147789A1
• Filed: 07/13/2012
• Published: 06/13/2013
• Est. Priority Date: 12/08/2011
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Young-Hee
• Granted Patent: US 8,872,817 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A real-time three-dimensional (3D) real environment reconstruction apparatus and method are provided. The real-time 3D real environment reconstruction apparatus reconstructs a 3D real environment in real-time, or processes data input through RGB-D cameras in real-time using a plurality of GPUs so as to reconstruct a wide ranging 3D real environment.

Claims

1. A real-time three-dimensional (3D) real environment reconstruction apparatus, comprising:

a plurality of RGB-D cameras that acquire frame information including RGB color image information and depth image information; and

a processor that includes a plurality of GPUs for processing the frame information acquired by the plurality of RGB-D cameras and a CPU for aligning a processing result of each of the GPUs, and reconstructs a 3D real environment in real-time,

wherein the RGB-D cameras the GPUs are configured as sets including one RGB-D camera and one GPU.

2. The real-time 3D real environment reconstruction apparatus according to claim 1, wherein each of the GPUs is a worker node and the CPU is a master node.

3. The real-time 3D real environment reconstruction apparatus according to claim 1, wherein each of the GPUs comprises:

a feature extraction and matching unit that receives, from the CPU, the frame information input from the RGB-D camera in the same set, extracts feature point information from frame information of a present view point and frame information of a previous view point, and extracts matching points of feature points;

a local 3D information generation unit that generates local 3D coordinates of each of the matching points extracted from each of the view points by the feature extraction and matching unit;

a local 3D information transmission unit that transmits the local 3D coordinates generated by the local 3D information generation unit to the CPU to convert the local 3D coordinates into global 3D coordinates; and

a feature point information transmission unit that transmits the feature point information extracted by the feature extraction and matching unit to the CPU.

4. The real-time 3D real environment reconstruction apparatus according to claim 1, wherein the CPU comprises:

a frame information transmission unit that transmits the frame information input from the RGB-D camera to the GPU in each set;

a local 3D information collection unit that collects local 3D coordinates from each of the GPUs;

a feature point information collection unit that collects feature point information from each of the GPUs; and

is a global 3D information generation unit that generates global 3D coordinates by matching the local 3D coordinates using correction information of each of the RGB-D cameras and the feature point information.

5. The real-time 3D real environment reconstruction apparatus according to claim 4, wherein the CPU further comprises:

a loop closure processing unit that processes a loop closure.

6. A real-time 3D real environment reconstruction apparatus, comprising:

an RGB-D camera that acquires frame information including RGB color image information and depth image information; and

a processor that includes a plurality of GPUs for processing the frame information and a CPU for sequentially transmitting frame information input from the RGB-D camera to each of the GPUs over time and matching a processing result of each of the GPUs, and reconstructs a 3D real environment in real time.

7. The real-time 3D real environment reconstruction apparatus according to claim 6, wherein each of the GPUs is a worker node and the CPU is a master node.

8. The real-time 3D real environment reconstruction apparatus according to claim 6, wherein each of the GPUs comprises:

a feature extraction and matching unit that receives the frame information input from the RGB-D camera from the CPU, extracts feature point information from frame information of a predetermined view point, and extracts matching points of feature points;

a local 3D information generation unit that generates local 3D coordinates of each of the matching points extracted from the predetermined view point by the feature extraction and matching unit;

a local 3D information transmission unit that transmits, to the CPU, the local 3D coordinates generated by the local 3D information generation unit to convert the local 3D coordinates into global 3D coordinates; and

a feature point information transmission unit that transmits the feature point information extracted by the feature extraction and matching unit to the CPU.

9. The real-time 3D real environment reconstruction apparatus according to claim 6, wherein the CPU comprises:

a frame information transmission unit that sequentially transmits the frame information input from the RGB-D camera to each of the GPUs;

a local 3D information collection unit that collects local 3D coordinates from each of the GPUs;

a feature point information collection unit that collects feature point information from each of the GPUs; and

a global 3D information generation unit that generates global 3D coordinates by matching the local 3D coordinates using correction information of the RGB-D camera and the feature point information.

10. A real-time 3D real environment reconstruction method performed by a GPU, comprising:

receiving, from a CPU, frame information input through each RGB-D camera in a plurality of sets including one RGB-D camera and one GPU, extracting feature point information from frame information of a present view point and frame information of a previous view point, extracting matching points of feature points, and transmitting the extracted feature point information to the CPU; and

generating local 3D coordinates of each of the matching points extracted from the present view point and the previous view point, and transmitting the generated local 3D coordinates to the CPU to convert the local 3D coordinates into global 3D coordinates.

11. A real-time 3D real environment reconstruction method performed by a CPU, comprising:

transmitting frame information input from an RGB-D camera to a GPU in each of a plurality of sets including one RGB-D camera and one GPU;

collecting local 3D coordinates and feature point information from each of the GPUs in accordance with a frame information transmission result; and

generating global 3D coordinates by matching the local 3D coordinates using correction information of each of the RGB-D cameras and the feature point information.