Method and apparatus for matching color image and depth image

• US 20090201384A1
• Filed: 09/30/2008
• Published: 08/13/2009
• Est. Priority Date: 02/13/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Sim, Jae-Young, Kang, Byong-min, Lim, Hwa-sup
• Granted Patent: US 8,717,414 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and apparatus for matching images are provided. The image matching method includes: obtaining a depth image and an intensity image of an object using a depth camera installed at a first position and a color image of the object using a color camera installed at a second position other than the first poison;

transforming the obtained depth image and intensity image into a depth image and an intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and matching the transformed depth image and intensity image and the obtained color image. In this way, a depth image and intensity image obtained separately from a 2-dimensional (2D) color image can be accurately matched to the 2D color image, thereby allowing a reliable 3D image to be obtained.

Claims

1. An image matching method comprising:

obtaining a depth image and an intensity image of an object using a depth camera installed at a first position and a color image of the object using a color camera installed at a second position other than the first position;

transforming the obtained depth image and intensity image into a depth image and an intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and

matching the transformed depth image and intensity image and the obtained color image.

2. The method of claim 1, wherein in the transforming of the obtained depth image and intensity image into a depth image and intensity image,

the obtained depth image and intensity image are transformed into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position, by using a depth value according to the obtained depth image.

3. The method of claim 1, wherein the transforming of the obtained depth image and intensity image comprises:

calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position, corresponding to each pixel forming the obtained depth image and intensity image, by using the depth value according to the obtained depth image; and

by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values, and thus transforming the obtained depth image and intensity image into the depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position.

4. The method of claim 3, wherein in the generating of the depth image and intensity image,

the depth value corresponding to each integer coordinate is calculated by using coordinates of a predetermined number of pixels from among the calculated pixels, and a depth image is generated according to the calculated depth values, and the intensity value corresponding to each integer coordinate is calculated by using coordinates of a predetermined number of pixels from among the calculated pixels, and an intensity image is generated according to the calculated intensity values.

5. The method of claim 1, wherein the matching of the transformed depth image and intensity image and the obtained color image comprises:

adjusting the transformed depth image and intensity image so that the resolutions of the transformed depth image, and intensity image and the obtained color image can match;

detecting at least one or more first edges in the adjusted intensity image, and detecting at least one or more second edges in the obtained color image; and

performing the matching, by making the adjusted intensity image and the obtained color image overlap so that the detected first edges and second edges can overlap, and making the adjusted depth image and the obtained color image overlap as much as the intensity image and the color image overlap.

11. A computer readable recording medium having embodied thereon a computer program for executing the method of claim 1.

6. An image matching apparatus comprising:

a depth camera installed at a first position so as to obtain a depth image and an intensity image of an object;

a color camera installed at a second position other than the first position so as to obtain a color image of the object;

an image transform unit transforming the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and

an image matching unit matching the transformed depth image and intensity image and the obtained color image.

7. The apparatus of claim 6, wherein the image transform unit transforms the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position, by using a depth value according to the obtained depth image.

8. The apparatus of claim 6, wherein the image transform unit comprises:

a coordinate calculation unit calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position, corresponding to each pixel forming the obtained depth image and intensity image, by using the depth value according to the obtained depth image; and

a depth image/intensity image generation unit, by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values,

9. The apparatus of claim 8, wherein the depth image/intensity image generation unit calculates the depth value corresponding to each integer coordinate by using coordinates of a predetermined number of pixels from among the calculated pixels, and generates a depth image according to the calculated depth values, and calculates the intensity value corresponding to each integer coordinate by using coordinates of a predetermined number of pixels from among the calculated pixels, and generates an intensity image according to the calculated intensity values.

10. The apparatus of claim 6, wherein the image matching unit comprises:

a resolution conversion unit adjusting the transformed depth image and intensity image so that the resolutions of the transformed depth image, and intensity image and the obtained color image can match;

an edge detection unit detecting at least one or more first edges in the adjusted intensity image, and detecting at least one or more second edges in the obtained color image; and

a matching unit performing the matching, by making the adjusted intensity image and the obtained color image overlap so that the detected first edges and second edges can overlap, and making the adjusted depth image and the obtained color image overlap as much as the intensity image and the color image overlap.