Method and apparatus for matching color image and depth image

• US 8,717,414 B2
• Filed: 09/30/2008
• Issued: 05/06/2014
• Est. Priority Date: 02/13/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Byong-min, Lim, Hwa-sup, Sim, Jae-Young
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Coleman, Stephen
• Application Number:
  US12/285,246
• Publication Number:
  US 20090201384A1
• Time in Patent Office: 2,044 Days
• Field of Search: 348/222.1, 348/211.11, 348/262, 348/239, 348/47, 348/153, 348/159, 348/42, 348/60, 348/48, 396/325, 382/154, 345/419, 345/653, 345/664
• US Class Current: 348/42
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and apparatus for matching images are provided. The image matching method includes: obtaining a depth image and an intensity image of an object using a depth camera installed at a first position and a color image of the object using a color camera installed at a second position other than the first position; transforming the obtained depth image and intensity image into a depth image and an intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and matching the transformed depth image and intensity image and the obtained color image. In this way, a depth image and intensity image obtained separately from a 2-dimensional (2D) color image can be accurately matched to the 2D color image, thereby allowing a reliable 3D image to be obtained.

Claims

1. An image matching method comprising:

obtaining a depth image and an intensity image of an object using a real depth camera installed at a first position with a first viewpoint and obtaining a color image of the object using a real color camera installed at a second position with a second viewpoint other than the first viewpoint;

transforming each of the obtained depth image and the intensity image into a depth image and an intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position with the second viewpoint; and

matching the transformed depth image and intensity image with the obtained color image,

wherein the transforming of the obtained depth image and intensity image comprises;

calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position based on a distance between the first viewpoint and the second viewpoint, corresponding to each pixel forming the obtained depth image and intensity image, by using a depth value according to the obtained depth image.

2. The method of claim 1, wherein in the transforming of each of the obtained depth image and intensity image into a depth image and intensity image,

the obtained depth image and intensity image are transformed into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position, by using each of the pixels of the obtained depth image.

3. The method of claim 1, wherein the transforming of the obtained depth image and intensity image further comprises:

by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values, and thus transforming the obtained depth image and intensity image into the depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position.

4. The method of claim 1, wherein the matching of the transformed depth image and intensity image and the obtained color image comprises:

adjusting the transformed depth image and intensity image so that the resolutions of the transformed depth image, and intensity image and the obtained color image can match;

detecting at least one or more first edges in the adjusted intensity image, and detecting at least one or more second edges in the obtained color image; and

performing the matching, by making the adjusted intensity image and the obtained color image overlap so that the detected first edges and second edges can overlap, and making the adjusted depth image and the obtained color image overlap as much as the intensity image and the color image overlap.

5. A non-transitory computer readable recording medium having embodied thereon a computer program for executing the method of claim 1.

6. The image matching method of claim 1, wherein the transforming of the obtained depth image comprises calculating new coordinates of each of the pixels forming the obtained depth image, such that the new coordinates form a new depth image with the second viewpoint.

7. An image matching method comprising:

obtaining a depth image and an intensity image of an object using a depth camera installed at a first position and a color image of the object using a color camera installed at a second position other than the first position;

transforming the obtained depth image and intensity image into a depth image and an intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and

matching the transformed depth image and intensity image and the obtained color image,

wherein the transforming of the obtained depth image and intensity image comprises;

calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position, corresponding to each pixel forming the obtained depth image and intensity image, by using the depth value according to the obtained depth image; and

by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values, and thus transforming the obtained depth image and intensity image into the depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position,

wherein in the generating of the depth image and intensity image,

the depth value corresponding to each integer coordinate is calculated by using coordinates of a predetermined number of pixels from among the calculated pixels, and a depth image is generated according to the calculated depth values, and the intensity value corresponding to each integer coordinate is calculated by using coordinates of a predetermined number of pixels from among the calculated pixels, and an intensity image is generated according to the calculated intensity values.

8. An image matching apparatus comprising:

a real depth camera installed at a first position with a first viewpoint so as to obtain a depth image and an intensity image of an object;

a real color camera installed at a second position with a second viewpoint other than the first viewpoint so as to obtain a color image of the object;

an image transform unit transforming each of the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position with the second viewpoint; and

an image matching unit matching the transformed depth image and intensity image and the obtained color image,

wherein the image transform unit comprises;

a coordinate calculation unit calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position based on a distance between the first viewpoint and the second viewpoint, corresponding to each pixel forming the obtained depth image and intensity image, by using a depth value according to the obtained depth image.

9. The apparatus of claim 8, wherein the image transform unit transforms the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position, by using each pixel of the obtained depth image.

10. The apparatus of claim 8, wherein the image transform unit further comprises:

a depth image/intensity image generation unit, by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values.

11. The apparatus of claim 8, wherein the image matching unit comprises:

a resolution conversion unit adjusting the transformed depth image and intensity image so that the resolutions of the transformed depth image, and intensity image and the obtained color image can match;

an edge detection unit detecting at least one or more first edges in the adjusted intensity image, and detecting at least one or more second edges in the obtained color image; and

a matching unit performing the matching, by making the adjusted intensity image and the obtained color image overlap so that the detected first edges and second edges can overlap, and making the adjusted depth image and the obtained color image overlap as much as the intensity image and the color image overlap.

12. The apparatus of claim 8, wherein the image transform unit transforms the obtained depth image by calculating new coordinates of each of the pixels forming the obtained depth image, such that the new coordinates form a new depth image with the second viewpoint.

13. An image matching apparatus comprising:

a depth camera installed at a first position so as to obtain a depth image and an intensity image of an object;

a color camera installed at a second position other than the first position so as to obtain a color image of the object;

an image transform unit transforming the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position; and

an image matching unit matching the transformed depth image and intensity image and the obtained color image,

wherein the image transform unit transforms the obtained depth image and intensity image into a depth image and intensity image, respectively, that could be obtained if the object were photographed by a camera at the second position, by using a depth value according to the obtained depth image,

wherein the image transform unit comprises;

a coordinate calculation unit calculating the coordinates of each pixel of the image that could be obtained if the object were photographed by a camera at the second position, corresponding to each pixel forming the obtained depth image and intensity image, by using the depth value according to the obtained depth image; and

a depth image/intensity image generation unit, by using the calculated coordinates of each pixel, calculating the depth value and intensity value corresponding to each integer coordinate, and generating a depth image and intensity image according to the calculated depth values and intensity values,

wherein the depth image/intensity image generation unit calculates the depth value corresponding to each integer coordinate by using coordinates of a predetermined number of pixels from among the calculated pixels, and generates a depth image according to the calculated depth values, and calculates the intensity value corresponding to each integer coordinate by using coordinates of a predetermined number of pixels from among the calculated pixels, and generates an intensity image according to the calculated intensity values.