Image processing apparatus and method, and program

• US 8,687,048 B2
• Filed: 12/17/2010
• Issued: 04/01/2014
• Est. Priority Date: 12/25/2009
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Morifuji, Takafumi
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): WILLIAMS, JEFFERY A
• Application Number:
  US13/201,364
• Publication Number:
  US 20120057001A1
• Time in Patent Office: 1,201 Days
• Field of Search: 348/42, 348/43, 348/50
• US Class Current: 348/50
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present invention relates to image processing apparatus and method, and a program which can display a stereoscopic image that provides more natural depth perception. A panoramic image generating section 25-1 generates a panoramic image PR for the right eye by extracting and synthesizing each strip region TRR in a plurality of captured images that are captured successively in the state in which an imaging apparatus ii is moving. A panoramic image generating section 25-2 generates a panoramic image PL for the left eye by extracting and synthesizing each strip region TRL in the plurality of captured images. From the panoramic image PR and the panoramic image PL, a disparity controlling section 26 detects the dominant disparity of these panoramic images, and shifts the entire panoramic images in accordance with the detection result to thereby adjust disparity. A display section 27 simultaneously displays the disparity-adjusted pair of panoramic images, thereby displaying a stereoscopic panoramic image with more natural depth perception. The present invention can be applied to a camera.

Claims

1. An image processing apparatus which generates a first synthetic image and a second synthetic image used for displaying a stereoscopic synthetic image and having disparity with respect to each other, on the basis of a plurality of captured images obtained by capturing with imaging means while moving the imaging means, comprising:

a first synthetic image generating means for generating the first synthetic image by synthesizing each of images in a first strip region of the captured images;

a second synthetic image generating means for generating the second synthetic image by synthesizing each of images in a second strip region of the captured images which is different from the first strip region; and

disparity controlling means for adjusting the disparity of the first synthetic image and the second synthetic image, by shifting at least one of the first synthetic image and the second synthetic image on the basis of a predetermined amount of shift,

wherein the disparity controlling means computes a correlation value indicating a degree of correlation between a region in the first synthetic image and a region in the second synthetic image, detects a dominant disparity of the first synthetic image and the second synthetic image by using the correlation value, and computes the amount of shift that cancels out the dominant disparity.

2. The image processing apparatus according to claim 1, wherein the disparity controlling means computes a difference between the first synthetic image and the second synthetic image as the correlation value, while displacing a position of the second synthetic image with respect to the first synthetic image.

3. The image processing apparatus according to claim 1, wherein the disparity controlling means computes the amount of shift on the basis of a distance between the first strip region and the second strip region in each of the captured images.

4. An image processing method for an image processing apparatus that generates a first synthetic image and a second synthetic image used for displaying a stereoscopic synthetic image and having disparity with respect to each other, on the basis of a plurality of captured images obtained by capturing with imaging means while moving the imaging means, the image processing apparatus including

a first synthetic image generating means for generating the first synthetic image by synthesizing each of images in a first strip region of the captured images,

a second synthetic image generating means for generating the second synthetic image by synthesizing each of images in a second strip region of the captured images which is different from the first strip region, and

disparity controlling means for adjusting the disparity of the first synthetic image and the second synthetic image, by shifting at least one of the first synthetic image and the second synthetic image on the basis of a predetermined amount of shift, the image processing method comprising the steps of:

the first synthetic image generating means generating the first synthetic image by synthesizing each of the images in the first strip region of the captured images;

the second synthetic image generating means generating the second synthetic image by synthesizing each of the images in the second strip region of the captured images; and

the disparity controlling means adjusting the disparity of the first synthetic image and the second synthetic image on the basis of the amount of shift,

wherein the disparity controlling means computes a correlation value indicating a degree of correlation between a region in the first synthetic image and a region in the second synthetic image, detects a dominant disparity of the first synthetic image and the second synthetic image by using the correlation value, and computes the amount of shift that cancels out the dominant disparity.

6. The image processing method of claim 4, wherein the disparity controlling means computes a difference between the first synthetic image and the second synthetic image as the correlation value, while displacing a position of the second synthetic image with respect to the first synthetic image.

7. The non-transitory, computer-readable medium of claim 6, wherein adjusting the disparity further comprises computing a difference between the first synthetic image and the second synthetic image as the correlation value, while displacing a position of the second synthetic image with respect to the first synthetic image.

5. A non-transitory, computer-readable medium comprising instructions that, when executed, cause a computer to execute processing, the computer controlling an image processing apparatus that generates a first synthetic image and a second synthetic image used for displaying a stereoscopic synthetic image and having disparity with respect to each other, on the basis of a plurality of captured images obtained by capturing with imaging means while moving the imaging means, the processing including the steps of:

generating the first synthetic image by synthesizing each of images in a first strip region of the captured images;

generating the second synthetic image by synthesizing each of images in a second strip region of the captured images which is different from the first strip region; and

adjusting the disparity of the first synthetic image and the second synthetic image, by shifting at least one of the first synthetic image and the second synthetic image on the basis of a predetermined amount of shift,

wherein adjusting the disparity further comprises computing a correlation value indicating a degree of correlation between a region in the first synthetic image and a region in the second synthetic image, detecting a dominant disparity of the first synthetic image and the second synthetic image by using the correlation value, and computing the amount of shift that cancels out the dominant disparity.