STEREOSCOPIC IMAGE ALIGNING APPARATUS, STEREOSCOPIC IMAGE ALIGNING METHOD, AND PROGRAM OF THE SAME

• US 20120147139A1
• Filed: 03/14/2011
• Published: 06/14/2012
• Est. Priority Date: 03/19/2010
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Li, Bo, Lasang, Pongsak, Ong, Chin Phek, Shen, Sheng Mei, Tagawa, Junichi
• Granted Patent: US 8,890,934 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/373 for tracking forward-backward translational head movements, i.e. longitudinal movements
• CPC Class Codes: H04N 2013/0074 Stereoscopic image analysis

Abstract

A stereoscopic image aligning apparatus (200) automatically aligns image pairs for stereoscopic viewing in a shorter amount of time than conventional apparatuses, which is applicable to image pairs captured by a single sensor camera or a variable baseline camera, without relying on camera parameters. The stereoscopic image aligning apparatus (200) includes: an image pair obtaining unit (205) obtaining an image pair including a left-eye image and a right-eye image corresponding to the left-eye image; a corresponding point detecting unit (252) detecting a corresponding point representing a set of a first point included in a first image that is one of the images of the image pair and a second point included in a second image that is the other of the images of the image pair and corresponding to the first point; a first matrix computing unit (254) computing a homography transformation matrix for transforming the first point such that a vertical parallax between the first and second points is smallest and an epipolar constraint is satisfied; a transforming unit (260) transforming the first image using the homography transformation matrix; and an output unit (210) outputting: a third image that is the transformed first image; and the second image.

Claims

1. A stereoscopic image aligning apparatus, comprising:

an image pair obtaining unit configured to obtain an image pair including a left-eye image and a right-eye image that corresponds to the left-eye image;

a corresponding point detecting unit configured to detect a corresponding point that represents a set of a first point and a second point, the first point being included in a first image that is one of the images of the image pair, and the second point being included in a second image that is the other of the images of the image pair and corresponding to the first point;

a first matrix computing unit configured to compute a homography transformation matrix for transforming the first point such that a vertical parallax between the first point and the second point is smallest and an epipolar constraint is satisfied;

a transforming unit configured to transform the first image using the homography transformation matrix; and

an output unit configured to output a third image and the second image, the third image being the transformed first image.

2. The stereoscopic image aligning apparatus according to claim 1,

wherein said image pair obtaining unit is configured to obtain, as the image pair, two images concurrently captured using (i) two image sensors and two lenses, (ii) one image sensor and two lenses, or (iii) two image sensors and one lens.

3. The stereoscopic image aligning apparatus according to claim 1,

wherein said image pair obtaining unit is configured to obtain, as the image pair, two images captured at different times from different viewpoints, using one image sensor and one lens.

4. The stereoscopic image aligning apparatus according to claim 1, further comprising:

a second matrix computing unit configured to compute a skewing transformation matrix for transforming the third image such that a coordinate system of the third image has orthogonality and the third image has a same aspect ratio as an aspect ratio of the first image; and

a matrix combining unit configured to generate a combined transformation matrix by combining the homography transformation matrix and the skewing transformation matrix,

wherein said transforming unit is configured to transform the first image using the combined transformation matrix.

5. The stereoscopic image aligning apparatus according to claim 4, further comprising

a viewing zone computing unit configured to compute a viewing zone that is a region common to the first image and the second image,

wherein said second matrix computing unit is configured to compute the skewing transformation matrix such that a coordinate system of a portion included in the viewing zone of the third image has the orthogonality and the portion included in the viewing zone of the third image has a same aspect ratio as an aspect ratio of a portion included in the viewing zone of the first image.

6. The stereoscopic image aligning apparatus according to claim 1,

wherein said corresponding point detecting unit is configured to select small regions evenly distributed in a predetermined area extending from a center of each of the first image and the second image included in the image pair and detect the corresponding point from the small regions.

7. The stereoscopic image aligning apparatus according to claim 1, further comprising

a blur region detecting unit configured to detect a blur region in the image pair,

wherein said corresponding point detecting unit is configured to detect the corresponding point from regions other than the blur region in the image pair.

8. The stereoscopic image aligning apparatus according to claim 1,

wherein said output unit is configured to obtain adjustment information that indicates at least one of an input operation from a viewing user, a size of a display device, and a distance between the display device and the viewing user, and to adjust and output at least one of a parallax between the third image and the second image, a three-dimensional (3D) pop-up effect, and a 3D deep-in effect, based on the adjustment information.

9. The stereoscopic image aligning apparatus according to claim 1,

wherein said image pair obtaining unit is configured to (A) obtain a plurality of image pair candidates each of which is a candidate for the image pair, (B) specify, for each of the image pair candidates, a common viewing zone that is a viewing zone common to the image pair candidate and an other image pair candidate and obtain quality information that is information related to at least one of an amount of light, brightness, color, and definition, in the common viewing zone, and (C) select, as the image pair, one of the image pair candidates having a highest image quality indicated by the quality information.

10. The stereoscopic image aligning apparatus according to claim 1,

wherein, when (i) an image group is encoded and stored in a memory card or (ii) the image group is encoded and transmitted from an external device through a network, said image pair obtaining unit is configured to decode the encoded image group, the image group including the left-eye image and the right-eye image.

11. The stereoscopic image aligning apparatus according to claim 1,

wherein said corresponding point detecting unit is configured to compute a rotation parameter and a shift parameter so as to match the rotation parameter and the shift parameter to an output from a Gyro sensor that outputs information corresponding to a rotation angle, the rotation parameter and the shift parameter being for use in generating the homography transformation matrix.

12. The stereoscopic image aligning apparatus according to claim 1,

wherein said stereoscopic image aligning apparatus is (i) incorporated in a two-dimensional (2D) camera to form a 3D camera, (ii) incorporated in a 3D display as a 3D processing engine to convert a group of 2D images into 3D images or 3D video, and (ii) incorporated in a player or a recorder to convert the group of 2D images into the 3D images or the 3D video.

17. A digital still camera comprising the stereoscopic image aligning apparatus according to claim 3.

18. The stereoscopic image aligning apparatus according to claim 3, further comprising

a blur region detecting unit configured to detect a blur region in the image pair,

wherein said corresponding point detecting unit is configured to detect the corresponding point from regions other than the blur region in the image pair.

13. The stereoscopic image aligning method, comprising:

obtaining an image pair including a left-eye image and a right-eye image that corresponds to the left-eye image;

detecting a corresponding point that represents a set of a first point and a second point, the first point being included in a first image that is one of the images of the image pair, and the second point being included in a second image that is the other of the images of the image pair and corresponding to the first point;

computing a homography transformation matrix for transforming the first point such that the first point and the second point have a same vertical position value and an epipolar constraint is satisfied;

transforming the first image using the homography transformation matrix; and

outputting a third image and the second image, the third image being the transformed first image.

15. A non-transitory computer-readable recording medium having a program recorded thereon for causing the computer to execute the stereoscopic image aligning method according to claim 13.

14. (canceled)

16. An integrated circuit comprising:

an image pair obtaining unit configured to obtain an image pair including a left-eye image and a right-eye image;

a corresponding point detecting unit configured to detect a corresponding point that represents a set of a first point and a second point, the first point being included in a first image that is one of the images included in the image pair, and the second point being included in a second image that is the other of the images of the image pair and corresponding to the first point;

a first matrix computing unit configured to compute a homography transformation matrix for transforming the first point such that the first point and the second point have a same vertical position value and an epipolar constraint is satisfied;

a transforming unit configured to transform the first image using the homography transformation matrix; and

an output unit configured to output a third image and the second image, the third image being the transformed first image.