SYSTEM AND METHOD OF IMAGE PROCESSING

• US 20120114224A1
• Filed: 05/11/2011
• Published: 05/10/2012
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Xu, Jin, Lee, Chao Hsu, Yoshigahara, Takayuki, Zhang, Jian
• Granted Patent: US 8,849,013 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems

Abstract

A method of image processing comprising receiving a plurality of interpolated images, interpolated from two adjacent camera positions having different image planes, applying a transformation to each interpolated image to a respective one of a plurality intermediate image planes, wherein each intermediate image plane is oriented intermediate to the image planes of the two adjacent camera positions depending on a viewing angle of that interpolated image relative to the adjacent camera positions. Also an integrated circuit or processor, an apparatus for capturing images and an apparatus for displaying images.

Claims

1. A method of image processing comprising

receiving a plurality of interpolated images, interpolated from two adjacent camera positions having different image planes;

and

applying a transformation to each interpolated image to a respective one of a plurality intermediate image planes, wherein each intermediate image plane is oriented intermediate to the image planes of the two adjacent camera positions depending on a viewing angle of that interpolated image relative to the adjacent camera positions.

2. A method according to claim 1, wherein the transformation to each interpolated image comprises applying a transformation matrix, the transformation matrix obtained by multiplying a scaling matrix with a keystone compensation matrix.

3. A method according to claim 2, wherein the keystone compensation matrix is obtained by interpolating between an identity matrix and a rectification matrix, the rectification matrix associated with one of the adjacent camera positions.

4. A method according to claim 3, wherein the rectification matrix is selected based on a sequence number of the interpolated image.

5. A method according to claim 2, wherein the scaling matrix comprises

a translation parameter indicative of an image origin of the interpolated image;

and

a scaling factor indicative of a viewing distance of the interpolated image.

6. A method according to claim 5, wherein the translation parameter is obtained by interpolating between unity and a center translation parameter, said center translation parameter obtained from an interpolated image centered between the plurality of images.

7. A method according to claim 5, wherein the scaling factor is obtained by interpolating between unity and a center scaling factor, said center scaling factor obtained from an interpolated image centered between the plurality of images.

8. A method according to claim 7, wherein the center translation parameter is further obtained using the center scaling factor.

9. A method according to claim 7, wherein the center scaling factor is obtained using an angle indicative of a field of view of the centered interpolated image.

10. A method according to claim 6, wherein the center translation parameter is associated with a width of the interpolated image.

11. A method according to claim 6, wherein the center translation parameter is associated with a height of the interpolated image.

12. A method according to claim 3, wherein the receiving the plurality of images comprises rectifying the plurality of images by transforming the image plane of the plurality of images onto a common image plane.

13. A method according to claim 12, wherein the transforming the image plane of the plurality of images comprises applying the rectification matrix to an image of the one of the adjacent camera positions.

14. A method according to claim 1, wherein the viewing angle of each interpolated image is determined about a camera subject.

15. A method according to claim 14, wherein the camera subject is visible in each interpolated image.

16. A method according to claim 15, wherein the camera subject visible in each interpolated image has different sizes.

17. A method according to claim 14, wherein the camera subject is visible in the plurality of images captured from the plurality of camera positions.

21. An integrated circuit or processor configured to perform the method of image processing according to claim 1.

22. An apparatus for capturing images configured to perform the method of image processing according to any of claim 1.

23. An apparatus for displaying images configured to display a processed image obtained using the method of image processing according to claim 1.

18. A method of image processing comprising

receiving a plurality of interpolated images, interpolated from images captured from a plurality of spaced camera positions;

and

applying a transformation to each interpolated image to a designated image plane, the designated image plane substantially aligned with the plurality of spaced camera positions.

19. A method according to claim 18, wherein the plurality of spaced camera positions are located on a substantially straight line and one axis of the designated image plane is parallel to the straight line.

20. A method according to claim 18 or 19, wherein the transformation is obtained by multiplying an inverse of a first camera specific rectification matrix with a second camera specific rectification matrix, the first and second camera specific rectification matrices associated with a camera position of the plurality of spaced camera positions.

24. An integrated circuit or processor configured to perform the method of image processing according to claim 18.

25. An apparatus for capturing images configured to perform the method of image processing according to any of claim 18.

26. An apparatus for displaying images configured to display a processed image obtained using the method of image processing according to claim 18.