Method and apparatus for providing panoramic view with geometric correction

• US 7,899,270 B2
• Filed: 10/17/2006
• Issued: 03/01/2011
• Est. Priority Date: 01/09/2006
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Do Kyoon, Konouchine, Anton, Jung, Seok Yoon, Vezhnevets, Vladimir, Lee, Hee Sae, Kim, Keun Ho
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/581,401
• Publication Number:
  US 20070159527A1
• Time in Patent Office: 1,596 Days
• Field of Search: None
• US Class Current: 382/284
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and apparatus of correcting geometric information and providing a panoramic view. In the method of providing a panoramic view, a tilting vector (X′, Y′) is generated by applying matrices H(a,b) and HF to a source image vector (X,Y). The matrix H(a,b) is defined by distortion angles “b” towards a plane direction and a distortion angle “a” towards a depth direction of the source image, and the matrix HF is defined according to a camera parameter matrix K. Also, a two-dimensional vector (x,y) of a source image is converted into a three-dimensional vector (X,Y,W) on a retinal image plane having a unit radial distance from an origin of a ground plane. Also, a warped image width (WIW) is estimated from a width and a field of view (FOV) of the source image. A warping vector (x′,y′) is generated based on the converted vector (X,Y,W) and the WIW.

Claims

1. A method of providing a panoramic view, the method comprising:

receiving a source image with respect to a plurality of pixels taken with a camera, and a location vector of each of the plurality of pixels;

generating a transformation matrix for tilting the source image based on a distortion angle towards a plane direction of the source image, a distortion angle towards a depth direction of the source image, a camera parameter matrix and a location correction matrix; and

geometrically tilting the source image by obtaining a tilting vector with respect to the location vector from the transformation matrix and relocating the pixels of the source image to the tilting vector.

2. The method of claim 1, wherein the transformation matrix is determined according to a matrix that is defined based on the distortion angle towards the plane direction and the distortion angle towards the depth direction, an inverse matrix of the camera parameter matrix and the location correction matrix.

3. The method of claim 1, wherein the generating of the transformation matrix comprises:

generating a basic matrix by calculating a plurality of element values from the distortion angle towards the plane direction and the distortion angle towards the depth direction;

generating an inverse matrix of the camera parameter matrix;

multiplying the basic matrix by the inverse matrix to obtain a first multiplication result;

multiplying the camera parameter matrix by the first multiplication result to obtain a second multiplication result; and

generating a final transformation matrix by multiplying the location correction matrix by the second multiplication result to generate a final transformation matrix.

4. The method of claim 1, wherein the transformation matrix for tilting is determined by equation,

HF=VKH(a,b)Inv(K)

where HF is the transformation matrix, K is the camera parameter matrix, V is the location correction matrix, Inv(K) is an inverse matrix of K, H(a,b) is a basic transformation matrix, “

a”

is a distortion angle towards a depth direction and “

b”

is a distortion angle towards a plane direction of the source image.

5. The method of claim 4, wherein each of the matrices V, K, and H(a,b) is a three-dimensional square matrix.

6. The method of claim 4, wherein the equation is adopted by,

7. The method of claim 1, further comprising:

acquiring a warping vector based on a size and a field of view (FOV) of the source image or the tilted source image, and the camera parameter matrix; and

geometrically warping the source image or the tilted source image by relocating pixels of the source image or the tilted source image to the warping vector, to obtain the warped image.

8. The method of claim 7, further comprising:

estimating a warped image width from the size of the input image;

converting the two-dimensional location vector of the input image to be warped into a three-dimensional vector on a retinal image plane contacting a cylinder and having a unit radial distance from an origin of a ground plane by using the camera parameter matrix and the warped image width; and

generating the warping vector according to a defined value based on the three-dimensional vector and the warped image width.

9. The method of claim 7, wherein the acquiring of the warping vector comprises:

calculating a warped image width from a width and the FOV of the input image to be warped;

calculating an angle of a left-most side pixel and an angle of a right-most side pixel with respect to a certain line, from the size;

calculating an inverse matrix of the camera parameter matrix;

multiplying the inverse matrix by a vector based on the location vector of the pixel of the input image to be warped to obtain an element value;

calculating an angle of the source image pixel from the element value associated with that pixel;

determining a first coordinate value of a pair of coordinates of the warping vector from the angle of the source image pixel using the warped image width and the angle of the left-most side pixel and the angle of the right-most side pixel with respect to the certain line; and

determining a second coordinate value of the pair of coordinates of the warping vector based on the element value.

10. The method of claim 9, wherein the warped image width is calculated from locations of a central pixel of the input image and a horizontally adjacent pixel of the central pixel.

11. The method of claim 9, wherein the certain line for calculating the angle of the left-most side pixel and the angle of the right-most side pixel is a central line of the input image.

12. The method of claim 7, wherein the warping vector is determined by,

13. The method of claim 7, further comprising:

processing the image that is tilted or warped by at least one of stitching, color blending, and shear correcting; and

rendering the image that is further processed.

14. A non-transitory computer readable recording medium storing a program for implementing the method of claim 1.