Image converter

• US 8,363,089 B2
• Filed: 08/20/2009
• Issued: 01/29/2013
• Est. Priority Date: 09/03/2008
• Status: Expired due to Fees

• Current Assignee: Dai Nippon Printing Company Limited
• Original Assignee: Dai Nippon Printing Company Limited
• Inventors: Inagaki, Koji
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US12/544,240
• Publication Number:
  US 20100053325A1
• Time in Patent Office: 1,258 Days
• Field of Search: 348143-159, 348 36- 39
• US Class Current: 348/39
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

Any given part is cut out from a distorted circular image photographed by use of a fisheye lens and converted into a planar regular image with less distortion. A virtual sphere H having a radius R on a distorted circular image S on an XY plane is defined, thereby allowing a user to designate a cut-out center point P, a magnification m, and a planar inclination angle φ. A visual line vector n passing through an intersecting point Q immediately above the point P is determined to define an UV orthogonal coordinate system having an orientation depending on the angle φ on a plane orthogonal to a visual line vector n at a point G in which a distance between two points OG is given as m·R. The UV orthogonal coordinate system is curved along the side face C of a “virtual cylindrical column in which the point G forms one point on the side face to have a straight line V′ parallel to the V axis and also passing through the point O as a central axis,” thereby defining the UV curved coordinate system. Correspondence relationship equations between a point Ci (ui, vi) on the UV curved coordinate system and a point Si (xi, yi) on the XY coordinate system are used to obtain an image in the vicinity of a point P on the UV curved coordinate system, and the image is expanded on a plane T to obtain a planar regular image.

Claims

1. An image converter for performing processes of cutting out a part of a distorted circular image photographed by use of a fisheye lens and converting said part into a planar regular image, the image converter comprising:

a distorted circular image memory for storing a distorted circular image constituted with an aggregate of many pixels arranged at a position indicated by coordinates (x, y) on a two-dimensional XY orthogonal coordinate system and having a radius R taking an origin O of the two-dimensional XY orthogonal coordinate system as a center;

a planar regular image memory for storing a planar regular image constituted with an aggregate of many pixels arranged at a position indicated by coordinates (u, v) on a two-dimensional UV orthogonal coordinate system;

a parameter input unit in which in a three-dimensional XYZ orthogonal coordinate system including the two-dimensional XY orthogonal coordinate system, a visual line vector n facing any given direction, with the origin O given as a starting point, is input as a parameter indicating a cut-out position of the planar regular image, a predetermined planar inclination angle φ

is input as a parameter indicating a cut-out orientation of the planar regular image, and a predetermined magnification m is input as a parameter indicating a cut-out dimension of the planar regular image;

a corresponding coordinate calculating unit calculating corresponding coordinates (x, y) which correspond to any given coordinates (u, v) by using predetermined correspondence relationship equations showing a correspondence relationship between coordinates (u, v) on a two-dimensional UV cylindrical coordinate system and coordinates (x, y) on the two-dimensional XY orthogonal coordinate system, wherein said two-dimensional UV cylindrical coordinate system is defined by curving a two-dimensional UV orthogonal coordinate system which is arranged on a plane passing through a point G given as an origin and orthogonal to the visual line vector n to have an orientation according to the planar inclination angle φ

, said point G being away from the origin O by “

a product m≅

→

R of the magnification m and the radius R”

on the visual line vector n, along a side face of a “

virtual cylindrical column in which the point G gives one point on the side face thereof to have a central axis parallel to a V axis of the two-dimensional UV orthogonal coordinate system,”

; and

a planar regular image forming unit giving coordinates (u, v) of a target pixel constituting the planar regular image to the corresponding coordinate calculating unit to obtain corresponding coordinates (x, y), reading out a pixel value of a pixel arranged in the vicinity of the obtained corresponding coordinates (x, y) inside the distorted circular image memory, determining a pixel value of the target pixel on the basis of a read pixel value, thereby forming the planar regular image by determining pixel values of individual pixels, and writing the pixel values into the planar regular image memory;

wherein

when, with respect to a virtual sphere having the radius R taking the origin O as a center, a corresponding point Qi is taken on the sphere which corresponds to a point Si indicated by coordinates (xi, yi) on the two-dimensional XY orthogonal coordinate system depending on a projection method of a fisheye lens used, and coordinates (ui, vi) are taken on the two-dimensional UV cylindrical coordinate system at an intersecting point Ci between a straight line connecting the origin O with the corresponding point Qi on the sphere and a cylindrical column side-face coordinate plane of the two-dimensional UV cylindrical coordinate system, the corresponding coordinate calculating unit uses correspondence relationship equations by which the coordinates (xi, yi) are determined as corresponding coordinates which correspond to the coordinates (ui, vi);

when a distorted circular image stored in the distorted circular image memory is an orthogonally projected image photographed by use of a fisheye lens based on an orthogonal projection method, the corresponding coordinate calculating unit uses correspondence relationship equations of orthogonally projected images in which, with respect to a point Si indicated by coordinates (xi, yi), a point indicated by coordinates (xi, yi, zi) given as an intersecting point between a straight line passing through the point Si and parallel to a Z axis and the virtual sphere is given as a corresponding point Qi on the sphere, and

when a distorted circular image stored in the distorted circular image memory is a non-orthogonally projected image photographed by use of a fisheye lens based on a non-orthogonal projection method, the corresponding coordinate calculating unit uses correspondence relationship equations of non-orthogonally projected images obtained by using coordinate conversion equations between coordinates on the orthogonally projected image and coordinates on the non-orthogonally projected image so as to correct the correspondence relationship equations of orthogonally projected images;

the corresponding coordinate calculating unit defines the two-dimensional UV cylindrical coordinate system by using the virtual cylindrical column in which the point G gives one point on the side face thereof to have a central axis parallel to the V axis of the two-dimensional UV orthogonal coordinate system and also passing through the origin O of the three-dimensional XYZ orthogonal coordinate system; and

the corresponding coordinate calculating unit uses the following equations as correspondence relationship equations of orthogonally projected images indicating a correspondence relationship between coordinates (u, v) and coordinates (x, y),

x=G(u′

A+vB+w′

C)

y=G(u′

D+vE+w′

F),

under the following definitions in which an angle formed between an orthogonal projection on the XY plane of a visual line vector n and a Y axis is given as an azimuthal angle α and

an angle formed between the visual line vector n and a positive direction of the Z axis is given as a zenithal angle β

,

A=cos(φ

)cos(α

)−

sin(φ

)sin(α

)cos(β

)

B=−

sin(φ

)cos(α

)−

cos(φ

)sin(α

)cos(β

)

C=sin(β

)sin(α

)

D=cos(φ

)sin(α

)+sin(φ

)cos(α

)cos(β

)

E=sin(φ

)sin(α

)+cos(φ

)cos(α

)cos(β

)

F=−

sin(β

)cos(α

)

G=R/√

(w²+v²)

w=mR

u′

=w′

sin(u/w)

w′

=w′

cos(u/w).

2. The image converter according to claim 1, wherein the corresponding coordinate calculating unit including:

a rotational coefficient operation unit in which, when the visual line vector n and the planar inclination angle φ

are given from the parameter input unit, on the basis of the visual line vector n, the azimuthal angle α and

the zenithal angle β

, are determined, and rotational coefficients A, B, C, D, E, F are calculated on the basis of the following operational equations,

A=cos(φ

)cos(α

)−

sin(φ

)sin(α

)cos(β

)

B=−

sin(φ

)cos(α

)−

cos(φ

)sin(α

)cos(β

)

D=cos(φ

)sin(α

)+sin(φ

)cos(α

)cos(β

)

E=sin(φ

)sin(α

)+cos(φ

)cos(α

)cos(β

)

F=−

sin(β

)cos(α

)

F=−

sin(β

)cos(α

)

G=R/√

(w²+v²)

a common coefficient operation unit in which, when the magnification m is given from the parameter input unit and a coordinate v is given from the planar regular image forming unit, the radius R of the distorted circular image is used to calculate a common coefficient G on the basis of the following operational equations,

w=mR

G=R/√

(w²+v²);

a curved coordinate correcting unit in which, when the magnification m is given from the parameter input unit and a coordinate u is given from the planar regular image forming unit, the radius R of the distorted circular image is used to calculate u′ and

w′

on the basis of the following operational equations,

w=mR

u′

=w′

sin(u/w)

w′

=w′

cos(u/w); and

an xy coordinate value calculating unit in which the coordinate v given from the planar regular image forming unit, the rotational coefficients A, B, C, D, E, F calculated by the rotational coefficient operation unit, the common coefficient G calculated by the common coefficient operation unit, the u′ and

w′

calculated by the curved coordinate correcting unit, and the radius R of the distorted circular image are used to calculate x and y on the basis of the following operational equations,

x=G(u′

A+vB+w′

C)

y=G(u′

D+vE+w′

F).

3. The image converter according to claim 2, wherein

the parameter input unit has a function of inputting the radius R as a variable parameter on the basis of instructions from a user or a distorted circular image stored in a distorted circular image memory, and

the common coefficient operation unit and the curved coordinate correcting unit use the radius R input by the parameter input unit to perform an operation.

4. The image converter according to claim 1, wherein

when a distorted circular image stored in the distorted circular image memory is an equidistantly projected image photographed by use of a fisheye lens based on the equidistance projection method, the corresponding coordinate calculating unit uses the following coordinate conversion equations for converting coordinates (xa, ya) on the orthogonally projected image into coordinates (xb, yb) on the equidistantly projected image,

xb=xa(2R/π

r)sin<sup>−

1</sup>(r/R)

yb=ya(2R/π

r)sin<sup>−

1</sup>(r/R)

wherein the condition is r=√

(xa²+ya²), thereby correcting the correspondence relationship equations of orthogonally projected images.

5. The image converter according to claim 1, wherein

the parameter input unit inputs coordinates (x0, y0) of a cut-out center point P on the two-dimensional XY orthogonal coordinate system as a parameter for defining the visual line vector n and defines a virtual sphere having the radius R at the center of an origin O in a three-dimensional XYZ orthogonal coordinate system,

when a distorted circular image stored in the distorted circular image memory is an orthogonally projected image photographed by use of a fisheye lens based on the orthogonal projection method, the parameter input unit defines an intersecting point Q between a straight line passing through the cut-out center point P and parallel to the Z axis and the virtual sphere, thereby giving a vector which starts from the origin O to pass through the intersecting point Q as the visual line vector n, and

when a distorted circular image stored in the distorted circular image memory is a non-orthogonally projected image photographed by use of a fisheye lens based on the non-orthogonal projection method, the parameter input unit uses coordinate conversion equations between coordinates on the orthogonally projected image and coordinates on the non-orthogonally projected image to correct coordinates of the cut-out center point P and defines an intersecting point Q between a straight line passing through a point after correction and parallel to the Z axis and the virtual sphere, thereby giving a vector which starts from the origin O to pass through the intersecting point Q as the visual line vector n.

6. The image converter according to claim 1, wherein

the corresponding coordinate calculating unit arranges the two-dimensional UV orthogonal coordinate system in such an orientation that an angle formed between a rotational reference axis J given as an axis passing through an origin G, parallel to the XY plane and also orthogonal to the visual line vector n and the U axis is equal to the planar inclination angle φ

.

7. The image converter according to claim 1, wherein

the planar regular image stored in the planar regular image memory is rectangular in contour, the two-dimensional UV orthogonal coordinate system having the U axis in a direction parallel to the longer side of the rectangle and the V axis in a direction parallel to the shorter side is curved along the side face of the virtual cylindrical column having a central axis parallel to the V axis, thereby defining the two-dimensional UV cylindrical coordinate system.

8. The image converter according to claim 1, wherein

the planar regular image forming unit performs interpolation operation for pixel values of plural reference pixels on a distorted circular image arranged in the vicinity of a position indicated by corresponding coordinates (x, y) upon determination of a pixel value of a target pixel arranged at a position indicated by the coordinates (u, v).

9. A non-transitory computer readable recording medium containing a program for allowing a computer to function as the image converter according to claim 1.

10. A semiconductor integrated circuit into which electronic circuits functioning as a corresponding coordinate calculating unit and a planar regular image forming unit which are constituents of the image converter according to claim 1 are incorporated.

11. A fisheye lens monitoring system including:

the image converter according to claim 1;

a digital camera equipped with a fisheye lens; and

a monitoring device for displaying a planar regular image on a screen;

wherein a distorted circular image photographed by use of the digital camera is stored in the distorted circular image memory, and a planar regular image obtained in the planar regular image memory is displayed by the monitoring device.