IMAGE CONVERTER AND IMAGE CONVERTING METHOD

• US 20090167886A1
• Filed: 12/10/2008
• Published: 07/02/2009
• Est. Priority Date: 12/26/2007
• Status: Active Grant

• Current Assignee: Dai Nippon Printing Company Limited
• Original Assignee: Dai Nippon Printing Company Limited
• Inventors: TONOMURA, Motonobu
• Granted Patent: US 8,363,981 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces

Abstract

On conversion of a distorted circular image photographed by use of a fisheye lens to a planar regular image, parameter setting is performed by a user'"'"'s intuitive manipulation to relieve the operation load. A dome-shaped virtual sphere H is defined on a distorted circular image S on the XY plane, and a user is allowed to designate a cut-out center point P (x₀, y₀) and an auxiliary point Q (x₁, y₁) on the distorted circular image S. An intersecting point G (x₀, y₀, z₀) immediately above the point P is determined, and a UV plane is defined on a tangent plane S2 with respect to the virtual sphere H. An angle φ formed between the U axis and the X axis is determined on the basis of an angle θ formed between a reference straight line J passing through two points P, Q and the X axis. Transformation equations based on the orthogonal projection method including the coordinate values x₀, y₀, z₀ and the angle φ as parameters are utilized to allow the coordinates (u, v) to correspond to coordinates (x, y), a distance between the two points P, Q is given as a conversion magnification m, thereby a part in the vicinity of the point P of the distorted circular image S is converted to a plane regular image on the UV plane.

Claims

1. An image converter for performing processes of cutting out a part of a distorted circular image photographed by use of a fisheye lens and converting it into a planar regular image, the image converter comprising:

a distorted circular image storage unit which stores a distorted circular image constituted with aggregates of pixels arranged at a position indicated by coordinates (x, y) on a two-dimensional XY orthogonal coordinate system and having a radius R taking an origin O of the XY coordinate system as a center;

a planar regular image storage unit which stores a planar regular image constituted with aggregates of pixels arranged at a position indicated by coordinates (u, v) on a two-dimensional UV orthogonal coordinate system;

a distorted circular image display unit which displays the distorted circular image stored in the distorted circular image storage unit;

an instruction input unit which inputs a position of a cut-out center point P and a cut-out orientation on the distorted circular image displayed on the basis of instructions by a user;

an intersecting point operation unit which, when a virtual sphere having a radius R taking the origin O as a center is defined in a three-dimensional XYZ orthogonal coordinate system including the two-dimensional XY orthogonal coordinate system, determines position coordinates (x₀, y₀, z₀) of an intersecting point G between a straight line passing through the cut-out center point P and parallel with a Z axis and the virtual sphere;

an angle determining unit which determines on the basis of the cut-out orientation a planar inclination angle φ

given as an angle formed between a vector U facing an U axis of the two-dimensional UV orthogonal coordinate system which is defined on a tangent plane in contact with the virtual sphere at the intersecting point G and a vector X facing an X axis of the two-dimensional XY orthogonal coordinate system; and

a transformation unit which makes correspondence between coordinates (u, v) and coordinates (x, y) by utilizing transformation equations based on an orthogonal projection method including the position coordinates (x₀, y₀, z₀) and the planar inclination angle φ

as parameters, determines a pixel value of a pixel on the planar regular image arranged at a position indicated by coordinates (u, v) on the basis of a pixel value of a reference pixel on the distorted circular image arranged at a position indicated by corresponding coordinates (x, y), and forms the planar regular image with regard to a partial image cut out from the distorted circular image in an orientation indicated by the planar inclination angle φ

taking the cut-out center point P as a center, so that the thus formed planar regular image is stored in the planar regular image storage unit.

2. The image converter according to claim 1, wherein

the transformation unit uses the following equations as transformation equations based on the orthogonal projection method,

x=R[(u−

x₀)A+(v−

y₀)B+(w−

z₀)E]/√

((u−

x₀)²+(v−

y₀)²+(w−

z₀)²)

y=R[(u−

x₀)C+(v−

y₀)D+(w−

z₀)F]/√

((u−

x₀)²+(v−

y₀)²+(w−

z₀)²)

where

A=1−

(1−

cos φ

)(y₀²+z₀²)

B=z₀ sin φ

+x₀y₀(1−

cos φ

)

C=z₀ sin φ

+x₀y₀(1−

cos φ

)

D=1−

(1−

cos φ

)(z₀²+x₀²)

E=y₀ sin φ

+z₀x₀(1−

cos φ

)

F=−

x₀ sin φ

+y₀x₀(1−

cos φ

)

w=mR (m is a predetermined conversion magnification).

3. The image converter according to claim 1, wherein

the instruction input unit has a function of entering instructions for defining a reference straight line J drawn on the distorted circular image, and

the angle determining unit determines the planar inclination angle φ

on the basis of an angle θ

formed between the reference straight line J and the X axis (on the condition that θ

=0°

where both of them are parallel).

4. The image converter according to claim 3, wherein

the instruction input unit has a function of entering instructions for designating positions of two points, a cut-out center point P and an auxiliary point Q, on the distorted circular image and giving a straight line connecting the cut-out center point P with the auxiliary point Q as the reference straight line J.

5. The image converter according to claim 3, wherein

the instruction input unit has a function of entering a numerical value indicating an angle θ

formed between the reference straight line J and the X axis on a predetermined input screen and also has a function of entering instructions for designating a position of the cut-out center point P on the distorted circular image.

6. The image converter according to claim 3, wherein

the angle determining unit gives the angle θ

formed between the reference straight line J and the X axis approximately as the planar inclination angle φ

.

7. The image converter according to claim 2, wherein

the instruction input unit has a function of entering a conversion magnification m on the basis of instructions by a user, and

the transformation unit makes the correspondence by using the entered conversion magnification m in the transformation equations.

8. The image converter according to claim 7, wherein

the instruction input unit has;

a function of entering instructions for designating positions of two points, a cut-out center point P and an auxiliary point Q, on the distorted circular image and uses a numerical value given as m=k/d (k is a predetermined proportional constant) on the basis of a distance d between the cut-out center point P and the auxiliary point Q as a conversion magnification m.

9. The image converter according to claim 1, wherein

the transformation unit performs, in order to determine a pixel value of a pixel on the planar regular image arranged at a position indicated by coordinates (u, v), interpolation operation for pixel values of a plurality of reference pixels on the distorted circular image arranged in the vicinity of a position indicated by the corresponding coordinates (x, y).

10. The image converter according to claim 1, wherein

the image converter has a function to perform image conversion when an image stored in the distorted circular image storage unit is not an orthogonally projected image photographed by use of a fisheye lens based on the orthogonal projection method but a non-orthogonally projected image photographed by use of a fisheye lens based on a non-orthogonal projection method,

the image converter includes first coordinate conversion equations for converting coordinates on the non-orthogonally projected image to coordinates on the orthogonally projected image and second coordinate conversion equations for converting coordinates on the orthogonally projected image to coordinates on the non-orthogonally projected image,

the intersecting point operation unit converts the coordinates of the cut-out center point P by using the first coordinate conversion equations and determines the position coordinates (x₀, y₀, z₀) of the intersecting point by using coordinates after conversion, and

the transformation unit uses a transformation equation based on the orthogonal projection method to determine coordinates (x, y) corresponding to coordinates (u, v), thereafter, the second coordinate conversion equation is used to convert the coordinates (x, y), and the coordinates after conversion are used to identify a position of a reference pixel on the distorted circular image.

11. The image converter according to claim 10, wherein

when an image stored in the distorted circular image storage unit is an equidistantly projected image photographed by use of a fisheye lens based on an equidistance projection method,

the intersecting point operation unit uses the following equations as the first coordinate conversion equations for converting coordinates (x′

, y′

) on the equidistantly projected image to the coordinates (x, y) on the orthogonally projected image

x=sinc(π

/2·

√

(x′

²+y′

²))×

π

/2·

x′

y=sinc(π

/2·

√

(x′

²+y′

²))×

π

/2·

y′

, and

the transformation unit uses the following equations as the second coordinate conversion equations for converting the coordinates (x, y) on the orthogonally projected image to the coordinates (x′

, y′

) on the equidistantly projected image

x′

=2/π

·

x/sinc(π

/2·

√

(x²+y²))

y′

=2/π

·

y/sinc(π

/2·

√

(x²+y²)).

12. A program recording medium, said program allowing a computer to function as the image converter according to claim 1.

13. A semiconductor integrated circuit in which an electronic circuit functioning as the transformation unit of the image converter according to claim 1 is assembled.

14. A fisheye lens monitoring system including

the image converter according to claim 1, a camera using a fisheye lens and a monitoring device of displaying a planar regular image on a screen, wherein

the fisheye lens monitoring system is constituted so that a distorted circular image photographed by using the camera is stored in the distorted circular image storage unit and the planar regular image obtained in the planar regular image storage unit is displayed by the monitoring device.

15. The image converter according to claim 2, wherein

the transformation unit has a first function table which allows values of “

function f(c)=1/c”

to correspond to values of various variables c and a second function table which allows values of “

function f(ξ

)=1/√

ξ

”

to correspond to values of various variables ξ

,

the transformation unit makes an operation to determine values of c and ξ

with the following equations,

a=u−

x₀, b=v−

y₀, c=w−

z₀,

ξ

=(a/c)²+(b/c)²+1

so that values of the functions f(c) and f(ξ

) which correspond to the thus determined values of c and g are determined by referring to the first function table and the second function table, and a value of √

((u−

x₀)²+(v−

y₀)²+(w−

z₀)²) in the transformation equation based on the orthogonal projection method is determined by operation of f(c)×

f(ξ

).

16. The image converter according to claim 1, wherein

the transformation unit comprising:

an even-number function table in which among variables t giving a discrete value of an interval W according to predetermined effective digits, a value of a predetermined function f(t) is allowed to correspond to an even-number variable t;

an odd-number function table in which among variables t giving a discrete value of the interval W according to the effective digits, a value of the function f(t) is allowed to correspond to an odd-number variable t;

a T resistor which stores variables t constituted with higher-order bits made up of the effective digits and lower-order bits showing digits lower than the effective digits;

an even-number reading unit which reads from the even-number function table a value of the function f(t) allowed to correspond to an even-number variable t shown by the higher-order bits when the higher-order bits show an even-number and reads from the even-number function table a value of the function f(t) allowed to correspond to a minimum even-number variable t greater than an odd-number variable t shown by the higher-order bits when the higher-order bits show an odd-number;

an odd-number reading unit which reads from the odd-number function table a value of the function f(t) allowed to correspond to an odd-number variable t shown by the higher-order bits when the higher-order bits show an odd-number and reads from the odd-number function table a value of the function f(t) allowed to correspond to a minimum odd-number variable t greater than an even-number variable t shown by the higher-order bits when the higher-order bits show an even-number;

an A resistor which stores a given value of the function f(t) read from the even-number function table or the odd-number function table;

a B resistor which stores a given value of the function f(t) read from the even-number function table or the odd-number function table;

an even-odd selector which gives a value of the function f(t) read by the even-number reading unit to the A resistor and gives a value of the function f(t) read by the odd-number reading unit to the B resistor when the higher-order bits show an even-number, and which gives a value of the function f(t) read by the odd-number reading unit to the A resistor and gives a value of the function f(t) read by the even-number reading unit to the B resistor when the higher-order bits show an odd-number; and

an interpolation operating unit in which a value stored in the A resistor is given as f(A) and a value stored in the B resistor is given as f(B), and a value indicated by the lower-order bit is given as δ

, and a value of the function f(t) after interpolation is determined by the following operation,

f(t)=((W−

δ

)/W)×

f(A)+(δ

/W)×

f(B), and

wherein a value of the function f(t) after interpolation which is determined by the interpolation operating unit is utilized to perform the operation.

17. The image converter according to claim 11, wherein

the intersecting point operation unit operates the function sinc (t) in the first coordinate conversion equations on the basis of the following form of equation subjected to Taylor expansion

sinc(t)=1−

t²/3!+t⁴/5!−

t⁶/7!+t⁸/9!−

. . . , and

the transformation unit operates the function 1/sinc (t) in the second coordinate conversion equations on the basis of the following form of equation using predetermined coefficient values of a₂, a₄, a₆, a₈, . . . .

1/sinc(t)=1+a₂t²+a₄t⁴+a₆t⁶+a₈t⁸+ . . . .

18. The image converter according to claim 2, wherein

the transformation unit determines a value of cos φ

by dividing an inner product of the vector U and the vector X by a product of a size of the vector U and that of the vector X.

19. The image converter according to claim 2, wherein

the instruction input unit has a function of entering instructions for defining the reference straight line J drawn on the distorted circular image, and

the transformation unit defines a vector J facing a direction of the reference straight line J and determines a value of cos φ

by dividing an inner product of the vector J and the vector X by a product of a size of the vector J and that of the vector X.

20. The image converter according to claim 2, wherein

the transformation unit determines a value of sin φ

on the basis of an operation of sin φ

=√

(1−

cos²φ

).

21. An image converter for performing processes of cutting out a part of a distorted circular image photographed by use of a fisheye lens and converting it into a planar regular image, the image converter comprising:

a distorted circular image storage unit which stores a distorted circular image constituted with aggregates of pixels arranged at a position indicated by coordinates (x, y) on a two-dimensional XY orthogonal coordinate system;

a planar regular image storage unit which stores a planar regular image constituted with aggregates of pixels arranged at a position indicated by coordinates (u, v) on a two-dimensional UV orthogonal coordinate system;

a distorted circular image display unit which displays the distorted circular image stored in the distorted circular image storage unit;

an instruction input unit which inputs positions of two points, a cut-out center point P and an auxiliary point Q on the basis of instructions given by a user on the distorted circular image displayed, and recognizes a straight line connecting the cut-out center point P with the auxiliary point Q as a reference straight line J, and a numerical value given as m=k/d (k is a predetermined proportional constant) as a conversion magnification m on the basis of a distance d between the cut-out center point P and the auxiliary point Q; and

a transformation unit which determines a pixel value of a pixel on the planar regular image arranged at a position indicated by coordinates (u, v) on the basis of a pixel value of a reference pixel on the distorted circular image arranged at a position indicated by corresponding coordinates (x, y) by utilizing a transformation equation for allowing the coordinates (u, v) to correspond to the coordinates (x, y), and performs an operation for forming a planar regular image subjected to scaling on the basis of the conversion magnification m with regard to a partial image cut out from the distorted circular image to an orientation in accordance with the reference straight line J taking the cut-out center point P as a center, so that the thus formed planar regular image is stored in the planar regular image storage unit.

22. An image converter for performing processes of cutting out a part of a distorted circular image photographed by use of a fisheye lens and converting it into a planar regular image, the image converter comprising:

a distorted circular image storage unit which stores a distorted circular image constituted with aggregates of pixels arranged at a position indicated by coordinates (x, y) on a two-dimensional XY orthogonal coordinate system;

a planar regular image storage unit which stores a planar regular image constituted with aggregates of pixels arranged at a position indicated by coordinates (u, v) on a two-dimensional UV orthogonal coordinate system;

a distorted circular image display unit which displays the distorted circular image stored in the distorted circular image storage unit;

an instruction input unit which inputs an angle θ

defined as an angle formed between a reference straight line J and the X axis and a conversion magnification m on the basis of instructions given by a user on a input screen, and a position of a cut-out center point P on the basis of instructions given by the user on the distorted circular image displayed; and

a transformation unit which determines a pixel value of a pixel on the planar regular image arranged at a position indicated by coordinates (u, v) on the basis of a pixel value of a reference pixel on the distorted circular image arranged at a position indicated by corresponding coordinates (x, y) by utilizing a transformation equation for allowing the coordinates (u, v) to correspond to the coordinates (x, y), and performs an operation for forming a planar regular image subjected to scaling on the basis of the conversion magnification m with regard to a partial image cut out from the distorted circular image to an orientation in accordance with the reference straight line J taking the cut-out center point P as a center, so that the thus formed planar regular image is stored in the planar regular image storage unit.

23. An image converting method for performing processes of cutting out a part of a distorted circular image photographed by use of a fisheye lens and converting it into a planar regular image, the image converting method which allows a computer or an electronic circuit to perform the following steps:

storing in a distorted circular image storage unit a distorted circular image constituted with aggregates of pixels arranged at a position indicated by coordinates (x, y) on a two-dimensional XY orthogonal coordinate system and having a radius R taking an origin O of the XY coordinate system as a center;

displaying on a display the distorted circular image stored in the distorted circular image storage unit;

entering position and cut-out orientation of a cut-out center point P on the distorted circular image displayed on the display on the basis of instructions by a user;

determining position coordinates (x₀, y₀, z₀) of an intersecting point G between a straight line passing through the cut-out center point P and parallel with a Z axis and a virtual sphere, by defining a virtual sphere having the radius R taking the origin O as a center in a three-dimensional XYZ orthogonal coordinate system including the two-dimensional XY orthogonal coordinate system;

determining on the basis of the cut-out orientation a planar inclination angle φ

given as an angle formed between a vector U facing a direction of an U axis of a two-dimensional UV orthogonal coordinate system to be defined on a tangent plane in contact with the virtual sphere on the intersecting point G and a vector X facing a direction of an X axis of the two-dimensional XY orthogonal coordinate system; and

performing operation for forming a planar regular image with regard to a partial image cut out from the distorted circular image to an orientation indicated by the planar inclination angle φ

taking the cut-out center point P as a center by utilizing transformation equations based on an orthogonal projection method including the position coordinates (x₀, y₀, z₀) and the planar inclination angle φ

as parameters to allow coordinates (u, v) to correspond to coordinates (x, y), and determining a pixel value of each pixel on the planar regular image constituted with aggregates of pixels arranged at a position indicated by the coordinates (u, v) on the basis of a pixel value of a reference pixel on the distorted circular image arranged at a position indicated by the corresponding coordinates (x, y).

24. The image converting method according to claim 23, wherein

the following equations are used as transformation equations based on the orthogonal projection method,

x=R[(u−

x₀)A+(v−

y₀)B+(w−

z₀)E]/√

((u−

x₀)²+(v−

y₀)²+(w−

z₀)²)

y=R[(u−

x₀)C+(v−

y₀)D+(w−

z₀)F]/√

((u−

x₀)²+(v−

y₀)²+(w−

z₀)²)

where

A=1−

(1−

cos φ

)(y₀²+z₀²)

B=−

z₀ sin φ

+x₀y₀(1−

cos φ

)

C=z₀ sin φ

+x₀y₀(1−

cos φ

)

D=1−

(1−

cos φ

)(z₀²+x₀²)

E=y₀ sin φ

+z₀x₀(1−

cos φ

)

F=−

x₀ sin φ

+y₀x₀(1−

cos φ

)

w=m R (m is a predetermined conversion magnification).

25. The image converting method according to claim 23, wherein

instructions are entered for designating positions of two points, the cut-out center point P and an auxiliary point Q, on the distorted circular image displayed on the display, a straight line connecting the cut-out center point P with the auxiliary point Q is given as a reference straight line J, and a planar inclination angle φ

is determined on the basis of an angle θ

formed between the reference straight line J and the X axis (on the condition that θ

is equal to 0°

where they are parallel).

26. The image converting method according to claim 23, wherein

an instruction for designating a position of the cut-out center point P is entered on the distorted circular image displayed on the display, and an instruction for designating an angle θ

between the reference straight line J and the X axis on the distorted circular image is entered on an input screen, and the planar inclination angle φ

is determined on the basis of the angle θ

.