Coordinate computation device and method, and an image processing device and method

• US 9,436,973 B2
• Filed: 05/23/2014
• Issued: 09/06/2016
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Aoki, Toru, Matoba, Narihiro
• Primary Examiner(s): Liu, Li
• Application Number:
  US14/785,801
• Publication Number:
  US 20160078590A1
• Time in Patent Office: 837 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

In an image processing for correcting a distorted image obtained by photography by use of a super-wide angle optical system such as a fisheye lens or an omnidirectional mirror, to obtain an image of a perspective projection method, a composite index (Rn) combining a height on the projection sphere with a distance from the optical axis is computed (301), and a distance (Rf) from an origin in the distorted image is computed (302), using the composite index (Rn). Further, two-dimensional coordinates (p, q) in the distorted image are computed (303), using the distance (Rf) from the origin, and a pixel value in an output image is determined using a pixel at a position in the distorted image specified by the two-dimensional coordinates, or a pixel or pixels neighboring the specified position. It is possible to perform the projection from the projection sphere to the image plane, that is, the computation of the coordinates on the coordinate plane, while restricting the amount of computation.

Claims

1. A coordinate computation device that transforms three-dimensional coordinates on a projection sphere to two-dimensional coordinates on a distorted image, comprising:

a composite index computation unit for computing a composite index which is a combination of a height of a point of interest on the projection sphere and a distance of the point of interest from an optical axis which are obtained from said three-dimensional coordinates;

a distance computation unit for computing a distance from an origin in said distorted image, of a point in said distorted image corresponding to said point of interest, from said composite index; and

a coordinate computation unit for computing the two-dimensional coordinates in said distorted image, from the distance from said origin in said distorted image,

wherein, when said three-dimensional coordinates are represented by (x, y, z), and a radius of said projection sphere is denoted by Rb, said composite index computation unit determines said composite index Rn according to;

Rn=√

(x²+y²) if z≧

√

(x²+y²), and

Rn=√

2×

Rb−

zif z<

√

(x²+y²).

2. The coordinate computation device as set forth in claim 1, wherein, when a zenith angle on said projection sphere is denoted by θ

, and the distance from said origin of said corresponding point in said distorted image is denoted by Rf, said distance computation unit determines the distance Rf from said origin, according to;

3. The coordinate computation device as set forth in claim 2, wherein said coordinate computation unit determines said two-dimensional coordinates (p, q), according to:

p=x×

Rf/√

(x²+y²)

q=y×

Rf/√

(x²+y²)

if Rf≠

0; and

according to;

p=0

q=0

if Rf=0.

4. The coordinate computation device as set forth in claim 1, wherein said distance computation unit

has a lookup table storing a relationship between an input value DN and an output value DF which approximates, by a polyline, a relationship between said composite index Rn and the distance Rf from said origin, finds, based on said composite index Rn computed by said composite index computation unit, DN[n], DN[n+1] satisfying

DN[n]≦

Rn<

DN[n+1],

reads output values DF[n], DF[n+1] corresponding to DN[n], DN[n+1], and

determines the distance Rf from said origin according to;

5. The coordinate computation device as set forth in claim 4, wherein said coordinate computation unit determines said two-dimensional coordinates (p, q), according to:

p=x×

Rf/√

(x²+y²)

q=y×

Rf/√

(x²+y²)

if Rf≠

0; and

according to;

p=0

q=0

if Rf=0.

6. An image processing device having:

a coordinate computation device as set forth in claim 1;

a spherical coordinate computation unit for taking each pixel in the output image as a pixel of interest, and computing the three-dimensional coordinates on said projection sphere corresponding to coordinates of the pixel of interest;

a pixel value computation unit for computing a pixel value of said pixel of interest in said output image, based on the two-dimensional coordinates in said distorted image computed by said coordinate computation device;

wherein

said coordinate computation device determines the two-dimensional coordinates representing a position in said distorted image corresponding to the three-dimensional coordinates computed by said spherical coordinate computation unit, as the coordinates representing a position corresponding to the coordinates of said pixel of interest; and

if there is a pixel at the position in said distorted image corresponding to the coordinates of said pixel of interest, said pixel value computation unit uses a pixel value of the pixel at said corresponding position, as the pixel value of said pixel of interest; and

if there is no pixel at the position in said distorted image corresponding to the coordinates of said pixel of interest, said pixel value computation unit determines the pixel value of said pixel of interest by interpolation from a pixel value or pixel values of one or more pixels in the neighborhood of said corresponding position.

7. A coordinate computation method that transforms three-dimensional coordinates on a projection sphere to two-dimensional coordinates on a distorted image, comprising:

a composite index computation step for computing a composite index which is a combination of a height of a point of interest on the projection sphere and a distance of the point of interest from an optical axis which are obtained from said three-dimensional coordinates;

a distance computation step for computing a distance from an origin in said distorted image, of a point in said distorted image corresponding to said point of interest, from said composite index; and

a coordinate computation step for computing the two-dimensional coordinates in said distorted image, from the distance from said origin in said distorted image,

wherein, when said three-dimensional coordinates are represented by (x, y, z), and a radius of said projection sphere is denoted by Rb, said composite index computation step determines said composite index Rn according to;

Rn=√

(x²+y²) if z≧

√

(x²+y²), and

Rn=√

2×

Rb−

z if z<

√

(x²+y)².

8. An image processing method comprising:

said composite index computation step, said distance computation step, and said coordinate computation step constituting the coordinate computation method as set forth in claim 7;

a spherical coordinate computation step for taking each pixel in the output image as a pixel of interest, and computing the three-dimensional coordinates on said projection sphere corresponding to coordinates of the pixel of interest;

a pixel value computation step for computing a pixel value of said pixel of interest in said output image, based on the two-dimensional coordinates in said distorted image computed in said coordinate computation step;

wherein

said coordinate computation step determines the two-dimensional coordinates representing a position in said distorted image corresponding to the three-dimensional coordinates computed in said spherical coordinate computation step, as the coordinates representing a position corresponding to the coordinates of said pixel of interest; and

if there is a pixel at the position in said distorted image corresponding to the coordinates of said pixel of interest, said pixel value computation step uses a pixel value of the pixel at said corresponding position, as the pixel value of said pixel of interest; and

if there is no pixel at the position in said distorted image corresponding to the coordinates of said pixel of interest, said pixel value computation step determines the pixel value of said pixel of interest by interpolation from a pixel value or pixel values of one or more pixels in the neighborhood of said corresponding position.