Image processing apparatus, image processing system, imaging apparatus and image processing method

• US 7,596,286 B2
• Filed: 07/30/2004
• Issued: 09/29/2009
• Est. Priority Date: 08/06/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Aoyama, Koji, Kurata, Tohru, Nose, Atsushi, Wada, Kimitaka
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): Patel; Jayesh A
• Application Number:
  US10/902,140
• Publication Number:
  US 20050053307A1
• Time in Patent Office: 1,887 Days
• Field of Search: None
• US Class Current: 382/300
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations

Abstract

An image processing apparatus includes an zoom position acquiring section for acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking the inputted image, a zooming-direction decoding section for decompressing correction parameter associated with the zoom position, which is acquired by the zoom position acquiring section, according to a zoom compression parameter, into which a correction parameter for distortion correction corresponding to each lens position condition is compressed by utilizing zoom partitioning points at which lens position conditions of the optical zoom mechanism are sequentially partitioned into plural levels that are set between a wide-end and a tele-end and that include both ends, and an image correction section for correcting distortion of the inputted image according to the correction parameter decompressed by the zooming-direction decoding section.

Claims

1. An image processing apparatus for correcting distortion in an image, comprising:

a converging lens group having an optical zoom mechanism;

an imaging element for receiving an input image taken by using the converging lens group having an optical zoom mechanism;

a zoom position acquiring device configured to acquire a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a coordinate designating device configured to designate coordinates of a pixel, which is an object for distortion correction;

a parameter decoding device configured to calculate correction parameters corresponding to the coordinates designated by the coordinate designating device and to the zoom position acquired by the zoom position acquiring device according to the correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

an image correction device configured to correct distortion of the input image according to the correction parameters calculated by the parameter decoding device;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding device performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and further performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction,

wherein the parameter decoding device comprises;

an interpolation calculation device configured to calculate interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

an Y-X-compressed data holding device configured to temporarily hold the correction parameters, which are interpolated in a zooming direction by the interpolation calculation device and needed for distortion correction of data of one line;

an X-compressed data holding device configured to temporarily hold the correction parameters, which are interpolated in a vertical direction by the interpolation calculation device and needed for distortion correction of data of one line;

an input selection device configured to selectively input to the interpolation calculation device the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding device, and the correction parameters read from the X-compressed data holding device; and

a control device configured to control a selecting operation in the input selection device according to the coordinates designated by the coordinate designating device, and to the zoom position acquired by the zoom position acquiring device, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation device.

2. The image processing apparatus according to claim 1, wherein:

the control device controls the input selecting device such that the input selecting device selects the correction parameters read from the Y-X-compressed data holding device after selecting the correction parameters, which correspond to the zoom position and are compressed in the horizontal direction, the vertical direction, and the zooming direction, in the horizontal retrace period, and that the input selecting device selects the correction parameters read from the X-compressed data holding device in an image effective period after termination of the horizontal retrace period.

3. The image processing apparatus according to claim 2, wherein:

if the control device causes the input selecting device to select the correction parameters compressed in the horizontal direction, the vertical direction and the zooming direction, the control device calculates the phase coefficient corresponding to the zooming direction;

if the control device causes the input selecting device to select the correction parameters read from the Y-X-compressed data holding device, the control device calculates the phase coefficient corresponding to the vertical direction; and

if the control device causes the input selecting device to select the correction parameters read from the X-compressed data holding device, the control device calculates the phase coefficient corresponding to the horizontal direction.

4. An image processing apparatus for correcting distortion in an image, comprising:

a converging lens group having an optical zoom mechanism;

an imaging element for receiving an input image taken by using the converging lens group having an optical zoom mechanism;

a zoom position acquiring device configured to acquire a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a coordinate designating device configured to designate coordinates of a pixel, which is an object for distortion correction;

a parameter decoding device configured to calculate correction parameters corresponding to the coordinates designated by the coordinate designating device and to the zoom position acquired by the zoom position acquiring device according to the correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

an image correction device configured to correct distortion of the input image according to the correction parameters calculated by the parameter decoding device,

wherein the correction parameters supplied to the parameter decoding device are generated by partitioning the image, whose distortion has been corrected, in a lattice form, and compressing the correction parameters corresponding to a lattice segment in the lateral direction and the longitudinal direction through utilization of a position of each lattice point, and by thereafter further compressing the correction parameters, which are compressed in the lateral direction and the longitudinal direction and correspond to each of the lens position conditions, in the zooming direction through utilization of zoom partitioning points, at which the lens position conditions in the optical zoom mechanism are serially partitioned between a wide-end and a tele-end into plural levels including both the ends,

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding device performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and further performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction, and

wherein the parameter decoding device performs interpolation calculation by linear interpolation according to the correction parameters and a phase coefficient, the correction parameters corresponding to a plurality of the zoom partitioning points selected in accordance with the zoom position and the lattice points in the lattice selected in accordance with the coordinates designated by the coordinate designating device, the phase coefficient representing relative-coordinates of the zoom position and the coordinates between the zoom partitioning points and within the lattice.

5. An image processing apparatus for correcting distortion in an image, comprising:

a converging lens group having an optical zoom mechanism;

an imaging element for receiving an input image taken by using the converging lens group having an optical zoom mechanism;

a zoom position acquiring device configured to acquire a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a coordinate designating device configured to designate coordinates of a pixel, which is an object for distortion correction;

a parameter decoding device configured to calculate correction parameters corresponding to the coordinates designated by the coordinate designating device and to the zoom position acquired by the zoom position acquiring device according to the correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

an image correction device configured to correct distortion of the input image according to the correction parameters calculated by the parameter decoding device;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding device performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and further performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction,

wherein the correction parameters supplied to the parameter decoding device are generated by partitioning the image, whose distortion has been corrected, in a lattice form, and compressing the correction parameters corresponding to a lattice segment in the lateral direction and the longitudinal direction through utilization of a position of each lattice point, and by thereafter further compressing the correction parameters, which are compressed in the lateral direction and the longitudinal direction and correspond to each of the lens position conditions, in the zooming direction through utilization of zoom partitioning points, at which the lens position conditions in the optical zoom mechanism are serially partitioned between a wide-end and a tele-end into plural levels including both the ends, and

wherein based on the correction parameters corresponding to two of the zoom partitioning points selected according to the zoom position and further corresponding to the lattice points selected in the lattice according to the coordinates designated by the coordinate designating device, and further corresponding to internal division points obtained by dividing each of an interval between the selected zoom partitioning points and the lattice segment in the lattice into n parts (where n is a natural number), the parameter decoding device performs interpolation calculation by restoring an nth degree polynomial, with which change of each of the correction parameters corresponding to each of the interval between the selected zoom partitioning points and the lattice segment in the lattice is approximated, and by applying a phase coefficient, which represents a relative coordinate of the zoom position and the coordinate in the interval of the zoom partitioning points and in the lattice, to the nth degree polynomial.

6. An image processing system for correcting distortion of an image, comprising:

a converging lens having an optical zoom, mechanism;

an imaging element receiving an input image taken by using the converging lens group having an optical zoom mechanism;

a zoom position acquiring device configured to acquire a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a coordinate designating device for designating coordinates of a pixel, which is an object for distortion correction;

a parameter decoding device configured to calculate correction parameters corresponding to the coordinates designated by the coordinate designating device and to the zoom position acquired by the zoom position acquiring device according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

an image correction device configured to correct distortion of the input image according to the correction parameters calculated by the parameter decoding device;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding device performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction,

wherein the parameter decoding device comprises;

an interpolation calculation device for calculating interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

an Y-X-compressed data holding device configured to temporarily hold the correction parameters, which are interpolated in a zooming direction by the interpolation calculation device and needed for distortion correction of data of one line;

an X-compressed data holding device for temporarily holding the correction parameters, which are interpolated in a vertical direction by the interpolation calculation device and needed for distortion correction of data of one line;

an input selection device for selectively inputting to the interpolation calculation device the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding device, and the correction parameters read from the X-compressed data holding device; and

a control device configured to control a selecting operation in the input selection device according to the coordinates designated by the coordinate designating device, and to the zoom position acquired by the zoom position acquiring device, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation device.

7. An imaging apparatus for correcting distortion of an image, comprising:

a converging lens group having an optical zoom mechanism;

an imaging element for imaging an image by using the converging lens group having an optical zoom mechanism;

a zoom position detecting device configured to detect a zoom position indicating a lens position condition of the optical zoom mechanism on taking an input image;

a coordinate designating device configured to designate coordinates of a pixel, which is an object for distortion correction;

a parameter decoding device configured to calculate correction parameters corresponding to the coordinates designated by the coordinate designating device and to the zoom position acquired by the zoom position acquiring device according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

an image correction device configured to correct distortion of the input image according to the correction parameters calculated by the parameter decoding device;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding device performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction, and

wherein the parameter decoding device comprises;

an interpolation calculation device configured to calculate interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

an Y-X-compressed data holding device configured to temporarily hold the correction parameters, which are interpolated in a zooming direction by the interpolation calculation device and needed for distortion correction of data of one line;

an X-compressed data holding device configured to temporarily hold the correction parameters, which are interpolated in a vertical direction by the interpolation calculation device and needed for distortion correction of data of one line;

an input selection device configured to selectively input to the interpolation calculation device the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding device, and the correction parameters read from the X-compressed data holding device; and

a control device configured to control a selecting operation in the input selection device according to the coordinates designated by the coordinate designating device, and to the zoom position detected by the zoom position detecting device, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation device.

8. An image processing method for correcting distortion of an image, comprising:

a first step of receiving an input image taken by using a converging lens group having an optical zoom mechanism;

a second step of acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a third step of calculating the correction parameters corresponding to coordinates designating a pixel, which is an object for distortion correction, and to the acquired zoom position by interpolation calculation according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected; and

a fourth step of correcting distortion of the input image according to the calculated correction parameters;

wherein in the third step, if a correction process in the horizontal direction is performed on data of one line, the interpolation calculation is performed on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and that in an image effective period after the termination of this horizontal retrace period, interpolation calculation is performed on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction, and

wherein in the third step, in the horizontal retrace period, interpolation calculation corresponding to the zooming direction is performed by using the correction parameter, which is compressed in the horizontal direction, the vertical direction, the zooming direction, and a phase coefficient corresponding to the zoom position, and the calculated correction parameters are temporarily stored in a first storage area;

subsequently, interpolation calculation corresponding to the vertical direction is performed by using the correction parameter read from the first storage area, and the phase coefficient obtained by the vertical coordinate corresponding to the one line, and the calculated correction parameter is temporarily stored in a second storage area; and

in the image effective period after termination of the horizontal retrace period, interpolation calculation corresponding to the horizontal direction is performed by using the correction parameter read from the third storage area, and further using the phase coefficient obtained from each of the coordinates in the horizontal direction.