Image processing apparatus, image processing system, imaging apparatus and image processing method

• US 20050053307A1
• Filed: 07/30/2004
• Published: 03/10/2005
• Est. Priority Date: 08/06/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Aoyama, Koji, Kurata, Tohru, Nose, Atsushi, Wada, Kimitaka
• Granted Patent: US 7,596,286 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations

Abstract

An image processing apparatus includes an zoom position acquiring section for acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking the inputted image, a zooming-direction decoding section for decompressing correction parameter associated with the zoom position, which is acquired by the zoom position acquiring section, according to a zoom compression parameter, into which a correction parameter for distortion correction corresponding to each lens position condition is compressed by utilizing zoom partitioning points at which lens position conditions of the optical zoom mechanism are sequentially partitioned into plural levels that are set between a wide-end and a tele-end and that include both ends, and an image correction section for correcting distortion of the inputted image according to the correction parameter decompressed by the zooming-direction decoding section.

Claims

1. An image processing apparatus for receiving an inputted image, which is taken by using a converging lens group having an optical zoom mechanism, and for correcting distortion of the image, the image processing apparatus comprising:

zoom position acquiring means for acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an inputted image;

zooming-direction decoding means for decompressing correction parameters associated to the zoom position, which is acquired by the zoom position acquiring means, according to a zoom compression parameter, into which the correction parameters for distortion correction corresponding to each lens position condition is compressed by utilizing zoom partitioning points at which lens position conditions of the optical zoom mechanism are sequentially partitioned into plural levels that are set between a wide-end and a tele-end and that include both the ends; and

image correction means for correcting distortion of the inputted image according to the correction parameter decompressed by the zooming-direction decoding means.

2. The image processing apparatus according to claim 1, wherein:

the zooming-direction decoding means acquires the correction parameters corresponding to a plurality of the zoom partitioning points selected according to the zoom position as the zoom compression parameters, and performs interpolation calculation by linear interpolation according to a relative position of the zoom position in an interval of the plurality of the zoom-partitioning points and to the zoom compression parameter.

3. The image processing apparatus according to claim 1,

wherein the zooming-direction decoding means restores an n-th degree polynomial (where n is a natural number) according to the zoom compression parameter, a change in the correction parameter corresponding to each of lens position conditions being approximated with the n-th degree polynomial in one of intervals of a plurality of the zoom partitioning points selected according to the zoom position, and

wherein the zooming-direction decoding means performs interpolation calculation by applying to the n-th degree polynomial a relative position of the zoom position in the one of the intervals of the plurality of the selected zoom partitioning points.

4. The image processing apparatus according to claim 1, further comprising:

parameter storing means having two storage areas for storing the zoom compression parameter;

parameter selecting means for selecting the zoom compression parameter, which is necessary for interpolation calculation by the zooming-direction decoding means, according to change of the zoom position;

storage area selecting means for selecting the storage area that stores the zoom compression parameter selected by the parameter selecting means; and

phase calculation means for calculating a relative position of the zoom position, which is located between the zoom partitioning points corresponding to the zoom compression parameter selected by the parameter selecting means, as a phase coefficient;

wherein the zooming-direction decoding means performs interpolation calculation according to the zoom compression parameter read from the storage areas and to the phase coefficient calculated by the phase calculation means.

5. The image processing apparatus according to claim 4, wherein:

if the zoom position changes, the parameter selecting means continues to select the zoom compression parameter corresponding to the zoom partitioning point, which is close to the changed zoom position, among the zoom portioning points corresponding to selection before the change, in a case where the changed zoom position is present outside a region between the zoom partitioning points corresponding to the zoom compression parameter selected before the change; and

the storage area selecting means selects the storage area in such a way as to write only the zoom compression parameter newly selected by the parameter selecting means after the zoom position changes.

6. The image processing apparatus according to claim 5, wherein:

the phase calculation means calculates a relative position of the zoom position, which is set by employing the zoom partitioning point corresponding to the zoom compression parameter stored in one of the storage area as a reference, as the phase coefficient at all times according to the zoom compression parameter selected by the parameter selecting means and to selection of the storage area by the storage area selecting means.

7. The image processing apparatus according to claim 1, further comprising:

X-Y-direction decoding means for decompressing X-Y-compressed parameters, which are obtained by compressing in a lateral direction and a longitudinal direction of an image, whose distortion has been corrected, by interpolation calculation;

wherein the zooming-direction decoding means receives compressed data, which is obtained by further compressing the X-Y-compressed parameters according to the zoom position, as the zoom compression parameters, and performs interpolation calculation and supplies decompressed X-Y-compressed parameters to the X-Y-direction decoding means; and

wherein the image correction means corrects distortion according to the correction parameters decompressed by the X-Y-direction decoding means by the interpolation calculation.

8. The image processing apparatus according to claim 7, wherein:

the X-Y-compressed parameters are generated by partitioning the image, whose distortion has been corrected, in a lattice form and by compressing the correction parameters corresponding to lattice lines by utilizing positions of each of lattice points; and

the X-Y-decoding means decompresses the X-Y-compressed parameter at each section of the lattice.

9. An image processing apparatus for receiving an input image taken by using a converging lens group having an optical zoom mechanism, and for correcting distortion of the image, the image processing apparatus comprising:

zoom position acquiring means for acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

coordinate designating means for designating coordinates of a pixel, which is an object for distortion correction;

parameter decoding means for calculating correction parameters corresponding to the coordinates designated by the coordinate designating means and to the zoom position acquired by the zoom position acquiring means according to the correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

image correction means for correcting distortion of the input image according to the correction parameters calculated by the parameter decoding means;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding means performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and further performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction.

10. The image processing apparatus according to claim 9, wherein the parameter decoding means comprises:

interpolation calculation means for calculating interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

Y-X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a zooming direction by the interpolation calculation means and needed for distortion correction of data of one line;

X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a vertical direction by the interpolation calculation means and needed for distortion correction of data of one line;

input selection means for selectively inputting to the interpolation calculation means the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding means, and the correction parameters read from the X-compressed data holding means; and

control means for controlling a selecting operation in the input selection means according to the coordinates designated by the coordinate designating means, and to the zoom position acquired by the zoom position acquiring means, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation means.

11. The image processing apparatus according to claim 10, wherein:

the control means controls the input selecting means such that the input selecting means selects the correction parameters read from the Y-X-compressed data holding means after selecting the correction parameters, which correspond to the zoom position and are compressed in the horizontal direction, the vertical direction, and the zooming direction, in the horizontal retrace period, and that the input selecting means selects the correction parameters read from the X-compressed data holding means in an image effective period after termination of the horizontal retrace period.

12. The image processing apparatus according to claim 11, wherein:

if the control means causes the input selecting means to select the correction parameters compressed in the horizontal direction, the vertical direction and the zooming direction, the control means calculates the phase coefficient corresponding to the zooming direction;

if the control means causes the input selecting means to select the correction parameters read from the Y-X-compressed data holding means, the control means calculates the phase coefficient corresponding to the vertical direction; and

if the control means causes the input selecting means to select the correction parameters read from the X-compressed data holding means, the control means calculates the phase coefficient corresponding to the horizontal direction.

13. The image processing apparatus according to claim 9, wherein:

the correction parameters supplied to the parameter decoding means are generated by partitioning the image, whose distortion has been corrected, in a lattice form, and compressing the correction parameters corresponding to a lattice segment in the lateral direction and the longitudinal direction through utilization of a position of each lattice point, and by thereafter further compressing the correction parameters, which are compressed in the lateral direction and the longitudinal direction and correspond to each of the lens position conditions, in the zooming direction through utilization of zoom partitioning points, at which the lens position conditions in the optical zoom mechanism are serially partitioned between a wide-end and a tele-end into plural levels including both the ends.

14. The image processing apparatus according to claim 13, wherein:

the parameter decoding means performs interpolation calculation by linear interpolation according to the correction parameters and a phase coefficient, the correction parameters corresponding to a plurality of the zoom partitioning points selected in accordance with the zoom position and the lattice points in the lattice selected in accordance with the coordinates designated by the coordinate designating means, the phase coefficient representing relative-coordinates of the zoom position and the coordinates between the zoom partitioning points and within the lattice.

15. The image processing apparatus according to claim 13, wherein:

based on the correction parameters corresponding to two of the zoom partitioning points selected according to the zoom position and further corresponding to the lattice points selected in the lattice according to the coordinates designated by the coordinate designating means, and further corresponding to internal division points obtained by dividing each of an interval between the selected zoom partitioning points and the lattice segment in the lattice into n parts (where n is a natural number), the parameter decoding means performs interpolation calculation by restoring an nth degree polynomial, with which change of each of the correction parameters corresponding to each of the interval between the selected zoom partitioning points and the lattice segment in the lattice is approximated, and by applying a phase coefficient, which represents a relative coordinate of the zoom position and the coordinate in the interval of the zoom partitioning points and in the lattice, to the nth degree polynomial.

16. An image processing system for receiving an input image taken by a converging lens group having an optical zoom mechanism and for correcting distortion of the image, the image processing system comprising:

zoom position acquiring means for acquiring a zoom position indicating a lens position condition of the optical zoom mechanism on taking an input image;

zooming-direction encoding means for compressing correction parameters for distortion correction, which correspond to each of a lens position condition by utilizing zoom petitioning points at which lens position conditions in the optical zoom mechanism are serially partitioned between a wide-end and a tele-end into plural levels including both the ends;

zooming-direction decoding means for decompressing the correction parameters corresponding to the zoom position acquired by the zoom position acquiring means by interpolation calculation according to the zoom compression parameters compressed by the zooming-direction encoding means; and

image correction means for correcting distortion of the input image according to the correction parameters decompressed by the zooming-direction decoding means.

17. The image processing system according to claim 16, further comprising:

parameter storing means having two storage areas for storing the zoom compression parameters outputted from the zooming-direction encoding means;

parameter selecting means for selecting the zoom compression parameter, which is needed for the interpolation calculation by the zooming-direction decoding means, according to change of the zoom position;

storage area selecting means for selecting the storage area that stores the zoom compression parameter selected by the parameter selecting means; and

phase computing means for calculating a relative position of the zoom position between the zoom partitioning points corresponding the zoom compression parameter, which is selected by the parameter selecting means, as a phase coefficient;

wherein the zooming-direction decoding means performs interpolation calculation according to the zoom compression parameter read from the storage area and to the phase coefficient calculated by the phase computing means.

18. An image processing system for receiving an input image taken by using a converging lens group having an optical zoom mechanism, and for correcting distortion of the image, the image processing system comprising:

zoom position acquiring means for acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

coordinate designating means for designating coordinates of a pixel, which is an object for distortion correction;

parameter decoding means for calculating correction parameters corresponding to the coordinates designated by the coordinate designating means and to the zoom position acquired by the zoom position acquiring means according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

image correction means for correcting distortion of the input image according to the correction parameters calculated by the parameter decoding means;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding means performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction.

19. The image processing system according to claim 18, wherein the parameter decoding means comprises:

interpolation calculation means for calculating interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

Y-X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a zooming direction by the interpolation calculation means and needed for distortion correction of data of one line;

X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a vertical direction by the interpolation calculation means and needed for distortion correction of data of one line;

input selection means for selectively inputting to the interpolation calculation means the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding means, and the correction parameters read from the X-compressed data holding means; and

control means for controlling a selecting operation in the input selection means according to the coordinates designated by the coordinate designating means, and to the zoom position acquired by the zoom position acquiring means, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation means.

20. An imaging apparatus for imaging an image by using a converging lens group having an optical zoom mechanism, the imaging apparatus comprising:

zoom position detecting means for detecting a zoom position indicating a lens position condition of the optical zoom mechanism on taking an input image;

zooming-direction decoding means for decompressing correction parameters, which corresponds to the zoom position detected by the zoom position detecting means, by interpolation calculation according to the zoom compression parameter obtained by compressing the correction parameters for distortion correction, which correspond to each of a lens position condition, through utilization of zoom petitioning points at which lens position conditions in the optical zoom mechanism are serially partitioned between a wide-end and a tele-end into plural levels including both the ends; and

image correction means for correcting distortion of the imaged image according to the correction parameters decompressed by the zooming-direction decoding means.

21. The imaging apparatus according to claim 20, further comprising:

parameter storing means having two storage areas for storing the zoom compression parameters;

parameter selecting means for selecting the zoom compression parameter, which is needed for the interpolation calculation by the zooming-direction decoding means, according to change of the zoom position;

storage area selecting means for selecting the storage area that stores the zoom compression parameter selected by the parameter selecting means; and

phase computing means for calculating a relative position of the zoom position between the zoom partitioning points corresponding the zoom compression parameter, which is selected by the parameter selecting means, as a phase coefficient;

wherein the zooming-direction decoding means performs interpolation calculation according to the zoom compression parameter read from the storage area and to the phase coefficient calculated by the phase computing means.

22. An imaging apparatus for imaging an image by using a converging lens group having an optical zoom mechanism, the imaging apparatus comprising:

zoom position detecting means for detecting a zoom position indicating a lens position condition of the optical zoom mechanism on taking an input image;

coordinate designating means for designating coordinates of a pixel, which is an object for distortion correction;

parameter decoding means for calculating correction parameters corresponding to the coordinates designated by the coordinate designating means and to the zoom position acquired by the zoom position acquiring means according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected, by interpolation calculation; and

image correction means for correcting distortion of the input image according to the correction parameters calculated by the parameter decoding means;

wherein, when a correction process in the horizontal direction is performed on data of one line, the parameter decoding means performs interpolation calculation on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and performs, in an image effective period after the termination of this horizontal retrace period, interpolation calculation on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction.

23. The imaging apparatus according to claim 22,

wherein the parameter decoding means comprises:

interpolation calculation means for calculating interpolation calculation according to a phase coefficient by using the inputted and compressed correction parameters;

Y-X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a zooming direction by the interpolation calculation means and needed for distortion correction of data of one line;

X-compressed data holding means for temporarily holding the correction parameters, which are interpolated in a vertical direction by the interpolation calculation means and needed for distortion correction of data of one line;

input selection means for selectively inputting to the interpolation calculation means the correction parameters that are compressed in the horizontal direction, the vertical direction and the zooming direction and correspond to the zoom position, the correction parameters read from the Y-X-compressed data holding means, and the correction parameters read from the X-compressed data holding means; and

control means for controlling a selecting operation in the input selection means according to the coordinates designated by the coordinate designating means, and to the zoom position acquired by the zoom position acquiring means, for calculating a phase coefficient, and for supplying the calculated phase coefficient to the interpolation calculation means.

24. An image processing method for receiving an inputted image, which is taken by using a converging lens group having an optical zoom mechanism, and for correcting distortion of the image, the image processing method comprising the steps of:

acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an inputted image;

decompressing the correction parameter associated to the zoom position according to a zoom compression parameter, into which a correction parameter for distortion correction corresponding to each lens position condition is compressed by utilizing zoom partitioning points at which lens position conditions of the optical zoom mechanism are sequentially partitioned into plural levels that are set between a wide-end and a tele-end and that include both the ends; and

correcting distortion of the inputted image according to the decompressed correction parameter.

25. The image processing method according to claim 24, further comprising the steps of:

selecting the zoom compression parameter that is necessary for interpolation calculation in a zooming direction according to change of the zoom position;

storing the selected zoom compression parameter in individual storage areas;

calculating a relative position of the zoom position in an interval between the zoom partitioning points corresponding the selected zoom compression parameter as a phase coefficient; and

performing interpolation calculation in the zooming direction according to the zoom compression parameter read from the storage areas and to the calculated phase coefficient.

26. The image processing method according to claim 25, wherein:

if the zoom position changes, the zoom compression parameter corresponding to the zoom partitioning point, which is close to the changed zoom position, is continuously selected among the zoom portioning points corresponding to selection before the change, and only the zoom compression parameter corresponding to the other zoom partitioning point is newly selected in a case where the changed zoom position is present outside a region between the zoom partitioning points corresponding to the zoom compression parameter selected before the change; and

after the zoom position changes, only the newly selected zoom compression parameter is written to the storage area, without newly writing data to the storage area in which the zoom compression parameter continuously selected is stored.

27. An image processing method for receiving an input image taken by using a converging lens group having an optical zoom mechanism and for correcting distortion of the image, the image processing method comprising:

a first step of acquiring a zoom position representing a lens position condition of the optical zoom mechanism on taking an input image;

a second step of calculating the correction parameters corresponding to coordinates designating a pixel, which is an object for distortion correction, and to the acquired zoom position by interpolation calculation according to correction parameters for distortion correction, compressed in a zooming direction, in which the lens position condition changes in the optical zoom mechanism, and in a horizontal direction and a vertical direction of the image whose distortion has been corrected; and

a third step of correcting distortion of the input image according to the calculated correction parameters;

wherein in the second step, if a correction process in the horizontal direction is performed on data of one line, the interpolation calculation is performed on the correction parameters, which are needed for performing the process on the data of one line and correspond to the zooming direction and the vertical direction, according to the zoom position and vertical coordinates corresponding to the one line in a horizontal retrace period, and that in an image effective period after the termination of this horizontal retrace period, interpolation calculation is performed on each of pixels arranged in a horizontal direction by using the correction parameters calculated and compressed in the horizontal direction.

28. The image processing method according to claim 27, wherein:

in the second step, in the horizontal retrace period, interpolation calculation corresponding to the zooming direction is performed by using the correction parameter, which is compressed in the horizontal direction, the vertical direction, and the zooming direction, and the phase coefficient corresponding to the zoom position, and the calculated correction parameters are temporarily stored in a first storage area;

subsequently, interpolation calculation corresponding to the vertical direction is performed by using the correction parameter read from the first storage area, and the phase coefficient obtained by the vertical coordinate corresponding to the one line, and the calculated correction parameter is temporarily stored in a second storage area; and

in the image effective period after termination of the horizontal retrace period, interpolation calculation corresponding to the horizontal direction is performed by using the correction parameter read from the second storage area, and further using the phase coefficient obtained from each of the coordinates in the horizontal direction.