IMAGE SIGNAL PROCESSOR AND IMAGE SIGNAL PROCESSING METHOD

• US 20130258048A1
• Filed: 12/20/2011
• Published: 10/03/2013
• Est. Priority Date: 12/27/2010
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Tabei, Kenji, Wang, Qinhe
• Granted Patent: US 9,609,181 B2
• Time in Patent Office: Days
• US Class Current: 348/38
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/407 Control or modification of tonal gradation or of extreme levels, e.g. background level
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An image signal processing apparatus and image signal processing method capable of obtaining a high super-resolution effect even for distorted images captured by a specialized optical system such as a 360-degree camera. Distortion-correction conversion-formula calculation section (123) calculates a distortion correction conversion formula for correcting the nonlinear distortion. Distortion correction and interpolation processing section (101) performs distortion correction and interpolation processing on each cropped region. Alignment conversion formula calculation section (104) calculates an alignment conversion formula for alignment between each cropped image after distortion correction and interpolation processing is completed. Pixel coordinate conversion section (111) rearranges pixel coordinates for each frame image before interpolation processing by using the distortion correction conversion formula and the alignment conversion formula corresponding to the cropped regions. Image synthesis section (112) synthesizes the cropped images, in which a plurality of frames have been rearranged, and which are obtained using the pixel coordinate conversion section (111).

Claims

1. A video signal processing apparatus that uses a plurality of frame images differing in non-linear distortion according to a cropped position and thereby obtains a super-resolution image, the apparatus comprising:

a first coordinate conversion formula calculation section that obtains, for each frame, a first coordinate conversion formula for correcting non-linear distortion in a cropped region of each of the frame images;

a distortion correction and interpolation processing section that performs distortion correction on the cropped region of each of the frame images by using the first coordinate conversion formula corresponding to the cropped region and performs interpolation processing on each of the cropped images on which the distortion correction has been performed;

a second coordinate conversion formula calculation section that obtains a second coordinate conversion formula for alignment between cropped images on a basis of results of matching between cropped images by using each of the cropped images on which the distortion correction and interpolation processing have been performed;

a pixel coordinate conversion section that performs rearrangement of pixel coordinates on each of the frame images on which the distortion correction and interpolation processing are not performed, by using the first coordinate conversion formula corresponding to the cropped region of the frame image and the second coordinate conversion formula corresponding to the cropped image; and

a synthesis section that synthesizes the cropped images in each of which the rearrangement has been made, the cropped images corresponding to the plurality of frames and being obtained by the pixel coordinate conversion section.

2. The video signal processing apparatus according to claim 1, further comprising an interpolation processing section that performs interpolation processing on a synthesized image obtained by the synthesis section.

3. A camera system comprising:

a camera that photographs an image having non-linear distortion; and

the video signal processing apparatus according to claim 1 that performs processing on the photographed image.

5. A camera system comprising:

a camera that photographs an image having non-linear distortion; and

the video signal processing apparatus according to claim 2 that performs processing on the photographed image.

4. A video signal processing method of using a plurality of frame images differing in non-linear distortion according to a cropped position and thereby obtaining a super-resolution image, the method comprising:

a first coordinate conversion formula calculation step of obtaining, for each frame, a first coordinate conversion formula for correcting non-linear distortion in a cropped region of each of the frame images;

a distortion correction and interpolation processing step of performing distortion correction on the cropped region of each of the frame images by using the first coordinate conversion formula corresponding to the cropped region and performing interpolation processing on each of the cropped images on which the distortion correction has been performed;

a second coordinate conversion formula calculation step of obtaining a second coordinate conversion formula for alignment between cropped images on a basis of results of matching between cropped images by using each of the cropped images on which the distortion correction and interpolation processing have been performed;

a pixel coordinate conversion step of performing rearrangement of pixel coordinates on each of the frame images on which the distortion correction and interpolation processing are not performed, by using the first coordinate conversion formula corresponding to the cropped region of the frame image and the second coordinate conversion formula corresponding to the cropped image; and

a synthesis step of synthesizing the cropped images in each of which the rearrangement has been made, the cropped images corresponding to the plurality of frames and being obtained in the pixel coordinate conversion step.