Image signal processor and method for synthesizing super-resolution images from non-linear distorted images
First Claim
1. A video signal processing apparatus, comprising:
- a first calculator that receives a plurality of original input image frames, each of which having a different non-linear distortion, and calculates first coordinate conversion formulas, corresponding to each of the original input image frames, correcting non-linear distortion corresponding to a cropped region, capturing an object of each of the frame images;
a distortion correction and interpolation processor that generates a plurality of distortion corrected images by performing distortion correction on the cropped region of each of the original input frame images according to each of the first coordinate conversion formulas and generates a plurality of interpolation processed images by performing interpolation processing on each of the distortion corrected images;
a second calculator that calculates second coordinate conversion formulas correcting the interpolation processed images so that a position of the object captured in each of the interpolation processed images matches each other;
an image coordinate conversion formula calculator that calculates integrated coordinate conversion formulas by integrating the first coordinate conversion formulas and the second coordinate conversion formulas;
a pixel coordinate converter that generates pixel rearranged images by performing rearrangement of pixel coordinates on at least the cropped region of each of the original input frame images using the integrated coordinate conversion formula corresponding to each of the original input image frames; and
a synthesizer that generates a synthesized image by synthesizing the rearranged images.
3 Assignments
0 Petitions
Accused Products
Abstract
An image signal processing apparatus and image signal processing method obtains a high super-resolution effect for distorted images captured by a specialized optical system, such as a 360-degree camera. A distortion-correction conversion-formula calculation section calculates a distortion correction conversion formula for correcting the nonlinear distortion. A distortion correction and interpolation processing section performs distortion correction and interpolation processing on each cropped region. An alignment conversion formula calculation section calculates an alignment conversion formula for alignment between each cropped image after distortion correction and interpolation processing is completed. A pixel coordinate conversion section rearranges pixel coordinates for each frame image before interpolation processing by using the distortion correction conversion formula and the alignment conversion formula corresponding to the cropped regions. An image synthesis section synthesizes the cropped images.
-
Citations
13 Claims
-
1. A video signal processing apparatus, comprising:
-
a first calculator that receives a plurality of original input image frames, each of which having a different non-linear distortion, and calculates first coordinate conversion formulas, corresponding to each of the original input image frames, correcting non-linear distortion corresponding to a cropped region, capturing an object of each of the frame images; a distortion correction and interpolation processor that generates a plurality of distortion corrected images by performing distortion correction on the cropped region of each of the original input frame images according to each of the first coordinate conversion formulas and generates a plurality of interpolation processed images by performing interpolation processing on each of the distortion corrected images; a second calculator that calculates second coordinate conversion formulas correcting the interpolation processed images so that a position of the object captured in each of the interpolation processed images matches each other; an image coordinate conversion formula calculator that calculates integrated coordinate conversion formulas by integrating the first coordinate conversion formulas and the second coordinate conversion formulas; a pixel coordinate converter that generates pixel rearranged images by performing rearrangement of pixel coordinates on at least the cropped region of each of the original input frame images using the integrated coordinate conversion formula corresponding to each of the original input image frames; and a synthesizer that generates a synthesized image by synthesizing the rearranged images. - View Dependent Claims (2, 3, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
4. A video signal processing method, comprising:
-
receiving a plurality of original input image frames, each having a different non-linear distortion; calculating first coordinate formulas, corresponding to each of the original input image frames, correcting non-linear distortion corresponding to a cropped region, capturing an object, of each of the frame images; generating a plurality of distortion corrected images by performing distortion correction on the cropped region of each of the original input frame images according to each of the first coordinate conversion formulas, and generating a plurality of interpolation processed images by performing interpolation processing on each of the distortion corrected images; calculating second coordinate conversion formulas correcting the interpolation processed images so that pixel positions in the object captured correspond with pixel positions in each of the interpolation processed images; calculating integrated coordinate conversion formulas by integrating the first coordinate conversion formulas and the second coordinate conversion formulas; generating pixel rearranged images by performing rearrangement of pixel coordinates on at least the cropped region of each of the original input frame images using the integrated coordinate conversion formula corresponding to each of the original input image frames; and generating a synthesized image by synthesizing the rearranged images.
-
-
13. A video signal processing apparatus, comprising:
-
a first calculator that receives a plurality of original input image frames, each of which having a different non-linear distortion, and calculates first coordinate conversion formulas, corresponding to each of the original input image frames, correcting non-linear distortion corresponding to a cropped region, capturing an object of each of the frame images; a distortion correction and interpolation processor that generates a plurality of distortion corrected images by performing distortion correction on the cropped region of each of the original input frame images according to each of the first coordinate conversion formulas and generates a plurality of interpolation processed images by performing interpolation processing on each of the distortion corrected images; a second calculator that calculates second coordinate conversion formulas correcting the interpolation processed images so that a position of the object captured in each of the interpolation processed images matches each other; a pixel coordinate converter that generates pixel rearranged images by performing rearrangement of pixel coordinates on at least the cropped region of each of the original input frame images using the first coordinate conversion formula for correcting each non-linear distortion of the original input image frames and the second coordinate conversion formula for correcting the original input image frames so that a position of the object captured in each of the original input image frames matches each other; and a synthesizer that generates a synthesized image by synthesizing the rearranged images.
-
Specification