Enhancing the resolution and quality of sequential digital images
First Claim
1. A method, comprising:
- receiving as input a sequence of two or more low resolution video frames, wherein one of the low resolution video frames is a current frame;
calculating a global transformation, relative to the current frame, for the sequence of video frames;
determining whether unexplained relative motion remains following calculation of the global transformation;
responsive to identifying the unexplained relative motion, calculating an independent object motion, relative to the current frame, for the sequence of video frames;
determining whether unexplained relative motion remains following calculation of the global transformation and any independent object motion;
responsive to identifying the unexplained relative motion, calculating an optical flow, relative to the current frame, for the sequence of video frames;
creating pixel motion trajectories, relative to the current frame, for the sequence of video frames, based on the calculated global transformation, any calculated independent object motion and any calculated optical flow;
interpolating an enhanced resolution frame for the current frame from the pixel motion trajectories using spatio-temporal interpolation; and
outputting the enhance resolution frame.
4 Assignments
0 Petitions
Accused Products
Abstract
In particular embodiments, a process relating to the creation of a high-resolution video from a low-resolution video. In a particular embodiment, the process receives as input a sequence of low-resolution video frames. The process first determines a matching score for consecutive frames in the sequence, where the matching score is based on a preliminary global transformation between consecutive frames. From the matching scores, the process determines a set of matching windows and relative-motion estimates and then uses the set and estimates to calculate more thorough global transformations and any residual relative-motion which can be explained using independent object motion and/or optical flow. The process uses the latter global transformation and any independent object motion and/or optical flow to create motion trajectories and to generate high-resolution frames by interpolating low-resolution frames at trajectory-defined points, using regular-to-irregular spatio-temporal interpolation. The process then restores the frames, using deblurring, and outputs them.
-
Citations
17 Claims
-
1. A method, comprising:
-
receiving as input a sequence of two or more low resolution video frames, wherein one of the low resolution video frames is a current frame; calculating a global transformation, relative to the current frame, for the sequence of video frames; determining whether unexplained relative motion remains following calculation of the global transformation; responsive to identifying the unexplained relative motion, calculating an independent object motion, relative to the current frame, for the sequence of video frames; determining whether unexplained relative motion remains following calculation of the global transformation and any independent object motion; responsive to identifying the unexplained relative motion, calculating an optical flow, relative to the current frame, for the sequence of video frames; creating pixel motion trajectories, relative to the current frame, for the sequence of video frames, based on the calculated global transformation, any calculated independent object motion and any calculated optical flow; interpolating an enhanced resolution frame for the current frame from the pixel motion trajectories using spatio-temporal interpolation; and outputting the enhance resolution frame. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. One or more non-transitory computer-readable media comprising computer-readable instructions for execution and when executed operable to cause one or more processors to:
-
receive as input a sequence of two or more low resolution video frames, wherein one of the low resolution video frames is a current frame; calculate a global transformation, relative to the current frame, for the sequence of video frames; determine whether unexplained relative motion remains following calculation of the global transformation; responsive to identifying the unexplained relative motion, calculate an independent object motion, relative to the current frame, for the sequence of video frames; determine whether unexplained relative motion remains following calculation of the global transformation and any independent object motion; responsive to identifying the unexplained relative motion, calculate an optical flow, relative to the current frame, for the sequence of video frames; create pixel motion trajectories, relative to the current frame, for the sequence of video frames, based on the calculated global transformation, any calculated independent object motion and any calculated optical flow; interpolate an enhanced resolution frame for the current frame from the pixel motion trajectories, using spatio-temporal interpolation; and output the enhanced resolution video frame. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. An apparatus, comprising:
-
a memory; one or more processors; and a non-transitory computer readable medium comprising computer readable instructions encoded thereon, the computer-readable instructions operative, when executed, to cause the one or more processors to; display an enhanced resolution video, one or more frames of the enhanced resolution video output from an enhanced resolution module comprising; one or more processors, and logic encoded in one or more computer-readable media for execution and when executed operable to; receive as input a sequence of two or more low resolution video frames; create pixel motion trajectories for the sequence of video frames by registering the frames; perform regular-to-irregular spatial interpolation on the registered frames, wherein the results of this spatial interpolation are noisy observations; perform temporal approximation on the noisy observations, wherein the results of this temporal approximation are high resolution video frames; and output a sequence of enhanced resolution video frames.
-
Specification