Method and apparatus for enhancing resolution of video image

• US 20090022226A1
• Filed: 07/18/2007
• Published: 01/22/2009
• Est. Priority Date: 07/18/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd., Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Original Assignee: Samsung Electronics Co. Ltd., Purdue Research Foundation (The Trustees of Purdue University (d/b/a Purdue University))
• Inventors: Bang, Yoo-sun, Zhu, Fengqing, Delp, Edward J., Lee, Ho-young, Choh, Heui-keun
• Granted Patent: US 8,331,451 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation

Abstract

Provided is a technology which can prevent deterioration of image quality when enhancing resolution of a predetermined key frame in a video sequence. Specifically, an apparatus to enhance resolution of a video frame is provided. The apparatus includes a frame extraction unit which extracts a key frame and one or more neighboring frames of the key frame from a video sequence; an upsampling unit which upsamples the key frame and the neighboring frames; a motion-vector search unit which calculates a motion vector of the upsampled key frame using the upsampled neighboring frames as reference frames; and a key-frame estimation unit which enhances quality of the upsampled key frame using temporal information obtained from the motion vector and spatial information in the key frame.

Claims

1. An apparatus to enhance resolution of video image, the apparatus comprising:

a frame extraction unit to extract a key frame and one or more neighboring frames of the key frame from a video sequence;

an upsampling unit to upsample the key frame and the neighboring frames;

a motion-vector search unit to calculate a motion vector of the upsampled key frame using the upsampled neighboring frames as reference frames; and

a key-frame estimation unit to enhance quality of the upsampled key frame using temporal information obtained from the motion vector and spatial information in the key frame.

2. The apparatus of claim 1, wherein the neighboring frames comprise one or more frames temporally adjacent to the key frame.

3. The apparatus of claim 1, wherein the key-frame estimation unit applies statistical estimation to the temporal information and the spatial information.

4. The apparatus of claim 3, wherein the statistical estimation is maximum a posteriori (MAP) estimation.

5. The apparatus of claim 4, wherein the key-frame estimation unit comprises:

a prior-image modeling unit to model the probability distribution of the upsampled key frame;

a conditional-density modeling unit to model probability distributions of the neighboring frames using the upsampled key frame as a constraint; and

an optimal-solution calculation unit to obtain an optimally upsampled key frame that minimizes the sum of the probability distribution of the key frame and those of the neighboring frames.

6. The apparatus of claim 5, wherein the prior-image modeling unit models the probability distribution of the key frame using spatial activities in the key frame and temporal activities between the key frame and the neighboring frames as constraints.

7. The apparatus of claim 5, wherein the conditional-density modeling unit models the probability distributions of the neighboring frames based on motion compensation errors between the key frame and the neighboring frames.

8. The apparatus of claim 5, wherein the optimal-solution calculation unit obtains the optimally upsampled key frame using a gradient optimization method.

9. The apparatus of claim 1, wherein the upsampling unit upsamples the key frame and the neighboring frames using at least one of bi-linear, bi-cubic, and zero-order-hold interpolation methods.

10. The apparatus of claim 1, wherein the motion-vector search unit calculates the motion vector using block matching.

11. A method of enhancing resolution of video image, the method, comprising:

extracting a key frame and one or more neighboring frames of the key frame from a video sequence;

upsampling the key frame and the neighboring frames;

calculating a motion vector of the upsampled key frame using the upsampled neighboring frames as reference frames; and

enhancing the quality of the upsampled key frame using temporal information obtained from the motion vector and spatial information in the key frame.

12. The method of claim 11, wherein the neighboring frames comprise one or more frames temporally adjacent to the key frame.

13. The method of claim 11, wherein the enhancing quality of the upsampled key frame comprises applying statistical estimation to the temporal information and the spatial information.

14. The method of claim 13, wherein the statistical estimation is MAP (maximum a posteriori) estimation.

15. The method of claim 14, wherein the enhancing quality of the upsampled key frame comprises:

modeling probability distribution of the upsampled key frame;

modeling probability distributions of the neighboring frames using the upsampled key frame as a constraint; and

obtaining an optimally upsampled key frame which minimizes the sum of the probability distribution of the key frame and those of the neighboring frames.

16. The method of claim 15, wherein the modeling of the probability distribution of the upsampled key frame comprises modeling the probability distribution of the key frame using spatial activities in the key frame and temporal activities between the key frame and the neighboring frames as constraints.

17. The method of claim 15, wherein the modeling of probability distributions of the neighboring frames comprises modeling the probability distributions of the neighboring frames based on motion compensation errors between the key frame and the neighboring frames.

18. The method of claim 15, wherein the obtaining of an optimally upsampled key frame comprises obtaining the optimally upsampled key frame using a gradient optimization method.

19. The method of claim 11, wherein the upsampling of the key frame and the neighboring frames comprises upsampling the key frame and the neighboring frames using at least one of bi-linear, bi-cubic, and zero-order-hold interpolation methods.

20. The method of claim 11, wherein the calculating of a motion vector comprises calculating the motion vector using block matching.

21. A computer-readable recording medium for controlling a computer on which a program to execute the method of claim 11 is recorded.

22. A method, comprising:

inputting a video sequence; and

enhancing a key frame of the video sequences using a multiframe based maximum, a posterior estimation technique based on a Bayesian interpolation method used for still images.

23. A method, comprising:

inputting a video sequence; and

enhancing quality of an upsampled key frame of the sequence using a sequence key frame motion vector and key frame spatial information.

24. An apparatus, comprising:

a computer enhancing quality of an upsampled key frame of the sequence using a sequence key frame motion vector and key frame spatial information; and

a display displaying the upsampled keyframe.