High resolution image reconstruction

• US 20050013509A1
• Filed: 07/16/2003
• Published: 01/20/2005
• Est. Priority Date: 07/16/2003
• Status: Active Grant

• Current Assignee: Dolby International AB (Dolby Laboratories Incorporated)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Samadani, Ramin
• Granted Patent: US 7,596,284 B2
• Time in Patent Office: Days
• US Class Current: 382/302
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4023 based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels

Abstract

A technique of reconstructing a high resolution image from at least one image sequence of temporally related high and low resolution image frames wherein each of said high resolution image frames includes a low spatial frequency component and a high spatial frequency component is described. The high-resolution image reconstruction technique uses spatial interpolation to generate a low spatial frequency component from a low-resolution image frame of the image sequence. The technique is adapted to generate a high spatial frequency component from at least one high resolution image frame of the image sequence which is closely related to the low resolution image frame, and to remap the high spatial frequency component to a motion-compensated high spatial frequency component estimate of the low resolution image frame. The motion-compensated high spatial frequency component estimate is added to the generated low spatial frequency component to form a reconstructed high-resolution image of the low-resolution image frame.

Claims

1. A system for reconstructing a high resolution image from at least one image sequence of temporally related high and low resolution image frames, each of said high resolution image frames including a low spatial frequency component and a high spatial frequency component, said system comprising:

a first spatial interpolator adapted to generate a low spatial frequency component from a low resolution image frame of said at least one image sequence;

a high spatial frequency component generator for generating a high spatial frequency component from at least one high resolution image frame of said at least one image sequence, said at least one high resolution image frame being closely related to said low resolution image frame;

a remapper for mapping said high spatial frequency component to a motion-compensated high spatial frequency component estimate of said low resolution image frame; and

an adder for adding said motion-compensated high spatial frequency component estimate of said low resolution image frame to said generated low spatial frequency component of said low resolution image frame to form a reconstructed high resolution image of said low resolution image frame.

2. The high resolution image reconstruction system of claim 1, further comprising a controller said adding of said motion-compensated high spatial frequency component estimate of said low resolution image frame to said generated low spatial frequency component of said low resolution image frame to optimize motion confidence.

3. The high resolution image reconstruction system of claim 1, wherein said first spatial interpolator utilizes a bicubic upsampling algorithm.

4. The high resolution image reconstruction system of claim 1, wherein said first spatial interpolator utilizes a bilinear upsampling algorithm.

5. The high resolution image reconstruction system of claim 1, wherein said first spatial interpolator utilizes a least squares error minimization algorithm.

6. The high resolution image reconstruction system of claim 1, wherein said high spatial frequency component generator includes a downsampler for downsampling at least one high resolution image frame of said at least one image sequence.

7. The high resolution image reconstruction system of claim 6, wherein said high spatial frequency component generator further includes a subpixel motion processor for generating a motion vector field and a confidence scalar field from said downsampled high resolution image frame and said low resolution image frame of said at least one image sequence.

8. The high resolution image reconstruction system of claim 7, wherein said high spatial frequency component generator further includes a second spatial interpolator adapted to generate a low spatial frequency component from said downsampled high resolution image frame.

9. The high resolution image reconstruction system of claim 8, wherein said high spatial frequency component generator further includes a subtractor for subtracting said generated low spatial frequency component from said at least one high resolution image frame of said at least one image sequence to obtain a high spatial frequency component of said at least one high resolution image frame of said at least one image sequence.

10. A method of reconstructing a high-resolution image from at least one image sequence of temporally related high and low resolution image frames, each of said high-resolution image frames including a low spatial frequency component and a high spatial frequency component comprising:

spatially interpolating to generate a low spatial frequency component from a low-resolution image frame of said at least one image sequence;

generating a high spatial frequency component from at least one high resolution image frame of said at least one image sequence, said at least one high resolution image frame being closely related to said low resolution image frame;

remapping said high spatial frequency component to a motion-compensated high spatial frequency component estimate of said low resolution image frame; and

adding said motion-compensated high spatial frequency component estimate of said low resolution image frame to said generated low spatial frequency component of said low resolution image frame to form a reconstructed high resolution image of said low resolution image frame.

11. The method of claim 10, further comprising controlling said adding of said motion-compensated high spatial frequency component estimate of said low resolution image frame to said generated low spatial frequency component of said low resolution image frame to optimize motion confidence.

12. The method of claim 10, wherein said spatially interpolating is performed by bicubic upsampling.

13. The method of claim 10, wherein said spatially interpolating is performed by bilinear upsampling algorithm.

14. The method of claim 10, wherein said spatially interpolating is performed by utilizes a least squares error minimization algorithm.

15. The method of claim 10, wherein said high spatial frequency component generating further comprises downsampling at least one high resolution image frame of said at least one image sequence.

16. The method of claim 15, wherein said high spatial frequency component generating further comprises subpixel motion processing for the purpose of generating a motion vector field and a confidence scalar field from said downsampled high resolution image frame and said low resolution image frame of said at least one image sequence.

17. The method of claim 16, wherein said high spatial frequency component generating further comprises spatially interpolating to generate a low spatial frequency component from said downsampled high-resolution image frame.

18. The method of claim 17, wherein said high spatial frequency component generating further comprises subtracting said generated low spatial frequency component from said at least one high resolution image frame of said at least one image sequence to obtain a high spatial frequency component of said at least one high resolution image frame of said at least one image sequence.