VIDEO PROCESSING SYSTEM AND METHOD FOR AUTOMATIC ENHANCEMENT OF DIGITAL VIDEO

• US 20060187359A1
• Filed: 04/28/2006
• Published: 08/24/2006
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Soupliotis, Andreas, Anandan, Padmanabhan
• Granted Patent: US 7,167,199 B2
• Time in Patent Office: Days
• US Class Current: 348/700
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229

Abstract

An automatic video enhancement system and method for automatically enhancing video. The automated video enhancement method uses frame-to-frame motion estimation as the basis of the video enhancement. Motion estimation includes the computation of global motion (such as camera motion) and the computation of local motion (such as pixel motion). The automated video enhancement method includes generating global alignment transforms, generating optic flow vectors, and using these global alignment transforms and optic flow vectors to enhance the video. The invention also includes video processing and enhancement techniques that use the frame-to-frame motion estimation. These techniques include a deinterlace process, a denoise process, and a warp stabilization process that performs both damped and locked stabilization.

Claims

1. A method for performing locked stabilization on a video frame sequence captured by a camera, comprising:

selecting a video frame from the video frame sequence to use as a reference frame;

computing frame pairwise global motion between each of the video frames in the video frame sequence;

calculating an affine transformation from each video frame in the video frame sequence to the referenced frame; and

removing pixels in a video frame discovered by the affine transformation.

2. The method as set forth in claim 1, further comprising replacing the removed pixels in a video frame by building a mosaic.

3. The method as set forth in claim 2, wherein building a mosaic further comprises borrowing pixels that are similar to the removed pixels from frames that neighbor the video frame containing the removed pixels.