Image sequence enhancement system and method

• US 20040131249A1
• Filed: 06/18/2003
• Published: 07/08/2004
• Est. Priority Date: 05/04/2001
• Status: Active Grant

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend Films Incorporated (Legend3D, Inc.)
• Inventors: Sandrew, Barry
• Granted Patent: US 7,181,081 B2
• Time in Patent Office: Days
• US Class Current: 382/162
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/37 using transform domain methods
• CPC Class Codes: H04N 9/43 Conversion of monochrome picture signals to colour picture signals for colour picture display

Abstract

Scenes in a motion picture to be colorized are broken up into separate elements, composed of backgrounds/sets or motion/onscreen-action. These backgrounds elements and motion elements are combined separately into single frame representations of multiple frames, as tiled frame sets or as a single frame composite of all elements (i.e., both motion and backgrounds/foregrounds) that then becomes a visual reference database that includes data for all frame offsets which are later used for the computer controlled application of masks within a sequence composed of a plurality of frames. Each pixel address within the visual reference database corresponds to a mask/lookup table address within the digital frame and X, Y, Z location of subsequent frames that were used to create the visual reference database. Masks are applied to subsequent frames of motion objects based on various differentiating image processing methods, including automated mask fitting of all masks in an entire frame, automated mask fitting of single masks in each frame, bezier and polygon tracing of selected regions aided by edge detected shaping and operator directed detection of subsequent regions corresponding to the original region of interest. In this manner, the gray scale actively determines the mask and corresponding color lookup from frame to frame that is applied in a keying fashion within regions of interest.

Claims

1. ) A method of enhancing an image comprising the steps of:

a) providing a series of image frames including at least a reference image frame and a subsequent image frame, each of the image frames including a region having a known feature;

b) providing for the region of at least each of the reference image frame and the subsequent image frame a mask that defines at least in part its related region, the mask of the subsequent image frame being displaced from its known feature in a manner differing from the displacement of the mask of the reference image frame from its related region, each of the masks comprising a plurality of pixels;

c) measuring the luminance value of selected pixels of each the reference image frame and the subsequent image frame; and

d) adjusting the displacement of the subsequent frame with respect to its feature and determining the difference between the luminance values of the pixels of the reference image frame with the luminance values of the corresponding pixels of the subsequent image frame to determine the displacement between the subsequent frame and its feature when the difference has been minimized.

2. ) A method of enhancing an image comprising the steps of:

a) providing a series of sequential image frames including at least a reference image frame and at least one subsequent image frame, each of the image frames having a similar or corresponding region with similar features;

b) identifying by a mask indicating the extent of at lest one region of each of the reference image frame and the subsequent image frame being displaced from its known feature relative to the mask of the reference image frame from its related region, each of the masks and related features comprising a plurality of pixels;

c) measuring the pattern of luminance values of selected feature pixels of each the reference image frame and the subsequent image frame; and

d) adjusting the displacement of the feature of in the subsequent frame with respect to its mask by determining the difference between the pattern of luminance values of the pixels underlying the mask in the displaced image frame relative to the pattern of luminance values of the reference image frame to determine the displacement between the subsequent frame and its feature when the difference has been minimized.