Computer-implemented digital video object compositing

• US 7,091,993 B1
• Filed: 01/24/2003
• Issued: 08/15/2006
• Est. Priority Date: 10/15/1997
• Status: Expired due to Term

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: Planet Electric Incorporated
• Inventors: Ahmad, Subutai
• Primary Examiner(s): Chauhan, Ulka
• Assistant Examiner(s): WOODS, ERIC V
• Application Number:
  US10/351,600
• Time in Patent Office: 1,299 Days
• Field of Search: 345619-641
• US Class Current: 345/629
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A system and method are disclosed for digitally compositing an object from an input image onto a destination image. The object is composited from an image having an arbitrary or non-uniform colored background containing some non-static elements onto a destination image with reduced effects from shadows cast by the object and with reduced gaps or holes within the object. Various improvements in the compositing procedure such as shadow reduction and hole filling, and less restrictive requirements regarding the object'"'"'s surroundings are disclosed. A background model is created and a frame of an input image containing the object is obtained. An alpha image is created in which each pixel is either a zero, indicating it is not part of the object, or a one, indicating that it is part of the object. The effect of shadows emanating from the object is reduced so that the composited object in the destination image contains only the object clearly outlined by the object'"'"'s physical boundaries without the effect of shadows cast by the object. A set of templates is then derived in which each template fits entirely within a designated part of the object. The templates allow holes or gaps in the object created during the compositing process to be filled to a large extent. The object is blended onto a destination image using the alpha image as a blending coefficient.

Claims

1. A computer implemented method of filling holes in a digital video object being composited comprising:

assigning to each column in an alpha image an index, wherein the columns in the alpha image are comprised of alpha pixels having values, and wherein the alpha image corresponds to an image that includes the digital video object being composited;

determining the values of the alpha pixels;

calculating for each index an index value that is based at least in part on the values of the alpha pixels comprising the column with which the index is associated;

determining indices of boundaries and sections of the object, thereby identifying the shape of the object;

computing a set of a plurality of templates using the indices of the boundaries and sections of the object such that the set of templates comprises the shape of the object wherein the set of templates lies entirely within the portion of the object it represents including all boundary pixels; and

adjusting one or more portions of the alpha image based at least in part on the set of templates such that the one or more portions of the alpha image that correspond to image locations that lie within the any template of the set of templates are within the set of templates are processed as being part of the object so that no holes appear in the image in image locations that correspond to alpha image locations that lie within any template of the set of templates.

2. A computer implemented method as recited in claim 1 further including computing a set of bounding rectangles such that portions of the object fit within the set of bounding rectangles and such that the set of bounding rectangles is computed using the indices of the boundaries and sections of the object.

3. A computer program product for filling holes in a digital video object being composited, the computer program product being embodied in a computer readable medium and comprising computer instructions for:

assigning to each column in an alpha image an index, wherein the columns in the alpha image are comprised of alpha pixels having values, and wherein the alpha image corresponds to an image that includes the digital video object being composited;

determining the values of the alpha pixels;

calculating for each index an index value that is based at least in part on the values of the alpha pixels comprising the column with which the index is associated;

determining indices of boundaries and sections of the object, thereby identifying the shape of the object;

computing a set of a plurality of templates using the indices of the boundaries and sections of the object such that the set of templates comprises the shape of the object wherein the set of templates lies entirely within the portion of the object it represents including all boundary pixels; and

adjusting one or more portions of the alpha image based at least in part on the set of templates such that the one or more portions of the alpha image that correspond to image locations that lie within the any template of the set of templates are processed as being part of the object so that no holes appear in the image in image locations that correspond to alpha image locations that lie within any template of the set of templates.

8. A computer program product as recited in claim 3, the computer program product further comprising computer instructions for computing a set of bounding rectangles such that portions of the object fit within the set of bounding rectangles and such that the set of bounding rectangles is computed using the indices of the boundaries and sections of the object.

4. A computer implemented system for filling holes in an object digitally composited onto a destination image comprising:

an image indexer for assigning to each column in an alpha image an index, wherein the columns in the alpha image are comprised of alpha pixels having values, and wherein the alpha image corresponds to an image that includes the digital video object being composited;

a pixel evaluator for determining the values of the alpha pixels;

an index calculator for calculating for each index an index value that is based at least in part on the values of the alpha pixels comprising the column with which the index is associated;

an index locator for determining indices of boundaries and sections of the object, thereby identifying the shape of the object;

a template configurator for computing a set of a plurality of templates using the indices of the boundaries and sections of the object such that the set of templates comprises the shape of the object wherein the set of templates lies entirely within the portion of the object it represents including all boundary pixels; and

an image modifier for adjusting one or more portions of the alpha image based at least in part on the set of templates such that the one or more portions of the alpha image that correspond to image locations that lie within any template of the set of templates are processed as being part of the object so that no holes appear in the image in image locations that correspond to alpha image locations that lie within any template of the set of templates.

5. A system as recited in claim 4 further comprising an object configurator for computing a set of bounding rectangles such that portions of the object fit within the set of bounding rectangles and the set of bounding rectangles is derived using the indices of the boundaries and sections of the object.

6. A system for filling holes in an object digitally composited onto a destination image, the system comprising:

means for assigning to each column in an alpha image an index, wherein the columns in the alpha image are comprised of alpha pixels having values, and wherein the alpha image corresponds to an image that includes the digital video object being composited;

means for determining the values of the alpha pixels;

means for calculating for each index an index value that is based at least in part on the values of the alpha pixels comprising the column with which the index is associated;

means for determining indices of boundaries and sections of the object, thereby identifying the shape of the object;

means for computing a set of a plurality of templates using the indices of the boundaries and sections of the object such that the set of templates comprises the shape of the object wherein the set of templates lies entirely within the portion of the object it represents including all boundary pixels; and

means for adjusting one or more portions of the alpha image based at least in part on the set of templates such that the one or more portions of the alpha image that correspond to image locations that lie within the any template of the set of templates are processed as being part of the object so that no holes appear in the image in image locations that correspond to alpha image locations that lie within any template of the set of templates.

7. A system as recited in claim 6 further comprising means for computing a set of bounding rectangles such that portions of the object fit within the set of bounding rectangles and the set of bounding rectangles is derived using the indices of the boundaries and sections of the object.