Model based compositing

• US 7,663,647 B2
• Filed: 06/27/2006
• Issued: 02/16/2010
• Est. Priority Date: 10/15/1997
• Status: Expired due to Fees

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: Electric Planet Interactive
• Inventors: Ahmad, Subutai
• Primary Examiner(s): Wu; Xiao M
• Assistant Examiner(s): Ma; Tize
• Application Number:
  US11/477,047
• Publication Number:
  US 20060244764A1
• Time in Patent Office: 1,330 Days
• Field of Search: 345619-641, 348/578, 348/584, 348586-587
• US Class Current: 345/629
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Digitally compositing an object from an input image onto a destination image is disclosed. The object is composited from an image having an arbitrary or non-uniform colored background containing some non-static elements onto a destination image with reduced effects from shadows cast by the object and with reduced gaps or holes within the object. The effect of shadows emanating from the object is reduced so that the composited object in the destination image contains only the object clearly outlined by the object'"'"'s physical boundaries without the effect of shadows cast by the object.

Claims

1. A computer-implemented digital compositing method of reducing the effect of shadows in a digital image cast by an object, the method comprising:

obtaining average image pixel data from an input image that does not contain the object;

comparing a brightness of a current pixel in a current input image to a brightness of a corresponding pixel of the average image pixel data, wherein the current input image contains the object;

determining whether a hue value of the current pixel is within a hue tolerance of a hue of the corresponding pixel of the average image pixel data;

comparing a pattern surrounding the current pixel to a pattern surrounding the corresponding pixel of the average image pixel data, based on a comparison of the brightness of the current pixel to a brightness of one or more pixels adjacent the current pixel, and a comparison of the brightness of the corresponding pixel of the average image pixel data to a brightness of one or more pixels surrounding the corresponding pixel; and

adjusting a corresponding pixel in an alpha image that corresponds to portions of the current input image if the brightness of the current pixel is less than the brightness of the corresponding pixel in the average image pixel data, if the hue of the current pixel is within the hue tolerance of the hue of the corresponding pixel in the average image pixel data, and if the pattern surrounding the current pixel is substantially the same as the pattern surrounding the corresponding pixel in the average image pixel data.

2. The method of claim 1, further comprising comparing the brightness of the current pixel to a predetermined factor of the brightness of the corresponding pixel of the average image pixel data.

3. A computer-readable storage medium having computer-executable program instructions stored thereon that are executable by a computer to perform the method of claim 1.

4. The method of claim 1, wherein comparing a pattern surrounding the current pixel to a pattern surrounding the corresponding pixel of the average image pixel data comprises:

calculating a first statistic representing a difference in brightness between the current pixel and one or more adjacent pixels of the current input image;

calculating a second statistic representing a difference in brightness between the corresponding pixel of the average image pixel data and one or more adjacent pixels of the average image pixel data; and

determining that the texture surrounding the current pixel is substantially the same as the texture surrounding the corresponding pixel in the average image pixel data if the first statistic matches the second statistic.

5. The method of claim 4, wherein the first statistic and the second statistic are each a rank based on differences in brightness between adjacent pixels.

6. The method of claim 5, wherein the first statistic matches the second statistic if a rank value of the first statistic is the same as the rank value of the second statistic.

7. The method of claim 1, wherein adjusting a corresponding pixel in an alpha image that corresponds to portions of the current input image comprises:

setting the corresponding pixel in the alpha image to a value indicating that the current pixel in the current input image should not be made part of a composite image when the brightness of the current pixel is less than the brightness of the corresponding pixel in the average image pixel data, the hue of the current pixel is within the hue tolerance of the hue of the corresponding pixel in the average image pixel data, and the pattern surrounding the current pixel is substantially the same as the pattern surrounding the corresponding pixel in the average image pixel data.

8. A computer-implemented system for reducing the effect of shadows cast by an object digitally composited onto a destination image, the system comprising:

a brightness comparator for comparing a brightness of a current pixel in an input image that contains an object to a brightness of a corresponding pixel in a background image model that does not contain the object;

a hue analyzer for determining whether hue values of the current pixel are within hue tolerances of the corresponding pixel in the background image model;

a texture comparator for comparing textures by comparing a pattern of brightness differences between the current pixel and pixels adjacent to the current pixel to a pattern of brightness differences between the corresponding pixel in the background image model and pixels adjacent to the corresponding pixel in the background image model; and

an image modifier for adjusting portions of an alpha image that correspond to portions of the input image that contains the object,

wherein the portions of the input image contain shadows cast by the object thereby conveying to the alpha image information on which portions of the input image that contains the object are shadows cast by the object, and

wherein the adjustment is based on the comparison of the brightness and texture of the pixels in the input image that contains the object and the brightness and texture of the corresponding pixels in the input image that does not contain the object and if the hue values of the pixels of the input image that contains the object are within a hue tolerance value of the corresponding pixels in the input image that does not contain the object.

9. The system of claim 8, wherein the texture comparator further comprises a rank assigner for assigning a rank to the current pixel based on differences in brightness between the current pixel and adjacent pixels in the input image and for assigning a rank to the corresponding pixel in the background image model based on differences in brightness between the corresponding pixel and adjacent pixels in the background image model.

10. The system of claim 9, wherein the texture comparator compares the rank of the current pixel to the rank of the corresponding pixel.

11. The system of claim 8, wherein the brightness comparator compares the brightness of the current pixel to a predetermined factor of the brightness of the corresponding pixel.

12. A system for reducing the effect of shadows cast by an object digitally composited onto a destination image, the system comprising:

means for comparing a brightness of a pixel in an input image that contains an object to a brightness of a corresponding pixel in an input image that does not contain the object;

means for determining whether hue values of the pixel in the input image that contains the object are within hue tolerances of the corresponding pixel in the input image that does not contain the object;

means for comparing a texture surrounding the pixel in the input image that contains the object to a texture surrounding the corresponding pixel in the input image that does not contain the object, based on a comparison of the brightness of the pixel in the input image that contains the object to a brightness of one or more pixels surrounding the pixel in the input image that contains the object, and a comparison of the brightness of the corresponding pixel in the input image that does not contain the object to a brightness of one or more pixels surrounding the corresponding pixel in the input image that does not contain the object; and

means for adjusting a corresponding pixel of an alpha image, wherein the input image contains shadows cast by the object, thereby conveying to the alpha image information on which portions of the input image that contains the object are shadows cast by the object.

13. The system of claim 12, wherein the means for comparing a texture surrounding the pixel in the input image that contains the object to a texture surrounding the corresponding pixel in the input image that does not contain the object comprises:

means for comparing the brightness of the pixel in the input image that contains the object to a brightness of a neighboring pixel in the input image that contains the object to determine a rank for the pixel in the input image that contains the object with respect to the neighboring pixel;

means for comparing the brightness of the corresponding pixel in the input image that does not contain the object to a brightness of a corresponding neighboring pixel in the input image that does not contain the object to determine a rank for the corresponding pixel in the input image that does not contain the object with respect to the corresponding neighboring pixel; and

means for determining that the texture surrounding the pixel in the input image that contains the object is substantially similar to the texture surrounding the corresponding pixel in the input image that does not contain the object when the rank of the pixel in the input image that contains the object matches the rank of the corresponding pixel in the input image that does not contain the object with respect to one or more neighboring pixels.

14. A tangible computer readable storage medium that includes instructions that, if executed by a computer system, cause the computer system to perform a method of reducing the effect of shadows in a digital image cast by an object by:

storing in a memory a first image that contains an object and a second background image that does not contain the object;

comparing a brightness and a hue of a pixel in a first image to a brightness and a hue of a corresponding pixel in the second image;

comparing a texture associated with the pixel in the first image to a texture associated with the corresponding pixel in the second image by comparing a difference in brightness between the pixel in the first image and a neighboring pixel in the first image to a difference in brightness between the corresponding pixel in the second image and a corresponding neighboring pixel in the second image; and

changing a value of a corresponding pixel of an alpha image that corresponds to the pixel in the first image to indicate that the corresponding pixel of the alpha image depicts part of a shadow cast by the object if the brightness of the pixel in the first image is less than brightness of the corresponding pixel in the second image and if the hue of the pixel in the first image and the corresponding pixel in the second image are substantially the same and if the texture associated with the pixel in the first image is substantially the same as the texture associated with the corresponding pixel in the second image.

15. A system for reducing the effect of shadows in a digital image cast by an object, the system including:

a processor that is configured to perform a sequence of programmed instructions to;

store in a memory a first image that contains an object and a background image model derived from an image that does not contain the object;

compare a brightness and a hue of a pixel in a first image to a brightness and a hue of a corresponding pixel in the background image model;

compare a pattern around the pixel in the first image to a pattern around the corresponding pixel in the background image model by comparing a difference in brightness between the pixel in the first image and a brightness of each neighboring pixel in the first image to a difference in brightness between the corresponding pixel in the background image model and a brightness of each corresponding neighboring pixel in the background image model; and

change a value of a corresponding pixel of an alpha image that corresponds to the pixel in the first image to indicate that the corresponding pixel contains part of a shadow cast by the object if the brightness of the pixel in the first image is less than brightness of the corresponding pixel in the second image and if the hue of the pixel in the first image and the corresponding pixel in the second image are substantially the same and if the pattern around the pixel in the first image is substantially the same as the pattern around the corresponding pixel in the second image.