METHOD AND APPARATUS FOR BACKGROUND REPLACEMENT IN STILL PHOTOGRAPHS

• US 20110134141A1
• Filed: 10/04/2010
• Published: 06/09/2011
• Est. Priority Date: 04/03/2007
• Status: Active Grant

• Current Assignee: Shutterfly, LLC
• Original Assignee: Lifetouch Incorporated (Shutterfly, LLC)
• Inventors: Adolphi, Earl John, Swanson, Richard Lee, Surma, Michael John, Reinert Nash, John Robert
• Granted Patent: US 8,134,576 B2
• Time in Patent Office: Days
• US Class Current: 345/634
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A first digital image is acquired of a framed area while illuminating the background and foreground object under a first lighting condition. A second digital image is then acquired of the same framed area while illuminating the background and foreground object under a second lighting condition. Preferably, the first lighting condition illuminates the background without illuminating the foreground object so that a silhouette of the foreground object is acquired in the first image. The second lighting condition illuminates the foreground object (e.g., with frontal lights). Due to the difference in the illumination between the background and silhouette in the first image, an alpha mask can be created from the first acquired image. Using the mask, the background from the second image can be removed and replaced by virtually any other desired background image.

Claims

1. A method of imaging a foreground object placed in front of a background, the method comprising:

acquiring a first image of the foreground object and the background while illuminating the background under a first lighting condition relative to the object;

acquiring a second image of the foreground object and the background while illuminating the background under a second lighting condition relative to the object;

computing a mixing function based on a selected one of the images; and

using the image that was not selected and the mixing function, computing an object image function relating to an image of the foreground object with a reduced background intensity relative to the foreground object as compared to the image that was not selected.

2. The method of claim 1, wherein the selected image is the first image and wherein the step of computing a mixing function includes computing an alpha value for alpha blending for each pixel of the first image.

3. The method of claim 1, wherein acquiring the first image under the first lighting condition comprises illuminating the background so that it is brighter than the foreground object.

4. The method of claim 1, further comprising combining the object image function and a third image to generate a composite image in which the third image appears to be behind the foreground object.

5. The method of claim 4, wherein:

computing a mixing function comprises computing an alpha value (α

) for alpha blending for each pixel of the first image;

computing the object image function comprises computing α

F^C for each pixel, where F^C denotes the color vector of the pixel of the image of the foreground object with the background removed; and

generating the composite image comprises computing each pixel of the composite image according to the formula;

R=α

F^c+(1−

α

)B^new,

where R is the color vector of each pixel of the composite image and B^new is the color vector of the third image.

6. The method of claim 1, further comprising determining a maximum acceptable movement speed of the foreground object and computing a time interval within which the first and second images should be acquired based on the maximum acceptable movement speed.

7. The method of claim 6, further comprising setting a maximum number of pixels by which the first and second images are permitted to be offset from each other, wherein computing the time interval comprises computing the time interval based on the number of pixels by which the first and second images are permitted to be offset from each other.

8. The method of claim 7, wherein computing the time interval comprises computing the time interval according to the formula

t=w_sd_t/(w_cv_t)

where v_t is the tolerated speed of the foreground object,

w_s is the width of the image at the foreground object,

d_t is the number of pixels by which the first and second images are permitted to be offset from each other,

w_c is the width (in pixels) of the first and second images, and

t is the time to capture both images.

9. The method of claim 7, wherein setting a maximum number pixels by which the first and second images are permitted to be offset from each other comprises limiting the offset to a fraction of a pixel.

10. The method of claim 6, wherein acquiring the first and second images comprises acquiring the first and second images under flash illumination and acquiring the second image after acquiring the first image.

11. The method of claim 1, wherein acquiring the first and second images comprises acquiring the first and second images under flash illumination and acquiring the second image within a time interval shorter than one (1) millisecond after acquiring the first image.

12. The method of claim 1, wherein the foreground object cannot move and wherein acquiring the first and second images comprises acquiring the first and second images under flash illumination and acquiring the second image at any given future time after acquiring the first image.

13. The method of claim 4, further comprising transmitting the object image function from a first location to a second location over a computer network, wherein generating the composite image comprises generating the compositing function based on the transmitted object image function.

14. A method of computing an image, the method comprising:

computing a mixing function from a first image taken of a scene having a foreground object placed in front of a background, where the background is illuminated under a first lighting condition relative to the object;

accessing a second image of the foreground object and the background, where the background is illuminated under a second lighting condition relative to the object; and

using the second image and the mixing function, computing an object image function relating to an image of the foreground object with a reduced background intensity relative to the foreground object as compared to the second image.

15. The method of claim 14, further comprising acquiring the first image of the scene, wherein the background is illuminated so that it is brighter than the foreground object under the first lighting condition.

16. The method claim 15, wherein the foreground object appears substantially black in the first image under the first lighting condition.

17. A method of imaging an object placed in front of a background, the method comprising:

acquiring a first image of the foreground object and the background while illuminating the background under a first lighting condition relative to the object;

acquiring a second image of the foreground object and the background while illuminating the background under a second lighting condition relative to the object within a predetermined time interval of acquiring the first image, the predetermined time interval being computed based on a maximum acceptable speed of movement of the foreground object.

18. The method of claim 17, further comprising setting a maximum number pixels by which the first and second images are permitted to be offset from each other, wherein computing the predetermined time interval comprises computing the predetermined time interval based on the maximum number of pixels by which the first and second images are permitted to be offset from each other.

19. The method of claim 18, wherein computing the predetermined time interval comprises computing the predetermined time interval according to the formula

t=w_sd_t/(w_cv_t)

where v_t is the tolerated speed of the foreground object,

w_s is the width of the image at the foreground object,

d_t is the number of pixels by which the first and second images are permitted to be offset from each other,

w_c is the width (in pixels) of the first and second images, and

t is the time to capture the two images.

20. The method of claim 18, wherein setting a maximum number pixels by which the first and second images are permitted to be offset from each other comprises limiting the offset to a fraction of a pixel.

21-45. -45. (canceled)