Method for replacing the background of an image

• US 5,923,380 A
• Filed: 01/25/1996
• Issued: 07/13/1999
• Est. Priority Date: 10/18/1995
• Status: Expired due to Term

• Current Assignee: Yama Capital, LLC
• Original Assignee: Polaroid Corporation (PLR IP Holdings LLC)
• Inventors: Hajjahmad, Ibrahim, Yang, Yibing, Bowman, John C.
• Primary Examiner(s): HSIA, SHERRIE Y
• Application Number:
  US08/591,727
• Time in Patent Office: 1,265 Days
• Field of Search: 348/586, 348/590, 348/587, 348/588, 348/591, 348/592, 348/598, 348/164, 348/162, 382/190, 382/195, 352/49, 352/89, 352/46, 352/48
• US Class Current: 348/586
• CPC Class Codes: G03B 17/53 for automatically delivering a finished picture after a signal causing exposure has been given, e.g. by pushing a button, by inserting a coin
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

The present invention is useful in taking a visible light image for identification and other purposes without the requirement of a photobooth, regardless of the background of the visible light image. The original background of the visible light image is replaced with a preselected background. Two IR images with different intensities of IR illumination in the foreground and background regions of the scene, respectively, are compared to produce a difference image of light intensity differences between corresponding pixels of the two images. A binarized image is generated by binarizing the difference image with respect to a predetermined threshold value θ. A connectivity constraint is used to generate a binary mask from the binarized image, then a gray-scale mask is produced by multiplying the binary mask times a preselected modulation function of the difference image. Warping the gray-scale mask produces a transformed mask. Finally, the background replaced visible light image is generated by blending the transformed mask with the visible light image, and replacing the original background with the preselected background.

Claims

1. A digital processing method of replacing an original background of a visible light image of a scene with a predetermined replacement background, said method comprising the steps of:

making a first infrared (IR) image of the scene at a first time while illuminating the original background with first IR radiation having a first intensity;

measuring the intensity of IR radiation at each pixel of said first IR image;

making a second IR image of the scene at a second time after deactivating said first IR radiation, while illuminating a foreground of the scene with second IR radiation having a second intensity less than said first intensity;

measuring the intensity of IR radiation at each pixel of said second IR image;

making the visible light image of the scene at a third time while illuminating the scene with visible lighting;

generating a transformed mask distinguishing said foreground from said original background by producing a difference image (DIFF), a binarized image, a binary mask and a gray-scale mask; and

producing a modified visible light image by blending said visible light image with said predetermined replacement background, using said transformed mask.

2. The method of claim 1, wherein said step of generating a transform mask further comprises the steps of:

producing the difference image by subtracting the second IR image from the first IR image;

producing the binarized image distinguishing said foreground from said original background by comparing pixels of said difference image to a predetermined parameter θ

, then setting a pixel of the binarized image to a logic high when said pixel of the binarized image is greater than θ

, otherwise setting said pixel of the binarized image to a logic low;

producing the binary mask from the binarized image by removing false foreground pixels of the binarized image according to a connectivity constraint;

producing the gray-scale mask specifying how much of each pixel of the binary mask is attributed to the foreground and how much of each pixel of the binary mask is attributed to the original background by applying a predetermined modulation function of the difference image to the binary mask; and

producing the transformed mask by pixel-to-pixel image registration of the gray-scale mask and the visible light image.

3. The method of claim 2, wherein for a given pixel of the gray-scale mask said predetermined modulation function equals:

(DIFF-θ

_L)/(θ

_H -θ

_L) if a corresponding pixel in the difference image is less than a predetermined parameter θ

_H and greater than a predetermined parameter θ

_L ;

0, if the corresponding pixel in the difference image is less than or equal to θ

_L ; and

1, if the corresponding pixel in the difference image is greater than or equal to θ

_H.

4. The method of claim 1, wherein said second IR radiation originates from a front light.

5. The method of claim 1, wherein said second IR radiation originates from one or more foreground lights.

6. The method of claim 1, wherein said second IR radiation originates from ambient light.

7. The method of claim 1, wherein said third time equals said second time.

8. The method of claim 1, wherein said first IR radiation originates from one or more background lights.

9. The method of claim 1, wherein said first IR radiation originates from ambient light.

10. The method of claim 1, wherein said first IR radiation and said second IR radiation both have wavelengths ranging from about 700 nanometers to about 1000 nanometers.

11. The method of claim 1, wherein a difference between said first time and said second time approximates a shutter speed of an imaging device for making said first IR image, said second IR image and said visible light image.

12. The method of claim 1, wherein a difference between said first time and said second time is about 1/30th of a second.

13. The method of claim 1, wherein a difference between said first time and said second time ensures negligible movement of objects within said scene while taking said first IR, second IR and visible light images so that said images will be in focus.

14. The method of claim 1, wherein both said first and second IR images are taken with a first digital imaging device, said visible light image is taken with a second digital imaging device, and said first and second digital imaging devices have the same virtual spatial location.