Background extraction in a video picture

• US 5,940,139 A
• Filed: 10/01/1996
• Issued: 08/17/1999
• Est. Priority Date: 08/07/1996
• Status: Expired due to Fees

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Bell Communications Research, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Smoot, Lanny Starkes
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): Natnael, Paulos M.
• Application Number:
  US08/725,346
• Time in Patent Office: 1,050 Days
• Field of Search: 348/584, 348/586, 348/587, 348/589, 348/590, 348/591, 348/592, 348/594, 348/595, 348/596, 348/597, 348/598, 348/601, 348/722, 348/661, 348/239
• US Class Current: 348/584
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A novel video object extraction has a first object 12, such as a foreground object, illuminated with visible light 16 and a second object 14, such as a background object, illuminated with a combination of infrared 18 and visible light 16. A novel camera system observes the scene. The novel camera system includes an infrared light camera 24 and a visible light camera 20. The infrared light camera output 24 contains bright images of the second object 14, with relatively little brightness of the first object 12. The signal from this infrared light camera 20, once thresholded, serves as a key to remove the second object 14 from the video image captured by the visible light camera 20.

Claims

1. A method for object extraction in a video picture, comprising the steps of:

a. illuminating a first object with light having a first wavelength spectrum;

b. illuminating a second object with the first wavelength spectrum light and a light having a second wavelength spectrum, the first and second wavelength spectra being different;

c. taking a video image of the first and second objects using a first wavelength spectrum light sensor and a second wavelength spectrum light sensor;

d. using an output of the second wavelength spectrum light sensor as a mask, extracting from the video image of the first wavelength light spectrum sensor one of the first and the second objects and retaining the other one of the first and the second objects;

e. providing a third video image.; and

f. after the step of extracting, combining a video image of the retained object and the third video image.

9. The method of claim 1, further comprising the steps of:

a. selecting the first wavelength spectrum light to be visible light; and

b. selecting the second wavelength spectrum light to be infrared light.

2. A method for object extraction in a video picture, comprising the steps of:

a. illuminating a first object with light having a first wavelength spectrum;

b. illuminating a second object with the first wavelength spectrum light and a light having a second wavelength spectrum, the first and second wavelength spectra being different;

c. taking a video image of the first and second objects using a first wavelength spectrum light sensor and a second wavelength spectrum light sensor;

d. using an output of the second wavelength spectrum light sensor as a mask, extracting from the video image of the first wavelength light spectrum sensor one of the first and the second objects and retaining the other one of the first and the second objects;

the method further comprising after the step of taking a video image of the first and second objects using the second wavelength spectrum light sensor;

e. thresholding the output of the second wavelength spectrum light sensor; and

f. using the thresholded output as the mask.

3. The method of claim 2, wherein the step of thresholding further comprises:

a. comparing the second wavelength spectrum light sensor output to a threshold value;

b. setting all pixels having a value lower than the threshold value to a first level; and

c. setting all pixels having a value higher than the threshold value to a second level.

4. The method of claim 3, further comprising selecting the first level to be a video black level and setting the second level to be a video white level.

5. The method of claim 3, wherein the step of combining further comprises:

a. providing a third video image;

b. when a pixel having the first level is received, switching in a corresponding pixel from the third video image; and

c. when a pixel having the second level is received, switching in a corresponding pixel from an output of the first wavelength spectrum light sensor.

6. The method of claim 3, wherein the step of combining further comprises:

a. providing a third video image;

b. when a pixel having the second level is received, switching in a corresponding pixel from the third video image; and

c. when a pixel having the first level is received, switching in a corresponding pixel from an output of the first wavelength spectrum light sensor.

7. The method of claim 2, further comprising the step of extracting from the first wavelength spectrum light sensor output pixels corresponding to pixels in the mask having a predetermined value.

8. The method of claim 7, further comprising replacing extracted pixels with corresponding pixels from a third video image.

10. A video object extraction system, comprising:

a. a first wavelength spectrum light sensor configured to take a video image of a scene including an object;

b. a second wavelength light spectrum source configured to illuminate the object;

c. a second wavelength spectrum light sensor configured to take a video image of the scene including the object;

d. a comparator connected to receive an output of the second wavelength spectrum light sensor and to output a mask signal corresponding to detection of second wavelength spectrum light in the scene; and

e. a video gate connected to receive the mask signal and an output of the first wavelength spectrum light sensor and to selectively connect the output of the first wavelength spectrum light sensor to an output of the video gate in response to the mask signal.

11. The system of claim 10, wherein:

a. the first wavelength spectrum light sensor is a visible light sensor;

b. the second wavelength spectrum light source is an infrared light source; and

c. the second wavelength spectrum light sensor is an infrared light sensor.

12. The system of claim 11, wherein the infrared light source further comprises an infrared light pass filter.

13. The system of claim 10, further comprising a first wavelength spectrum light source configured to illuminate a second object.

14. The system of claim 13, wherein the first wavelength spectrum light source is a visible light source having an infrared light blocking filter.

15. The system of claim 10, wherein the first wavelength spectrum light sensor is a CCD camera sensitive to visible light and having an infrared light filter.

16. The system of claim 10, wherein the second wavelength spectrum light sensor is a CCD camera sensitive to infrared light and having a visible light filter.

17. The system of claim 10, further comprising a beam splitter disposed between the first wavelength spectrum light sensor and the second wavelength spectrum light sensor.

18. The system of claim 17, wherein the beam splitter comprises a combination of an optical beam splitter and an inverting mirror.

19. The system of claim 10, wherein the comparator further includes a first input configured to receive the output of the second wavelength spectrum light sensor and a second input configured to receive a reference value.

20. The system of claim 19, wherein the reference value is a voltage and, further comprising a variable voltage source connected to the second input.

21. The system of claim 10, wherein the comparator is configured to set an output pixel to a first level if a corresponding pixel from the second wavelength spectrum light sensor output is above a threshold value and to set the output pixel to a second level if the corresponding pixel from the second wavelength spectrum light sensor output is below the threshold value.

22. The system of claim 10, wherein the video gate comprises:

a. a first input configured to receive the mask signal;

b. a second input configured to receive the output of the first wavelength spectrum light sensor;

c. a third input configured to receive a third video image;

d. an output; and

e. a switch responsive to the first input for selectively connecting one of the second and third inputs to the output.

23. The system of claim 10, wherein the first wavelength spectrum light sensor and the second wavelength spectrum light sensor are a single unit.

24. The system of claim 23, wherein the single unit includes a lens system configured to receive the first and second wavelength spectrum light, an optical beam splitter configured to direct first wavelength spectrum light to a first wavelength spectrum light sensor and to direct a second wavelength spectrum light to a second wavelength spectrum light sensor.