Photobooth compositing apparatus

• US 5,117,283 A
• Filed: 06/25/1990
• Issued: 05/26/1992
• Est. Priority Date: 06/25/1990
• Status: Expired due to Term

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Kroos, Donna S., Sanger, Kurt M., McCauley, David K.
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): VU, KIM Y
• Application Number:
  US07/542,619
• Time in Patent Office: 701 Days
• Field of Search: 358/22, 358/22 CR, 358/182
• US Class Current: 348/564
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

An image capture and compositing apparatus for a photobooth (10) photographs a customer against a fixed color background (28), extracts the customer from that image and then inserts the extracted customer image into an arbitrary background image in such a manner that there are no noticeable artifacts between the inserted image and the background image and so that it appears that the subject was actually situated in the background scene when photographed. The imagery processing mechanism employed generates a pair of masks, one of which is associated with the foreground image and the other of which is associated with the background image. These masks are used to extract the subject (160) from an image of the subject against a reference color backdrop, to define what portion of the background image is to be blanked out and to controllably blend the border of the inserted subject image with the background image. The photobooth contains a customer lighting system (50,42) located above and behind the position where the customer is seated. A separate backdrop lighting system (32) is located behind the customer and illuminates a blue reference backdrop against which the customer is photographed and which is photographed per se to provide a reference for mask production.

Claims

1. A method of combining an image of a subject with an image of an arbitrary background to thereby form a composite image of said subject and said arbitrary background comprising the steps of:

(a) generating a first image of a reference backdrop and storing a first plurality of first image data values respectively representative of the contents of said first image at a plurality of locations of an array of image locations within said first image;

(b) generating a second image of said subject situated in the presence of said reference backdrop and storing a second plurality of second image data values respectively representative of the contents of said second image at said plurality of locations of said array of image locations;

(c) storing a third plurality of third image data values respectively representative of the contents of a third image containing said arbitrary background at said plurality of locations of said array of image locations;

(d) for each of plural sets of image locations within said array, processing color contents of the associated data values for each of said first and second images, so as to generate a fourth plurality of first image mask values respectively representative of the contents of a first image mask associated with the image locations of said array containing said subject and a fifth plurality of second image mask values respectively representative of the contents of a second image mask associated with locations of said array other than said subject; and

(e) combining said fourth and fifth pluralities of respective first and second image mask values with said second plurality of second image data values and said third plurality of third image data values, to thereby obtain a sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

2. A method according to claim 1, wherein, in step (d), a respective set of plural image locations within said array of image locations comprises a two-dimensional sub-array of image locations the color contents of which are processed to obtain a respective first image mask value and a respective second image mask value for one of the image locations of said array of image locations.

3. A method according to claim 2, wherein said one of the image locations of said sub-array is surrounded by other image locations of said sub-array.

4. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes analyzing the color contents of the associated data values for each of said first and second images in accordance with a prescribed color attribute of said prescribed color backdrop.

5. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes analyzing the color contents of the associated data values for each of said first and second images in accordance with a color other than the predominant color in said prescribed color backdrop.

6. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes analyzing the color contents of the associated data values for each of said first and second images in accordance with the degree of predominance of said prescribed color therein.

7. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes analyzing the color contents of the associated data values for each of said first and second images in accordance with the degree of presence of colors other than said prescribed color therein.

8. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes generating said second image mask values respectively representative of the contents of said second image mask in dependence upon the degree of predominance of said prescribed color in said first and second images and generating said first image mask values respectively representative of the contents of said first image mask in dependence upon the degree of presence of colors other than said prescribed color in said first and second images.

9. A method according to claim 1, wherein said reference backdrop is that of a prescribed color backdrop and step (d) includes generating said first image mask values respectively representative of the contents of said first mask image in dependence upon a first degree of presence of colors other than said prescribed color in said first and second images and said second image mask values respectively representative of the contents of said second mask image in dependence upon a second degree of presence of colors other than said prescribed color in said first and second images.

10. A method of forming a composite image of a subject and an image of an arbitrary background comprising the steps of:

(a) illuminating a prescribed color backdrop in front of which said subject is to be situated and generating a first image of said prescribed color backdrop;

(b) storing a first plurality of first image data values respectively representative of the contents of said first image at a plurality of locations of an array of image locations within said first image;

(c) illuminating said backdrop and a subject situated in front of said prescribed color backdrop and generating a second image of said subject and said illuminated backdrop;

(d) storing a second plurality of second image data values respectively representative of the contents of said second image at said plurality of locations of said array of image locations;

(e) storing a third plurality of third image data values respectively representative of the contents of a third image containing said arbitrary background at said plurality of locations of said array of image locations;

(f) for each of plural sets of image locations within said array, processing color contents of the associated data values for each of said first and second images, so as to generate a fourth plurality of first image mask values respectively representative of the contents of a first image mask associated with the image locations of said array containing said subject and a fifth plurality of second image mask values respectively representative of the contents of a second image mask associated with locations of said array other than said subject; and

(g) combining said fourth and fifth pluralities of respective first and second image mask values with said second plurality of second image data values and said third plurality of third image data values to thereby obtain a sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

11. A method according to claim 10, wherein step (f) comprises analyzing the color contents of the associated data values for each of said first and second images in accordance a prescribed color attribute of said backdrop.

12. A method according to claim 10, wherein step (f) comprises analyzing the color contents of the associated data values for each of said first and second images in accordance with a color other than the predominant color in said prescribed color backdrop.

13. A method according to claim 10, wherein step (f) includes generating said second mask image values respectively representative of the contents of said second image mask in dependence upon the degree of predominance of said prescribed color in said first and second images and generating said first image mask values respectively representative of the contents of said first mask image in dependence upon the degree of presence of colors other than said prescribed color in said first and second images.

14. A method according to claim 10, wherein step (f) includes generating said first image mask values respectively representative of the contents of said first image mask in dependence upon a first degree of presence of colors other than said prescribed color in said first and second images and said second image mask values respectively representative of the contents of said second image mask in dependence upon a second degree of presence of colors other than said prescribed color in said first and second images.

15. A method according to claim 10, wherein step (c) comprises illuminating said subject from a plurality of different illumination directions.

16. A method according to claim 15, wherein step (c) comprises illuminating said subject from lighting locations above, to the rear and in front of said subject.

17. A method according to claim 10, wherein step (f) comprises processing color contents of the associated data values for each of said first and second images with reference to said prescribed color in said first and second images to identify which of the image locations within said array of image locations contain an image of said subject.

18. A method according to claim 10, wherein step (f) comprises processing color contents of the associated data values for each of said first and second images in dependence upon the presence of one or more colors other than said prescribed color in said first and second images to identify which of the image locations within said array of image locations contain an image of said subject.

19. A method according to claim 18, wherein step (f) comprises processing color contents of the associated data values for each of said first and second images in accordance with predetermined relationships between color contents of said first and second images.

20. A method according to claim 17, wherein step (f) comprises processing color contents of the associated data values for each of said first and second images in dependence upon the presence of one or more colors other than said prescribed color in said first and second images to identify at which of the image locations containing said subject, associated first and second image mask values are to be controllably weighted in the course of being combined with said third image data values and said second image data values to obtain said sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

21. A method of forming a composite image of a subject and an image of an arbitrary background comprising the steps of:

(a) illuminating a prescribed color backdrop in front of which said subject is to be situated and generating a first image of said prescribed color backdrop;

(b) storing, in a backdrop imagery database, a first plurality of first image data values respectively representative of the contents of said first image at a plurality of locations of an array of image locations within said first image;

(c) illuminating said backdrop and a subject situated in front of said prescribed color backdrop and generating a second image of said subject and said illuminated backdrop;

(d) storing, in a subject imagery database, a second plurality of second image data values respectively representative of the contents of said second image at said plurality of locations of said array of image locations;

(e) storing, in a background imagery database, a third plurality of third image data values respectively representative of the contents of a third image containing said arbitrary background at said plurality of locations of said array of image locations;

(f) for each respective pixel location within said array, processing color contents of associated data values for a plurality of pixel locations, including said each respective pixel location, of said first and second images contained in said backdrop and subject databases, so as to generate a fourth plurality of first image mask values respectively representative of the contents of a first image mask associated with the image locations of said array containing said subject and a fifth plurality of second image mask values respectively representative of the contents of a second image mask associated with locations of said array other than said subject; and

(g) combining said fourth and fifth pluralities of respective first and second image mask values with said third plurality of third image data values in said background database, and said second plurality of second image data values, to thereby obtain a sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

22. A method according to claim 21, wherein step (f) comprises the steps of:

(f1) generating a first value representative of the likelihood of the presence of a color other than said backdrop color for said plurality of pixel locations; and

(f2) in response to said first value exceeding a prescribed subject threshold associated with the presence of a subject region in said second image, setting a first code value of said first image mask for said respective pixel location at a first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting a second code value of said second image mask for said respective pixel location at a second mask value representative that the data value of said first image is to be deselected for said respective pixel location.

23. A method according to claim 22, wherein step (f) further comprises the steps of:

(f3) in response to said first value not exceeding said prescribed subject threshold, generating a second value representative of the degree of presence of a selected color other than the color of said backdrop for said plurality of pixel locations; and

(f4) in response to said second value exceeding a predetermined selected color threshold associated with the presence of a subject region in said second image, setting said first code value of said first image mask for said respective pixel location at said first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said second mask value representative that the data value of said first image is to be deselected for said respective pixel location.

24. A method according to claim 23, wherein step (f) further includes the steps of:

(f5) in response to said second value not exceeding said predetermined selected color threshold, comparing said first value to a backdrop threshold associated with the presence of a backdrop region in said second image;

(f6) in response to said first value not exceeding said backdrop threshold, generating a third value representative of the degree of presence of said backdrop color for said plurality of pixel locations; and

(f7) in response to said third value exceeding a predetermined backdrop color threshold, setting said first code value of said first image mask for said respective pixel location at said first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said second mask value representative that the data value of said first image is to be deselected for said respective pixel location, but otherwise setting said first code value of said first image mask for said respective pixel location at said second mask value representative that the imagery data value of said second image is to be deselected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said first mask value representative that the data value of said first image is to be selected for said respective pixel location.

25. A method according to claim 24, wherein step (f) further includes the steps of:

(f8) in response to said first value exceeding said backdrop threshold, generating a fourth value representative of the likelihood that said respective pixel location lies in a shadow region of said second image;

(f9) comparing said fourth value to a shadow threshold associated with the presence of said shadow region; and

(f10) in response to said fourth value not exceeding said shadow threshold, setting said first code value of said first image mask for said respective pixel location at said second mask value representative that the imagery data value of said second image is to be deselected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said first mask value representative that the data value of said first image is to be selected for said respective pixel location, but otherwise setting said first code value of said first image mask for said respective pixel location at a third mask value representative of a first fractional selection of the imagery data value of said second image for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at a fourth mask value representative of a second fractional selection of the imagery data value of said first image for said respective pixel location.

26. A photobooth image processing apparatus for generating a composite image of a subject and an arbitrary background, said photobooth having a reference colored backdrop situated behind a location whereat a subject is to be situated comprising, in combination:

an imaging camera arranged to view said backdrop, and said subject, when present in the photobooth, against said backdrop;

a backdrop imaging arrangement which illuminates said backdrop in front of which said subject is to be situated and causes said imaging camera to generate an image of said backdrop and produce a first plurality of backdrop image data values respectively representative of the contents of said backdrop image at a plurality of locations of an array of image locations within backdrop first image;

a backdrop image database which stores said first plurality of backdrop image data values;

a subject imaging arrangement which illuminates said backdrop and said subject against said backdrop and causes said imaging camera to generate a subject image of said subject against said backdrop and to produce a second plurality of subject image data values respectively representative of the contents of said subject image at said plurality of locations of said array of image locations;

a subject image database which stores said second plurality of subject image data values respectively representative of the contents of said subject image at said plurality of locations of said array;

a background image database which stores a third plurality of background image data values respectively representative of the contents of a background image containing said arbitrary background at said plurality of locations of said array; and

an imagery data processor which, for each of plural sets of image locations within said array, processes color contents of the associated data values for each of said backdrop and subject images, so as to generate a fourth plurality of first image mask values respectively representative of the contents of a first image mask associated with the image locations of said array containing said subject and a fifth plurality of second image mask values respectively representative of the contents of a second image mask associated with locations of said array other than said subject, said imagery data processor combining said fourth and fifth pluralities of respective first and second image mask values with said second plurality of subject image data values and said third plurality of background image data values to thereby obtain a sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

27. An apparatus according to claim 26, wherein said processor is programmed to analyze the color contents of the associated data values for each of said backdrop and subject images in accordance with a prescribed color attribute of said backdrop.

28. An apparatus according to claim 26, wherein said processor is programmed to analyze the color contents of the associated data values for each of said backdrop and subject images in accordance with a color other than the predominant color in said backdrop.

29. An apparatus according to claim 26, wherein said processor generates said second mask image values respectively representative of the contents of said second image mask in dependence upon the degree of predominance of said reference color in said backdrop and subject images and generates said first image mask values respectively representative of the contents of said first mask image in dependence upon the degree of presence of colors other than said reference color in said backdrop and subject images.

30. An apparatus according to claim 26, wherein said processor generates said first image mask values respectively representative of the contents of said first image mask in dependence upon a first degree of presence of colors other than said reference color in said backdrop and subject images and said second image mask values respectively representative of the contents of said second image mask in dependence upon a second degree of presence of colors other than said reference color in said backdrop and subject images.

31. An apparatus according to claim 26, wherein said subject imaging arrangement includes a lighting configuration which illuminates said subject from a plurality of different illumination directions.

32. An apparatus according to claim 31, wherein said subject imaging arrangement includes a lighting configuration which illuminates said subject from lighting locations above, to the rear and in front of said subject.

33. An apparatus according to claim 26, wherein said processor is programmed to process color contents of the associated data values for each of said background and subject images in accordance with the presence of said reference color in said backdrop and subject images, so to identify which of the image locations within said array of image locations contain an image of said subject.

34. An apparatus according to claim 33, wherein said processor is programmed to process color contents of the associated data values for each of said backdrop and subject images in dependence upon the presence of one or more colors other than said reference color in said backdrop and subject images to identify which of the image locations within said array of image locations contain an image of said subject.

35. An apparatus according to claim 26, wherein said processor is programmed to process color contents of the associated data values for each of said backdrop and subject images in accordance with predetermined relationships between color contents of said backdrop and reference images.

36. An apparatus according to claim 33, wherein said processor is programmed to process color contents of the associated data values for each of said backdrop and subject in dependence upon the presence of one or more colors other than said reference color in said backdrop and subject images to identify at which of the image locations containing said subject, associated first and second image mask values are to be controllably weighted in the course of being combined with said background image data values and said subject image data values to obtain said sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

37. A photobooth image processing apparatus for generating a composite image of a subject and an arbitrary background, said photobooth having a prescribed color backdrop situated behind a location whereat a subject is to be situated when imaged comprising, in combination:

an imaging camera arranged to view said backdrop, and said subject, when present in the photobooth, against said backdrop;

a backdrop imaging arrangement which illuminates said backdrop in front of which said subject is to be situated and causes said imaging camera to generate an image of said backdrop and produce a first plurality of backdrop image data values respectively representative of the contents of said backdrop image at a plurality of locations of an array of image locations within backdrop first image;

a backdrop image database which stores said first plurality of backdrop image data values;

a subject imaging arrangement which illuminates said backdrop and said subject against said backdrop and causes said imaging camera to generate a subject image of said subject against said backdrop and to produce a second plurality of subject image data values respectively representative of the contents of said subject image at said plurality of locations of said array of image locations;

a subject image database which stores said second plurality of subject image data values respectively representative of the contents of said subject image at said plurality of locations of said array;

a background image database which stores a third plurality of background image data values respectively representative of the contents of a background image containing said arbitrary background at said plurality of locations of said array;

an imagery data processor which, for each respective pixel location within said array, processes color contents of associated data values for a plurality of pixel locations, including said each respective pixel location, of said backdrop and subject images data values for which are respectively stored in said backdrop and subject databases, so as to generate a fourth plurality of first image mask values respectively representative of the contents of a first image mask associated with the image locations of said array containing said subject and a fifth plurality of second image mask values respectively representative of the contents of a second image mask associated with locations of said array other than said subject; and

combining said fourth and fifth pluralities of respective first and second image mask values with said second plurality of subject image data values and said third plurality of background image data values stored in said background database, to thereby obtain a sixth plurality of sixth image data values representative of a composite image of said subject and said arbitrary background.

38. An apparatus according to claim 37, wherein said processor is programmed to execute a prescribed image processing program comprising the steps of:

(a) generating a first value representative of the likelihood of the presence of a color other than said backdrop color for said plurality of pixel locations; and

(b) in response to said first value exceeding a prescribed subject threshold associated with the presence of a subject region in said second image, setting a first code value of said first image mask for said respective pixel location at a first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting a second code value of said second image mask for said respective pixel location at a second mask value representative that the data value of said first image is to be deselected for said respective pixel location.

39. An apparatus according to claim 38, wherein said a prescribed image processing program further comprises the steps of:

(c) in response to said first value not exceeding said prescribed subject threshold, generating a second value representative of the degree of presence of a selected color other than the color of said backdrop for said plurality of pixel locations; and

(d) in response to said second value exceeding a predetermined selected color threshold associated with the presence of a subject region in said second image, setting said first code value of said first image mask for said respective pixel location at said first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said second mask value representative that the data value of said first image is to be deselected for said respective pixel location.

40. An apparatus according to claim 39, wherein said a prescribed image processing program further comprises the steps of:

(e) in response to said second value not exceeding said predetermined selected color threshold, comparing said first value to a backdrop threshold associated with the presence of a backdrop region in said second image;

(f) in response to said first value not exceeding said backdrop threshold, generating a third value representative of the degree of presence of said backdrop color for said plurality of pixel locations; and

(g) in response to said third value exceeding a predetermined backdrop color threshold, setting said first code value of said first image mask for said respective pixel location at said first mask value representative that the imagery data value of said second image is to be selected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said second mask value representative that the data value of said first image is to be deselected for said respective pixel location, but otherwise setting said first code value of said first image mask for said respective pixel location at said second mask value representative that the imagery data value of said second image is to be deselected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said first mask value representative that the data value of said first image is to be selected for said respective pixel location.

41. An apparatus according to claim 40, wherein said a prescribed image processing program further comprises the steps of:

(h) in response to said first value exceeding said backdrop threshold, generating a fourth value representative of the likelihood that said respective pixel location lies in a shadow region of said second image;

(i) comparing said fourth value to a shadow threshold associated with the presence of said shadow region; and

(j) in response to said fourth value not exceeding said shadow threshold, setting said first code value of said first image mask for said respective pixel location at said second mask value representative that the imagery data value of said second image is to be deselected for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at said first mask value representative that the data value of said first image is to be selected for said respective pixel location, but otherwise setting said first code value of said first image mask for said respective pixel location at a third mask value representative of a first fractional selection of the imagery data value of said second image for said respective pixel location and setting said second code value of said second image mask for said respective pixel location at a fourth mask value representative of a second fractional selection of the imagery data value of said first image for said respective pixel location.

42. An apparatus according to claim 37, wherein said subject imaging arrangement includes a lighting configuration which illuminates said subject from a plurality of different illumination directions.

43. An apparatus according to claim 42, wherein said subject imaging arrangement includes a lighting configuration which illuminates said subject from lighting locations above, to the rear and in front of said subject.

44. An apparatus according to claim 37, wherein said subject imaging arrangement includes a backlighting configuration which illuminates said subject so that the luminance component of the backlighting illumination is the same as the luminance component of backdrop lighting that has been reflected off said backdrop and is incident upon said subject.

45. An apparatus according to claim 37, wherein said subject imaging arrangement includes a backlighting configuration which illuminates said subject so that the chrominance component of the backlighting illumination is the complement of the chrominance component of the backdrop lighting that has been reflected off said backdrop and incident upon said subject.

46. An apparatus according to claim 37, wherein said subject imaging arrangement includes a subject lighting configuration which illuminates said subject so that the luminance component of the subject illumination is different from the luminance component of the backdrop lighting that has been reflected off said backdrop as imaged on said video camera.