System and method for segmentation of images of objects that are occluded by a semi-transparent material

• US 6,668,078 B1
• Filed: 09/29/2000
• Issued: 12/23/2003
• Est. Priority Date: 09/29/2000
• Status: Active Grant

• Current Assignee: Globalfoundries US Incorporated (GlobalFoundries, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Bolle, Rudolf M., Connell, Jonathan
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Alavi, Amir
• Application Number:
  US09/676,883
• Time in Patent Office: 1,180 Days
• Field of Search: 382/162, 382/164, 382/165, 382/110, 382/170, 382/171, 382/173, 382/167, 382/190, 382/218, 382/219, 382/274, 356/302, 356/303, 356/306, 356/307, 356/240.1, 356/237.1, 356/238.1, 356/239.3, 356/239.4, 356/239.7, 356/237.6, 356/328, 358/515, 358/518, 358/520, 358/464, 702/128
• US Class Current: 382/164
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An image processing system segments an object from the background of a scene where the scene is illuminated by unknown ambient light sources. An image of the scene with the object and an image of the background scene are captured by an image input device. These images are color corrected, using the image of a gray patch that is visible. A further transform converts the image from a red, green, blue format into a hue, saturation and intensity representation. The two images are then novelly compared on a pixel-by-pixel basis in the hue, saturation and intensity domain.

Claims

1. A computer imaging system, having one or more memories and one or more central processing units (CPU), the system segmenting images of one or more objects from a background image, the system comprising:

one or more image acquisition units that acquire the background image and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

an intensity transform that isolates a light intensity component from each of the combined image and background image, so that a relative reflectance value for each of the spectral bands in both the combined image and background image remains; and

a comparison process that performs a pixel by pixel comparison to determine a set of locations where the relative reflectance values differ between the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.

2. A system, as in claim 1, where the relative reflectance value includes any one or more of the following:

a hue, a saturation, a normalized color component, a combination of a hue and a saturation, a set of hues, a set of saturations, and a set of combined hues and saturations.

3. A system, as in claim 1, where the intensity transform includes any one or more of the following:

HSI, HSV, color component ratios, fractional color components, CIE L*u*v* hue angle and saturation, and CIE L*a*b* hue angle and saturation.

4. A computer imaging system, having one or more memories and one or more central processing units (CPU), the system segmenting images of one or more objects from a background image, the system comprising:

one or more image acquisition units that acquire the background imaged and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

a normalization function that normalizes the background image and the combined image so that corresponding pixels have the same response in both images;

an intensity transform that isolates a light intensity component from each of the combined image and background image, so that a pixel-wise reflectance value for each of the spectral bands in both the combined image and background image remains; and

a comparison process that performs a pixel by pixel comparison to determine a set of locations where the reflectance values are dissimilar in the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.

5. A system, as in claim 4, where the correspondence determination of the normalization function includes any one or more of the following:

a) pixels having the same image location in the combined image and the background image and the location having no image of the object, and b) a correspondence between two or more sets of pixel locations where the regions imaged are known to have the same response.

6. A system, as in claim 4, where the response includes any one or more of the follow:

a hue value, a saturation value, a combination of one or more hues and a saturation value, and a combination of one or more hues, a saturation, and an intensity value.

7. A system, as in claim 4, where the pixel-wise reflectance value includes any one or more of the following:

a hue, a saturation, a normalized color component, a combination of a hue and a saturation, a set of hues, and a set of combined hues and a saturation.

8. A system, as in claim 4, where the intensity transform includes any one or more of the following:

HSI, HSV, color component ratios, fractional color components, CIE L*u*v* hue angle and saturation, and CIE L*a*b* hue angle and saturation.

9. A computer imaging system, having one or more memories and one or more central processing units (CPU), the system segmenting images of one or more objects from a background image, the system comprising:

one or more image acquisition units that acquire the background imaged and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

a normalization function that normalizes the background image and the combined image so that corresponding pixels have the same response in both images;

an intensity transform that isolates a light intensity component from each of the combined image and background image, so that a pixel-wise reflectance value for each of the spectral bands in both the combined image and background image remains, the intensity transform also providing an intensity value for the associated pixel; and

a comparison process that performs a pixel by pixel comparison to determine a set of locations where any of the reflectance values and associated intensity value are dissimilar in the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.

10. A system, as in claim 9, further comprising a cleaning process that removes pixels from the set of locations where the saturation value of the pixels is different from an average saturation value of the pixels in the set by more than a saturation threshold.

11. A system, as in claim 9, further comprising a cleaning process that removes pixels from the set of locations where the intensity value of the pixels is different from an average intensity value of the pixels in the set by more than an intensity threshold.

12. A system, as in claim 9, further comprising a smoothing process that adds pixels to the set of locations where local density of foreground pixels exceeds a first smoothing threshold in order to smooth the boundary.

13. A system, as in claim 9, comprising a smoothing process that removes pixels from the set of locations where local density of foreground pixels falls below a second smoothing threshold in order to smooth the boundary.

14. A system, as in claim 9, further comprising a connecting process comprising the steps of:

determining a set of foreground connected components on the set of locations;

determining the area of each of the foreground connected components; and

removing any component that has an area less than a percentage of the area of the largest component in order to eliminate one or more outlying portions of the set of locations.

15. A system, as in claim 9, further comprising a connecting process comprising the steps of:

determining a set of foreground and background connected components on the set of locations;

determining the area of the foreground and background connected components; and

adding to the foreground set of locations any background component that has an area less than a percentage of the area of the largest foreground component in order to fill in one or more holes in the set of locations.

16. A method executing on a computer imaging system, the method comprising the steps of:

acquiring the background image and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

isolating a light intensity component from each of the combined image and background image, so that a relative reflectance value for each of the spectral bands in both the combined image and background image remains; and

performing a pixel by pixel comparison to determine a set of locations where the relative reflectance values differ between the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.

17. A computer imaging system, having one or more memories and one or more central processing units (CPU), the system segmenting images of one or more objects from a background image, the system comprising:

means for acquiring the background image and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

means for isolating a light intensity component from each of the combined image and background image, so that a relative reflectance value for each of the spectral bands in both the combined image and background image remains; and

means for performing a pixel by pixel comparison to determine a set of locations where the relative reflectance values that differ between the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.

18. A computer program product having a method comprising the steps of:

acquiring the background imaged and a combined image of one or more of the objects in front of the background image, the background image and the combined image having two or more spectral bands and being stored in one or more of the memories;

isolating a light intensity component from each of the combined image and background image, so that a relative reflectance value for each of the spectral bands in both the combined image and background image remains; and

performing a pixel by pixel comparison to determine a set of locations where the relative reflectance values differ between the combined image and background image, the set of locations representing a portion of the combined image corresponding to the objects without the background.