Image segmentation system and method

• US 7,116,800 B2
• Filed: 12/17/2001
• Issued: 10/03/2006
• Est. Priority Date: 05/30/2001
• Status: Expired due to Fees

• Current Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Original Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Inventors: Farmer, Michael Edward
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Seth, Manav
• Application Number:
  US10/023,787
• Publication Number:
  US 20030031345A1
• Time in Patent Office: 1,751 Days
• Field of Search: 382/103, 382/270, 382171-173, 382/164, 382/257, 382/154, 382/207, 358464-466, 358/538, 348/169
• US Class Current: 382/103
• CPC Class Codes: B60R 2021/0044 Chest
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01556 Child-seat detection systems
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/765 using rules for classification or partitioning the feature space
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

The present invention relates in general to systems used to process images. In particular, the present invention is an image segmentation system and method used to isolate the segmented image of a target person, animal, or object from an ambient image which includes the target person, animal, or object, in addition to the area surrounding the target. The invention supports the ability of an airbag deployment system to distinguish between different types of occupants by providing such deployment systems with a segmented image of the occupant. The invention is particularly useful at night or in other environments involving inadequate light or undesirable shadows. The invention can use histograms and cumulative distribution functions to perform image thresholding. Morphological erosion and dilation can be used to eliminate optical “noise” from the image. Gap filling is performed on the basis of the “momentum” and “gravity” of regions of similar pixel values. An upper ellipse can generated to represent the upper torso of the occupant. The invention is highly flexible, and can be modified in accordance with the desired use.

Claims

1. An image segmentation meted for use with an occupant, a sensor for generating sensor measurements, and an ambient image including an occupant and the area surrounding the occupant, said image segmentation method comprising the following steps:

receiving an ambient image represented by a plurality of pixels and a plurality of initial pixel values, wherein each said pixel has at least one said initial pixel value;

identifying one or more pixels as belonging to one of a plurality of pixel categories on the basis of one or more initial pixel values associated with the pixels being identified, wherein identifying one or more said pixels further includes generating an image threshold;

establishing a first revised pixel value for one or more pixels, wherein each pixel in the same pixel category has the same first revised pixel value;

setting a second revised pixel value for one or more of said pixels on the basis of one or more first revised pixel values that are associated with one or more pixels in the vicinity of the pixel being set, wherein setting the second revised pixel value includes a momentum-based heuristic; and

deriving a segmented image of the occupant from said first revised pixel value and said second revised pixel value.

2. An image segmentation method as in claim 1, wherein identifying one or more said pixels further includes comparing said plurality of initial pixel values to said image threshold.

3. An image segmentation method as in claim 1, wherein generating an image threshold further comprises analyzing the distribution of initial pixel values relating to a pixel characteristic.

4. An image segmentation method as in claim 3, wherein analyzing the distribution of initial pixel values further includes:

recording aggregate initial pixel values into a histogram;

translating the histogram into a cumulative distribution function; and

calculating an image threshold based on a predetermined percentage of initial pixel values falling below the image threshold.

5. An image segmentation method as in claim 3, wherein each pixel has only one initial pixel value and only one pixel characteristic.

6. An image segmentation method as in claim 5, wherein luminosity is said pixel characteristic.

7. An image segmentation method as in claim 2, wherein each said pixel has a pixel location with respect to the ambient image, and said pixel location determines which of a plurality of image thresholds are compared to said initial pixel value for said pixel in said pixel location.

8. An image segmentation method as in claim 7, wherein a higher image threshold is applied in pixel locations where there is brighter lighting.

9. An image segmentation method as in claim 1, wherein there are only two pixel categories.

10. An image segmentation method as in claim 1, wherein setting the second revised pixel value includes a morphological heuristic.

11. An image segmentation method as in claim 10, wherein the morphological heuristic is a morphological erosion.

12. An image segmentation method as in claim 11, wherein the morphological heuristic is a morphological dilation.

13. An image segmentation method as in claim 1, wherein a subset of said plurality of pixels are a group of adjacent pixels, wherein said plurality of pixel categories includes a first pixel category and a second pixel category, and wherein setting the second pixel value further comprises:

analyzing in a sequential manner the subset of pixels in the group of adjacent pixels;

adding to a counter value each time the next pixel in the sequence belongs to said first pixel category; and

subtracting from said counter value each time the next pixel in the sequence belongs to said second pixel category.

14. An image segmentation method as in claim 13, wherein the momentum-based heuristic stops when the counter value is less than or equal to zero.

36. An image segmentation method as in claim 7, wherein said plurality of image Thresholds includes a first image threshold and a second image threshold, said second image threshold being lower than said first image threshold, and wherein said first image threshold is associated with an upper area of the ambient image and said second image threshold is associated with a middle area of the ambient image.

37. An image segmentation method as in claim 36, wherein said plurality of image thresholds includes a third image threshold, said third image threshold being lower than said second image threshold, and wherein said third image threshold is associated with a bottom area of the ambient image.

40. An image segmentation method as in claim 1, wherein setting the second revised pixel value includes a gravity-based heuristic.

41. An image segmentation method as in claim 1, wherein the one or more pixels the vicinity of the pixel being set includes a subset of sequential pixels, said step of setting the second revised pixel value including using the momentum-based heuristic to determine the second revised pixel value based on the first revised pixel values associated with the subset of sequential pixels.

42. An image segmentation method as in claim 41, wherein the subset of sequential pixels is either horizontally sequential or vertically sequential.

43. An image segmentation method as in claim 41, said step of setting the second revised pixel value including:

sequentially traversing the subset of sequential pixels to identify the first revised values associated with the subset of sequential pixels;

incrementing or decrementing a counter value based on the first revised value associated with each of the sequential pixels; and

determining the second revised pixel value based on the counter value.

15. An image segmentation method for use with an occupant, a sensor for generating sensor measurements, and an ambient image of an occupant and the area surrounding the occupant said image segmentation method comprising the following steps:

receiving an ambient image represented by a plurality of pixels and a plurality of pixel values, wherein each of said pixels has at least one said initial pixel value;

recording aggregate initial pixel values in a histogram;

translating the histogram into a cumulative distribution function;

calculating an image threshold using the cumulative distribution function with a predetermined percentage of initial pixel values falling below the image threshold;

categorizing each pixel in the plurality of pixels into one of a plurality of pixel categories by comparing the image threshold to the initial pixel value for the pixel being categorized;

establisbing a first revised pixel value so that each pixel in the same pixel category shares the same first revised pixel value;

modifying said first revised pixel value into a second revised pixel value in accordance with morphological heuristic;

determining a third revised pixel value from said second revised pixel value or said first revised pixel value with a momentum-based heuristic;

identifying regions of pixels based on the first revised pixel value, the second revised pixel value, and the third revised pixel value;

generating a fourth revised pixel value from the regions of pixels in accordance with a gravity-based heuristic; and

deriving a segmented image of the occupant with the fourth revised pixel value.

16. An image segmentation method for use with an occupant as in claim 15, wherein deriving a segmented image of the occupant includes substituting the initial pixel value for said fourth revised pixel value representing the occupant image.

17. An image segmentation system for use with an occupant, a sensor for generating sensor measurements, and an ambient image of an occupant and the area around the occupant, said image segmentation system comprising:

an image thresholding subsystem, including a plurality of pixels representing the ambient image, and an image thresholding heuristic, said image thresholding subsystem categorizing said plurality of pixels in accordance with said image thresholding heuristic; and

a gap processing subsystem, including a gap processing heuristic, a subset of vicinity pixels in said plurality of pixels, and a plurality of pixel values, said gap processing subsystem selectively setting said pixel values in accordance with said gap processing heuristic and said pixel values belonging to said pixels in said subset of vicinity pixels, said gap processing heuristic including a momentum-based heuristic; and

wherein a segmented image is generated from said plurality of pixels.

18. An image segmentation system as in claim 17, said image thresbolding subsystem comprising a plurality of luminosity values, wherein each said pixel has at least one said luminosity value.

19. An image segmentation system as in claim 18, said image thresholding subsystem further comprising a histogram, wherein said histogram tabulates the number of said pixels having said luminosity value.

20. An image segmentation system as in claim 19, said image thresholding subsystem further comprising a cumulative distribution curve, wherein said histogram is converted into said cumulative distribution curve by said image thresholding subsystem.

21. An image segmentation system as in claim 20, said image thresholding subsystem further comprising a predetermined percentage and an image threshold, wherein said image threshold is calculated from said predetermined percentage and said cumulative distribution curve.

22. An image segmentation system as in claim 21, said image thresholding subsystem comprising a plurality of image thresholds and said pixel includes a pixel location, wherein said pixel location for said pixel determines which said image threshold is used for said pixel by said image thresholding subsystem.

23. An image segmentation system as in claim 21, said image thresholding subsystem further including a first subset of said plurality of pixels and a second subset of said plurality of pixels, said image thresholding subsystem dividing said plurality of pixels into said first subset of pixels and said second subset of pixels wit said image threshold.

24. An image segmentation system as in claim 23, said image thresholding subsystem further including a first binary value and a second binary value, wherein said first subset of said plurality of pixels is set to said first binary value and said second subset of said plurality of pixels is set to said second binary value.

25. An image segmentation system as in claim 24 wherein at least approximately half of said plurality of said pixels arc set to said first binary value.

26. An image segmentation system as in claim 17, said gap processing heuristic including a morphological heuristic.

27. An image segmentation system as in claim 26, said morphological heuristic comprising a morphological erosion.

28. An image segmentation system as in claim 26, said morphological heuristic comprising a morphological dilation.

29. An image segmentation system as in claim 26, said morphological heuristic comprising a vertically-based morphological heuristic.

30. An image segmentation system as in claim 26, said morphological heuristic comprising a horizontally-based morphological heuristic.

31. An image segmentation system as in claim 17, said subset of vicinity pixels comprising a subset of sequential pixels, arid said gap processing subsystem selectively setting said pixel value using said momentum-based heuristic and said pixel values relating to said subset of sequential pixels.

32. An image segmentation system as in claim 31, said sequential subset of pixels including a sequential vertical subset of pixels and a sequential horizontal subset of pixels, wherein said momentum-based heuristic analyzes said pixel values in said sequential vertical subset of pixels and said sequential horizontal subset of pixels, to determine said plurality of pixel values.

33. An image segmentation system as in claim 32, said momentum-based heuristic further comprising a momentum counter, said momentum counter determining when said gap processing subsystem terminates said momentum-based heuristic for said pixel value.

34. An image segmentation system as in claim 24

said gap processing heuristic comprising a gravity-based heuristic, a pixel region and a region characteristic;

said plurality of pixels including a target pixel;

wherein said pixel region includes one or more said pixels;

wherein said pixel region does not include said target pixel; and

wherein said pixel region is a subset of said vicinity pixels.

35. An image segmentation system as in claim 34

said gravity-based heuristic further including a region size and a region distance;

wherein said region size is the number of said pixels in said pixel region;

wherein said region distance is the distance between said target pixel and a center point in said pixel region; and

wherein said gap processing subsystem sets said target pixel in accordance with said region size, said region distance, and said gravity-based heuristic.

38. An image segmentation system as in claim 22, wherein said plurality of image thresholds includes a first image threshold and a second image threshold, said second image threshold being lower than said first image threshold, and wherein said first image threshold is associated wit an upper area of the ambient image and said second image threshold is associated with a middle area of the ambient image.

39. An image segmentation system as in claim 38, wherein said plurality of image thresholds includes a third image threshold, said third image threshold being lower than said second image Threshold, and wherein said third image threshold is associated with a bottom area of the ambient image.