Motion-based segmentor detecting vehicle occupants using optical flow method to remove effects of illumination

• US 20050129274A1
• Filed: 09/16/2004
• Published: 06/16/2005
• Est. Priority Date: 05/30/2001
• Status: Abandoned Application

• Current Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Original Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Inventors: Farmer, Michael, Wen, Li
• Application Number:
  US10/944,482
• Publication Number:
  US 20050129274A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 2021/0044 Chest
• CPC Class Codes: B60R 2021/01315 monitoring occupant displacement
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01556 Child-seat detection systems
• CPC Class Codes: B60R 21/01558 monitoring crash strength
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

An image segmentation method and apparatus are described. The inventive system and apparatus generates a segmented image of an occupant or other target of interest based upon an ambient image, which includes the target and the environment in the vehicle that surrounds the target. The inventive concept defines a bounding ellipse for the target. This ellipse may be provided to a processing system that performs tracking of the target. In one embodiment, an optical flow technique is used to compute motion and illumination field values. The explicit computation of the effects of illumination dramatically improves motion estimation and thereby facilitates computation of the bounding ellipses.

Claims

1. A method for isolating a current segmented image from a current ambient image, comprising the steps of:

a) computing a directional gradient image for the current ambient image;

b) computing a time difference image, wherein the time difference image comprises a difference between the current ambient image and a prior ambient image;

c) computing an optical flow field image and an illumination field image, responsive to the directional gradient image, the time difference image, and the current ambient image; and

d) performing an adaptive threshold computation on the optical flow field image thereby generating a binary image, wherein the current segmented image corresponds to and is associated with at least part of the binary image.

2. The method of claim 1, further comprising the step of computing ellipse parameters for a bounding ellipse corresponding to and associated with at least part of the binary image.

3. The method of claim 1, wherein the current ambient image is a subset of a larger current ambient image.

4. The method of claim 3, further comprising the steps of:

a) determining a region of interest within the larger current ambient image; and

b) selecting the subset of the larger current ambient image responsive to the region of interest.

5. The method of claim 4, further comprising the steps of:

a) receiving projected ellipse parameters, wherein the projected ellipse parameters are responsive to at least one prior segmented image;

b) computing a projected bounding ellipse corresponding to and associated with the projected ellipse parameters; and

c) determining the region of interest responsive to the projected bounding ellipse.

6. The method of claim 1, wherein the step of computing the optical flow field image includes a summation procedure over a window W centered around each pixel in the current ambient image, and wherein the window W is a region encompassing at least 3 by 3 pixels.

7. The method of claim 1, wherein the optical flow field image includes velocity components for at least one coordinate direction.

8. The method of claim 1, wherein the illumination field image includes at least one of the following:

a) a multiplicative illumination field image, and b) an additive illumination field image.

9. The method of claim 1, wherein the step (d) of performing the adaptive threshold computation generating a binary image further comprises the steps of:

i) computing a histogram function, wherein the histogram function corresponds to and is associated with at least part of the optical flow field image;

ii) computing a Cumulative Distribution Function (CDF) based on the histogram function;

iii) setting a threshold level for the CDF; and

iv) generating the binary image responsive to the threshold level.

10. The method according to claim 9, further comprising the steps of:

v) computing central moments and lower order moments relating to the binary image; and

vi) computing bounding ellipse parameters corresponding to and associated with the central moments and the lower order moments.

11. The method according to claim 1, further comprising the step of smoothing the current ambient image.

12. A segmentation system for isolating a current segmented image from a current ambient image, comprising:

a) a camera, wherein the camera outputs a the current ambient image and a prior ambient image, and wherein the current ambient image includes the current segmented image;

b) a directional gradient and time difference module, wherein the directional gradient and time difference module generates a directional gradient image and a time difference image based on the current ambient image and the prior ambient image;

c) an optical flow module, wherein the optical flow module calculates and outputs an optical flow field image and an illumination field image; and

d) an adaptive threshold module, wherein the adaptive threshold module generates a binary image, and wherein the current segmented image corresponds to and is associated with at least part of the binary image.

13. The segmentation system of claim 12, further comprising an ellipse fitting module wherein the ellipse fitting module computes bounding ellipse parameters corresponding to and associated with, at least part of the binary image.

14. The segmentation system of claim 12, wherein the current ambient image is a subset of a larger current ambient image.

15. The segmentation system of claim 14, further comprising a region of interest module, wherein the region of interest module determines a region of interest image, and wherein the subset of the larger current ambient image is generated responsive to the region of interest image.

16. The segmentation system of claim 12, wherein the illumination field image includes at least one of the following:

a) a multiplicative illumination field, and b) an additive illumination field.

17. The segmentation system of claim 12, further comprising an image smoothing module, wherein the image smoothing module smoothes the current ambient image to reduce effects of noise present in the current ambient image.

18. A segmentation system for isolating a current segmented image from a current ambient image, comprising:

a) means for computing a directional gradient image for the current ambient image;

b) means for computing a time difference image, wherein the time difference image comprises a difference between the current ambient image and a prior ambient image;

c) means for computing an optical flow field image and an illumination field image, responsive to the directional gradient image, the time difference image, and the current ambient image; and

a) means for performing an adaptive threshold computation on the optical flow field image thereby generating a binary image, wherein the current segmented image corresponds to and is associated with at least part of the binary image.

19. A computer program, executable on a general purpose computer, comprising:

a) a first set of instructions for computing a directional gradient image for the current ambient image;

b) a second set of instructions for computing a time difference image, wherein the time difference image comprises a difference between the current ambient image and a prior ambient image;

c) a third set of instructions for computing an optical flow field image and an illumination field image, responsive to the directional gradient image, the time difference image, and the current ambient image; and

d) a fourth set of instructions for performing an adaptive threshold computation on the optical flow field image thereby generating a binary image, wherein the current segmented image corresponds to and is associated with at least part of the binary image.