Occupant detection system

• US 20050111700A1
• Filed: 10/04/2004
• Published: 05/26/2005
• Est. Priority Date: 10/03/2003
• Status: Active Grant

• Current Assignee: Joyson Safety Systems Acquisition, LLC
• Original Assignee: TK Holdings Incorporated (Ningbo Junsheng Investment Group Co., Ltd.)
• Inventors: Davies, Owen A., O'Boyle, Michael E., Schamp, Gregory G.
• Granted Patent: US 7,406,181 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01556 Child-seat detection systems
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A camera (32) captures successive images of light stripes (22) projected onto an object by a light curtain (18) positioned by a light source positioner (28). A background image is subtracted (616) therefrom, and the resulting image is boosted by binning (618), binarized with a thresholding algorithm (620), skeletonized (622), interpolated (624) and stored (626). Interpolated images are acquired for a plurality of light stripes (22). A processor (30) generates (1604) a 3-D surface model from Cartesian coordinates computed for non-zero camera pixels. A volumetric representation is determined (1610) from the offset of the object surface model relative to a model of a proximate surface, e.g. a seating surface (24). The object is classified (1614), e.g. by a trainable pattern recognitions system, responsive to 3-D shape descriptors (1606) of the 3-D surface model and to the volumetric representation (1610) or portions (1612) thereof. The detection of an occupant (14) in a vehicle (12) may be used to control a safety restraint system (36).

Claims

1. A method of detecting an object, comprising:

a. setting an exposure time for a process of generating an image signal;

b. generating an image signal representative of a first image of the object, wherein said image signal is generated by a camera at a first location, and said image signal is generated responsive to said exposure time;

c. projecting from a second location at least one beam of light across at least a portion of the object so as to provide for generating at least one image of at least one light stripe of a plurality of light stripes across said object, wherein said second location is displaced from said first location, an intensity of each light stripe of said plurality of light stripes is less than a first threshold, and said first threshold corresponds to an intensity level that is safe for direct viewing by a human eye;

d. generating at least one image signal representative of at least one second image from said camera, wherein said at least one second image is responsive to said at least one beam of light projected on said object;

e. subtracting said first image from said at least one second image so as to form at least one difference image signal of at least one difference image;

f. processing said at least one difference image so as generate a corresponding at least one fourth image, wherein a signal level of at least a portion of said at least one fourth image associated with said at least one light stripe in said at least one fourth image is boosted relative to a corresponding signal level in said at least one first image; and

g. determining at least one profile of said object responsive to said at least one fourth image.

2. A method of detecting an object as recited in claim 1, wherein said first location is proximate to a headliner of a vehicle.

3. A method of detecting an object as recited in claim 1, wherein said first location is proximate to a centerline of said vehicle.

4. A method of detecting an object as recited in claim 1, wherein said first image is generated absent said at least one beam of light projected on said object.

5. A method of detecting an object as recited in claim 1, further comprising adjusting said exposure time responsive to at least a portion of said first image so at to either reduce saturation or increase dynamic range of at least a portion of said first image.

6. A method of detecting an object as recited in claim 1, wherein said second location is proximate to a headliner of a vehicle.

7. A method of detecting an object as recited in claim 1, wherein at least one of said plurality of light stripes is generated by scanning said at least one beam of light.

8. A method of detecting an object as recited in claim 7, wherein said at least one light beam comprises a spot of light that is scanned in a plurality of directions.

9. A method of detecting an object as recited in claim 7, wherein said at least one light beam comprises a light curtain.

10. A method of detecting an object as recited in claim 9, wherein said light curtain is substantially planar, and said light curtain is scanned in a direction that is substantially transverse to said light curtain.

11. A method of detecting an object as recited in claim 10, wherein said light curtain is substantially elevationally oriented and is scanned in a substantially azimuthal direction.

12. A method of detecting an object as recited in claim 10, wherein said light curtain is substantially azimuthally oriented and is scanned in a substantially elevational direction.

13. A method of detecting an object as recited in claim 1, wherein said plurality of light stripes are generated responsive to an intermittent activation of a light source.

14. A method of detecting an object as recited in claim 1, wherein said plurality of light stripes are generated responsive to an intermittent activation of said camera or a shutter associated with said camera.

15. A method of detecting an object as recited in claim 1, wherein said a least one beam of light comprises near-infrared light, and the operations of capturing said first and second images comprise capturing light that is within a frequency range that includes at least one frequency of said near-infrared light.

16. A method of detecting an object as recited in claim 1, wherein the operation of processing said at least one difference image so as generate a corresponding at least one fourth image comprises:

a. forming a binned image value of a binned image by accumulating a plurality of pixel values within a region of said difference image; and

b. repeating the operation of forming said binned image value for a plurality of adjacent regions of said difference image.

17. A method of detecting an object as recited in claim 1, further comprising binarizing said at least one fourth image prior to the operation of determining said at least one profile of said object, wherein the operation of binarizing is responsive to a threshold.

18. A method of detecting an object as recited in claim 1, further comprising skeletonizing said at least one fourth image prior to the operation of determining said at least one profile of said object.

19. A method of detecting an object as recited in claim 1, further comprising interpolating said at least one fourth image prior to the operation of determining said at least one profile of said object, wherein the operation of interpolating said at least one fourth image provides for filling gaps that may exist in said at least one light stripe within said at least one fourth image.

20. A method of detecting an object as recited in claim 1, wherein a plurality of said at least one fourth images corresponding to a plurality of said light stripes are stored as a composite fourth image, and the operation of determining said at least one profile of said object is responsive to said composite fourth image.

21. A method of detecting an object as recited in claim 1, wherein the operation of determining said at least one profile comprises determining object space coordinates of at least one non-zero pixel of said at least one fourth image.

22. A method of detecting an object as recited in claim 1, further comprising generating a characterization of at least one proximate surface proximate to which the object is located.

23. A method of detecting an object as recited in claim 22, wherein said object is located on a seat of a vehicle, and the operation of generating a characterization of said at least one proximate surface proximate to which said object is located comprises retrieving at least one representation of said at least one surface responsive to at least one signal representative of a position of said seat.

24. A method of detecting an object as recited in claim 22, wherein the operation of generating a characterization of said at least one proximate surface proximate to which said object is located is responsive to a profile determined from a portion of said at least one fourth image, wherein said portion of said at least one fourth image is representative of at least a portion of said at least one proximate surface proximate to which said object is located.

25. A method of detecting an object as recited in claim 1, further comprising:

a. determining at least one feature of said object responsive to said profile, and

b. classifying said object responsive to said at least one feature.

26. A method of detecting an object as recited in claim 25, further comprising:

a. generating a representation of a surface of said object illuminated by said at least one beam of light, wherein said representation of said surface is relative to said at least one proximate surface, and

b. generating a volumetric representation of said object responsive to said representation of said surface of said object, wherein the operation of classifying said object is further responsive to said volumetric representation of said object.

27. A method of detecting an object as recited in claim 26, wherein said volumetric representation is sectioned into first and second volumetric representations corresponding to first and second portions of said object, and the operation of classifying said object is further responsive to said first and second volumetric representations of said object.

28. A method of detecting an object as recited in claim 27, wherein said object is positioned proximate to a seat of a vehicle, said first volumetric representation is responsive to a volume of said object proximate to an upper portion of said seat, and said second volumetric representation is responsive to a volume of said object proximate to a lower portion of said sea.

29. A method of detecting an object as recited in claim 1, wherein the operation of classifying said object is responsive to a plurality of said features.

30. A method of detecting an object as recited in claim 1, wherein the operation of classifying said object is performed at least in part by a trainable pattern recognition system.

31. A method of detecting an object as recited in claim 30, wherein said trainable pattern recognition system comprises a back-propagated neural network.

32. A method of detecting an object as recited in claim 30, wherein said trainable pattern recognition system is trained to distinguish between at two classes selected from a rear facing infant seat, a forward facing infant seat, a small child, a normally seated occupant, and empty seat and an out-of-position occupant.

33. A method of detecting an object as recited in claim 30, wherein said trainable pattern recognition system is further trained to distinguish between at least two classes selected from a normally seated small child, a normally seated a fifth percentile female, a normally seated a fiftieth percentile male and a normally seated ninety-fifth percentile male.

34. A method of detecting an object as recited in claim 30, wherein said trainable pattern recognition system is further trained to distinguish between at least two classes selected from an out-of-position occupant with at least one hand in a predetermined avoidance zone, an out-of-position occupant whose head is in a predetermined avoidance zone, and an out-of-position occupant with at least one foot in a predetermined avoidance zone.

35. A method of detecting an object as recited in claim 25, further comprising controlling a safety restraint system of a vehicle responsive to the operation of classifying said object.

42. A method of providing for detecting an object as recited in claim 1, further comprising:

a. providing for determining at least one feature of said object responsive to said profile, and

b. providing for classifying said object responsive to said at least one feature.

43. A method of providing for detecting an object as recited in claim 42, further comprising:

a. providing for generating a representation of a surface of said object illuminated by said at least one beam of light, wherein said representation of said surface is relative to said at least one proximate surface, and

b. providing for generating a volumetric representation of said object responsive to said representation of said surface of said object, wherein the operation of classifying said object is further responsive to said volumetric representation of said object.

36. A method of providing for detecting an object, comprising:

a. providing for setting an exposure time for a process of generating an image signal;

b. providing for generating an image signal representative of a first image of the object, wherein said image signal is generated by a camera at a first location, and said image signal is generated responsive to said exposure time;

c. providing for projecting from a second location at least one beam of light across at least a portion of the object so as to provide for generating at least one image of at least one light stripe of a plurality of light stripes across said object, wherein said second location is displaced from said first location, an intensity of each light stripe of said plurality of light stripes is less than a first threshold, and said first threshold corresponds to an intensity level that is safe for direct viewing by a human eye;

d. providing for generating at least one image signal representative of at least one second image from said camera, wherein said at least one second image is responsive to said at least one beam of light projected on said object;

e. providing for subtracting said first image from said at least one second image so as to form at least one difference image signal of at least one difference image;

f. providing for processing said at least one difference image so as generate a corresponding at least one fourth image, wherein a signal level of at least a portion of said at least one fourth image associated with said at least one light stripe in said at least one fourth image is boosted relative to a corresponding signal level in said at least one first image; and

g. providing for determining at least one profile of said object responsive to said at least one fourth image.

37. A method of providing for detecting an object as recited in claim 36, further comprising providing for adjusting said exposure time responsive to at least a portion of said first image so at to either reduce saturation or increase dynamic range of at least a portion of said first image.

38. A method of providing for detecting an object as recited in claim 36, further comprising providing for binarizing said at least one fourth image prior to the operation of determining said at least one profile of said object, wherein the operation of binarizing is responsive to a threshold.

39. A method of providing for detecting an object as recited in claim 36, further comprising providing for skeletonizing said at least one fourth image prior to the operation of determining said at least one profile of said object.

40. A method of providing for detecting an object as recited in claim 36, further comprising providing for interpolating said at least one fourth image prior to the operation of determining said at least one profile of said object, wherein the operation of interpolating said at least one fourth image provides for filling gaps that may exist in said at least one light stripe within said at least one fourth image.

41. A method of providing for detecting an object as recited in claim 36, further comprising providing for generating a characterization of at least one proximate surface proximate to which the object is located.

44. A method of providing for detecting an object as recited in claim 36, further comprising providing for controlling a safety restraint system of a vehicle responsive to the operation of classifying said object.