Systems and methods for monitoring vehicle occupants

• US 9,517,679 B2
• Filed: 12/04/2013
• Issued: 12/13/2016
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Frank, Jeffrey D., Richards, Austin A., White, Victoria L., Fairfield, Nile E., Stout, Arthur, Lee, David W., Hgasten, Nicholas, Hoelter, Theodore R., Strandemar, Katrin
• Primary Examiner(s): Elahi, Shan
• Application Number:
  US14/097,197
• Publication Number:
  US 20140093133A1
• Time in Patent Office: 1,105 Days
• Field of Search: 348/148
• US Class Current: 1/1
• CPC Class Codes: B60H 1/00742 by detection of the vehicle occupants' presence; by detection of conditions relating to the body of occupants, e.g. using radiant heat detectors
• CPC Class Codes: B60R 21/015 including means for detecting the presence or position of passengers, passenger seats or child seats , and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
• CPC Class Codes: B60R 21/01532 using electric or capacitive field sensors
• CPC Class Codes: B60R 21/0154 in combination with seat heating
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 5/32 Transforming X-rays cameras or camera modules for generating image signals from X-rays H04N23/30; circuitry of SSIS for transforming X-rays into image signals H04N25/30
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques are disclosed for systems and methods using small form factor infrared imaging modules to monitor occupants in an interior compartment of a vehicle. For example, a vehicle-mounted system may include one or more infrared imaging modules, a processor, a memory, alarm sirens, and a communication module. The vehicle-mounted system may be mounted on, installed in, or otherwise integrated into a vehicle with an interior compartment. The infrared imaging modules may be configured to capture thermal images of desired portions of the interior compartments. Various thermal image processing and analytics may be performed on the captured thermal images to determine the presence and various attributes of one or more occupants. Based on the determination of the presence and various attributes, occupant detection information or control signals may be generated. Occupant detection information may be used to perform various monitoring operations, and control signals may adjust various vehicle components.

Claims

1. A vehicle comprising:

an interior compartment;

an adjustable component responsive to a control signal;

an infrared imaging device comprising a focal plane array (FPA) and configured to capture thermal images comprising pixels representing thermal radiation variations in at least a portion of the interior compartment; and

a processor configured to;

analyze the thermal radiation variations represented in the pixels of the thermal images;

detect one or more regions-of-interest (ROIs) in the thermal images based on the analysis of the thermal radiation variations;

analyze geometric and thermal properties of the one or more detected ROIs;

determine, based on the analysis of the geometric and thermal properties of the detected ROIs, a count of one or more occupants present in the portion of the interior compartment and approximate sizes and/or positions of the one or more occupants, and

generate the control signal based on the determination of the count and the approximate sizes and/or the positions of the one or more occupants.

2. The vehicle of claim 1, wherein:

the adjustable component is an airbag assembly configured to deploy one or more airbags according to one or more deployment parameters; and

the control signal is configured to adjust the deployment parameters including whether to enable or disable airbag deployment.

3. The vehicle of claim 2, wherein the one or more deployment parameters determine timing and/or intensity of airbag deployment based at least in part on the positions and/or the sizes of the one or more occupants.

4. The vehicle of claim 1, wherein the adjustable component is a motorized adjuster configured to adjust a position of at least one of a seat, a steering column, or a pedal in response to the control signal.

5. The vehicle of claim 1, wherein:

the adjustable component is a heating, ventilation, and air conditioning (HVAC) system configured to control a climate of the interior compartment according to climate settings;

and

the control signal is configured to adjust the climate settings of the HVAC system based at least in part on the count and the positions of the one or more occupants present in the portion of the interior compartment.

6. The vehicle of claim 1, wherein:

the thermal images are unblurred thermal images of the at least a portion of the interior compartment;

the infrared imaging device is configured to capture intentionally blurred thermal images of the at least a portion of the interior compartment; and

the processor is configured to determine a plurality of non-uniformity correction (NUC) terms based on the intentionally blurred thermal images and apply the NUC terms to the unblurred thermal images to remove noise form the unblurred thermal images.

7. The vehicle of claim 1, wherein the processor is configured to determine the count and the approximate sizes and/or the positions of the one or more occupants using the thermal images and without input from additional sensors.

8. The vehicle of claim 1, wherein:

the processor is configured to analyze the thermal radiation variations represented in the pixels of the thermal images at least by comparing the thermal radiation variations against a surface temperature range of a person;

the geometric property of the detected ROIs being analyzed comprises a shape and/or a size of the detected ROIs; and

the thermal property of the detected ROIs being analyzed comprises a surface temperature of the detected ROIs.

9. The vehicle of claim 1, wherein:

the thermal images are radiometrically normalized thermal images such that the pixels represent calibrated temperature data; and

the infrared imaging device comprises a shutter used as a temperature reference in a periodic calibration process for radiometric normalization of the thermal images.

10. A method comprising:

capturing, using a focal plane array (FPA) of an infrared imaging device, thermal images comprising pixels representing thermal radiation variations in at least a portion of an interior compartment of a vehicle, wherein the infrared imaging device is mounted in or on the vehicle so that the at least a portion of the interior compartment is within a field of view (FOV) of the infrared imaging device;

analyzing the thermal radiation variations represented in the pixels of the thermal images;

detecting one or more regions-of-interest (ROIs) in the thermal images based on the analysis of the thermal radiation variations;

analyzing geometric and thermal properties of the one or more detected ROIs;

determining, by a processor based on the analysis of the geometric and thermal properties of the detected ROIs, a count of one or more occupants present in the portion of the interior compartment and approximate sizes and/or positions of the one or more occupants;

generating, by the processor, a control signal for an adjustable component of the vehicle based on the determination of the count and the approximate sizes and/or the positions of the one or more occupants; and

providing the control signal to the adjustable component.

11. The method of claim 10, wherein:

the adjustable component is an airbag assembly configured to deploy one or more airbags according to one or more deployment parameters; and

the generating comprises configuring the control signal to adjust the deployment parameters including whether to enable or disable airbag deployment.

12. The method of claim 11, wherein the one or more deployment parameters determine timing and/or intensity of airbag deployment based at least in part on the positions and/or the sizes of the one or more occupants.

13. The method of claim 10, wherein:

the adjustable component is a seatbelt tensioner configured to provide appropriate tension to a seatbelt according to one or more parameters; and

the generating comprises configuring the control signal to adjust the parameters.

14. The method of claim 10, wherein:

the adjustable component is a heating, ventilation, and air conditioning (HVAC) system configured to control a climate of the interior compartment according to climate settings;

and

the generating of the control signal comprises configuring the control signal to adjust the climate settings of the HVAC system based at least in part on the count and the positions of the one or more occupants present in the portion of the interior compartment.

15. The method of claim 10, further comprising transmitting wirelessly the thermal images captured by the infrared imaging device to the processor.

16. The method of claim 10, wherein the thermal images are unblurred thermal images, the method further comprising:

capturing intentionally blurred thermal images of the at least a portion of the interior compartment;

determining a plurality of non-uniformity correction (NUC) terms based on the intentionally blurred thermal images; and

applying the NUC terms to the unblurred thermal images to remove noise from the unblurred thermal images.

17. The method of claim 10, wherein the determining of the count and the approximate sizes and/or the positions of the one or more occupants is performed using the thermal images and without input from additional sensors.

18. The method of claim 10, wherein:

the analyzing of the thermal radiation variations represented in the pixels of the thermal images comprises comparing the thermal radiation variations against a surface temperature range of a person;

the geometric property of the detected ROIs being analyzed comprises a shape and/or a size of the detected ROIs; and

the thermal property of the detected ROIs being analyzed comprises a surface temperature of the detected ROIs.

19. The method of claim 10, wherein:

the thermal images are radiometrically normalized thermal images such that the pixels represent calibrated temperature data; and

the method further comprises periodically and automatically calibrating the infrared imaging device for radiometric normalization of the thermal images.

20. A vehicle-mountable infrared camera system comprising:

an infrared imaging device configured to be mounted in an interior compartment of a vehicle, the infrared imaging device comprising a focal plane array (FPA) and configured to capture thermal images comprising thermal radiation variations in a portion of the interior compartment within a field of view (FOV);

a processor configured to;

analyze the thermal radiation variations represented in pixels of the thermal images;

detect one or more regions-of-interest (ROIs) in the thermal images based on the analysis of the thermal radiation variations;

analyze geometric and thermal properties of the one or more detected ROIs;

determine, based on the analysis of the geometric and thermal properties, a count of one or more occupants present in the portion of the interior compartment and approximate sizes and/or positions of the one or more occupants; and

generate a control signal for an adjustable component of the vehicle based on the determination of the count and the approximate sizes and/or the positions of the one or more occupants; and

a network interface controller (NIC) configured to manage wired and/or wireless communications with the adjustable component of the vehicle, wherein the processor is further configured to transmit the control signal to the adjustable component via the NIC.