Systems and methods for monitoring vehicle occupants

• US 9,998,697 B2
• Filed: 12/09/2016
• Issued: 06/12/2018
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Frank, Jeffrey D., Richards, Austin A., White, Victoria L., Fairfield, Nile E., Stout, Arthur, Lee, David W., Hgasten, Nicholas, Hoelter, Theodore R., Strandemar, Katrin
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/375,069
• Publication Number:
  US 20170088098A1
• Time in Patent Office: 550 Days
• Field of Search: 348148
• CPC Class Codes: B60H 1/00742 by detection of the vehicle occupants' presence; by detection of conditions relating to the body of occupants, e.g. using radiant heat detectors
• CPC Class Codes: B60H 1/00778 the input being a stationary vehicle position, e.g. parking or stopping
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques are disclosed for systems and methods using small form factor infrared imaging modules to monitor occupants in an interior compartment of a vehicle. For example, a vehicle-mounted system may include one or more infrared imaging modules, a processor, a memory, alarm sirens, and a communication module. The vehicle-mounted system may be mounted on, installed in, or otherwise integrated into a vehicle with an interior compartment. The infrared imaging modules may be configured to capture thermal images of desired portions of the interior compartments. Various thermal image processing and analytics may be performed on the captured thermal images to determine the presence and various attributes of one or more occupants. Based on the determination of the presence and various attributes, occupant detection information or control signals may be generated. Occupant detection information may be used to perform various monitoring operations, and control signals may adjust various vehicle components.

Claims

1. A vehicle-mountable thermal imaging system comprising:

an infrared imaging device configured to be installed in an interior compartment of a vehicle and comprising a focal plane array (FPA) configured to capture thermal images comprising pixels representing thermal radiation variations in at least a portion of the interior compartment; and

a processor configured to;

analyze the thermal radiation variations represented in the pixels of the thermal images;

detect one or more regions-of-interest (ROIs) in the thermal images based on the analysis of the thermal radiation variations;

analyze geometric and thermal properties of the one or more detected ROIs;

determine, based on the analysis of the geometric and thermal properties of the detected ROIs, temperature profiles of one or more objects present in the portion of the interior compartment and approximate sizes and/or shapes of the one or more objects;

determine whether one or more unwanted occupants are present in the interior compartment based on the determination of the temperature profiles and the approximate sizes and/or shapes of the one or more objects.

2. The vehicle-mountable thermal imaging system of claim 1, wherein the processor is configured to identify infants, small children, small animals, or intruders as the one or more unwanted occupants.

3. The vehicle-mountable thermal imaging system of claim 1, wherein the processor is further configured to generate an alarm in response to determining that the one or more unwanted occupants are present in the interior compartment.

4. The vehicle-mountable thermal imaging system of claim 3, further comprising a network interface controller (NIC) configured to manage wireless communications with external devices, wherein the processor is further configured to transmit the alarm to at least one of an on-board security system, a key fob, a remote monitoring station, a mobile phone, or a networked computer via the NIC.

5. The vehicle-mountable thermal imaging system of claim 4, wherein the processor is further configured to transmit the thermal images to at least one of the on-board security system, key fob, remote monitoring station, mobile phone, or networked computer via the NIC to be converted into user-viewable thermal images for viewing by a user.

6. The vehicle-mountable thermal imaging system of claim 1, wherein the processor is further configured to:

determine a condition of the vehicle, including whether the vehicle is parked, moving, or in a collision; and

determine whether the one or more unwanted occupants are present in the interior compartment further based on the determined condition of the vehicle.

7. The vehicle-mountable thermal imaging system of claim 6, further comprising a network interface controller (NIC) configured to manage wireless communications with external devices, the processor is further configured to:

generate vehicle occupant information including corresponding positions of one or more occupants and a count of the one or more occupants based on the determination of the temperature profiles and the approximate sizes and/or shapes of the one or more objects; and

transmit, in response determining that the vehicle is in a collision, the generated vehicle occupant information to at least one of a remote monitoring station, a mobile phone, or a networked computer via the NIC.

8. The vehicle-mountable thermal imaging system of claim 6, wherein the processor is further configured to generate a vehicle disable signal to disable the vehicle in response to determining that the vehicle is moving and that the one or more unwanted occupants are present.

9. The vehicle-mountable thermal imaging system of claim 1, wherein the thermal images captured by the infrared imaging device are radiometrically normalized thermal images such that the pixels represent calibrated temperature data.

10. The vehicle-mountable thermal imaging system of claim 8, the infrared imaging device comprises a shutter used as a temperature reference in a periodic calibration process for radiometric normalization of captured thermal images.

11. A vehicle comprising the vehicle-mountable thermal imaging system of claim 1.

12. A method of monitoring an interior compartment of a vehicle, the method comprising:

capturing, using a focal plane array (FPA) of an infrared imaging device, thermal images comprising pixels representing thermal radiation variations in at least a portion of an interior compartment of a vehicle, wherein the infrared imaging device is mounted in or on the vehicle so that the at least a portion of the interior compartment is within a field of view (FOV) of the infrared imaging device;

analyzing the thermal radiation variations represented in the pixels of the thermal images;

detecting one or more regions-of-interest (ROIs) in the thermal images based on the analysis of the thermal radiation variations;

analyzing geometric and thermal properties of the one or more detected ROIs;

determining, based on the analysis of the geometric and thermal properties of the detected ROIs, temperature profiles of one or more objects present in the portion of the interior compartment and approximate sizes and/or shapes of the one or more objects; and

determining whether one or more unwanted occupants are present in the interior compartment based on the determination of the temperature profiles and the approximate sizes and/or shapes of the one or more objects.

13. The method of claim 12, wherein the determining of whether one or more unwanted occupants are present comprises identifying infants, small children, small animals, or intruders as the one or more unwanted occupants.

14. The method of claim 12, further comprising generating an alarm in response to determining that the one or more unwanted occupants are present in the interior compartment.

15. The method of claim 13, further comprising wirelessly transmitting the alarm to at least one of an on-board security system, a key fob, a remote monitoring station, a mobile phone, or a networked computer.

16. The method of claim 12, further comprising wirelessly transmitting the thermal images to at least one of an on-board security system, a key fob, a remote monitoring station, a mobile phone, or a networked computer to be converted into user-viewable thermal images for viewing by a user.

17. The method of claim 12, further comprising determining a condition of the vehicle, including whether the vehicle is parked, moving, or in a collision, wherein the determining of whether the one or more unwanted occupants are present is further based on the determined condition of the vehicle.

18. The method of claim 17, further comprising:

generating vehicle occupant information including corresponding positions of one or more occupants and a count of the one or more occupants based on the determination of the temperature profiles and the approximate sizes and/or shapes of the one or more objects; and

wirelessly transmitting, in response determining that the vehicle is in a collision, the generated vehicle occupant information to at least one of a remote monitoring station, a mobile phone, or a networked computer.

19. The method of claim 17, further comprising generating a vehicle disable signal to disable the vehicle in response to determining that the vehicle is moving and that the one or more unwanted occupants are present.

20. The method of claim 12, wherein:

the thermal images are radiometrically normalized thermal images such that the pixels represent calibrated temperature data; and

the method further comprises periodically and automatically calibrating the infrared imaging device for radiometric normalization of the thermal images.