Optical identification and monitoring system using pattern recognition for use with vehicles

• US 5,835,613 A
• Filed: 06/07/1995
• Issued: 11/10/1998
• Est. Priority Date: 05/05/1992
• Status: Expired due to Term

• Current Assignee: American Vehicular Sciences LLC (Acacia Research Corporation)
• Original Assignee: Automotive Technologies International Incorporated
• Inventors: Breed, David S., Johnson, Wendell C., DuVall, Wilbur E.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US08/474,782
• Time in Patent Office: 1,252 Days
• Field of Search: 340/436, 382/104, 382/103, 382/291, 382/100, 280/735, 348/143, 348/148
• US Class Current: 382/100
• CPC Class Codes: B60J 10/00 Sealing arrangements
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/02246 Electric motors therefor
• CPC Class Codes: B60N 2/0244 with logic circuits
• CPC Class Codes: B60N 2/0248 with memory of positions
• CPC Class Codes: B60N 2/0268 using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant
• CPC Class Codes: B60N 2/0272 using sensors or detectors for detecting the position of seat parts
• CPC Class Codes: B60N 2/0276 reaction to emergency situations, e.g. crash
• CPC Class Codes: B60N 2/28 Seats readily mountable on, and dismountable from, existing seats or other parts of the vehicle
• CPC Class Codes: B60N 2/2806 for securing the child seat to the vehicle
• CPC Class Codes: B60N 2/2863 backward facing
• CPC Class Codes: B60N 2/829 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/853 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/888 with arrangements for protecting against abnormal g-forces, e.g. by displacement of the head-rest
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 11/0241 for telephones
• CPC Class Codes: B60R 16/0231 Circuits relating to the driving or the functioning of the vehicle monitoring tyres B60C23/00; indicating overspeed B60K31/00; for dash boards B60K37/00, B60Q3/10; for indicating emergencies B60Q1/52; brake control systems B60T; registering or indicating the working of vehicles G07C5/00; measuring distance G01C, e.g. combinations of speed and distance G01C23/00; engine indicators G01L; measuring speed or acceleration G01P
• CPC Class Codes: B60R 16/037 : for occupant comfort , e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
• CPC Class Codes: B60R 19/205 : inflatable in the direction of an obstacle upon impending impact, e.g. using air bags
• CPC Class Codes: B60R 19/42 : extending primarily along the sides of, or completely encircling, a vehicle ornamental or guard strips B60R13/04
• CPC Class Codes: B60R 2001/1223 : with sensors or transducers
• CPC Class Codes: B60R 2021/0006 : Lateral collision
• CPC Class Codes: B60R 2021/0027 : Post collision measures, e.g. notifying emergency services
• CPC Class Codes: B60R 2021/01315 : monitoring occupant displacement
• CPC Class Codes: B60R 2021/23332 : using independent bags, one within the other
• CPC Class Codes: B60R 2021/26058 : using a combination of inflators
• CPC Class Codes: B60R 2021/2765 : comprising means to control the venting
• CPC Class Codes: B60R 2022/208 : by automatic or remote control means
• CPC Class Codes: B60R 2022/288 : with means to adjust or regulate the amount of energy to be absorbed
• CPC Class Codes: B60R 2022/4825 : for sensing amount of belt winded on retractor
• CPC Class Codes: B60R 21/013 : including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/0134 : responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: B60R 21/0136 : responsive to actual contact with an obstacle , e.g. to vehicle deformation, bumper displacement or bumper velocity relative to the vehicle monitoring crash strength in combination with passenger seat detection B60R21/01558
• CPC Class Codes: B60R 21/015 : including means for detecting the presence or position of passengers, passenger seats or child seats , and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
• CPC Class Codes: B60R 21/01516 : using force or pressure sensing means
• CPC Class Codes: B60R 21/01534 : using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 : using ultrasonic waves
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/0154 : in combination with seat heating
• CPC Class Codes: B60R 21/01542 : detecting passenger motion
• CPC Class Codes: B60R 21/01544 : detecting seat belt parameters, e.g. length, tension or height-adjustment
• CPC Class Codes: B60R 21/01548 : sensing the amount of belt winded on retractor
• CPC Class Codes: B60R 21/01552 : detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01554 : Seat position sensors
• CPC Class Codes: B60R 21/21656 : Steering wheel covers or similar cup-shaped covers
• CPC Class Codes: B60R 21/23138 : specially adapted for side protection B60R21/23184, B60R21/232 take precedence
• CPC Class Codes: B60R 21/233 : comprising a plurality of individual compartments; comprising two or more bag-like members, one within the other B60R21/232 takes precedence
• CPC Class Codes: B60R 22/20 : adjustable in position, e.g. in height
• CPC Class Codes: B60R 22/26 : secured to the seat
• CPC Class Codes: B60R 22/321 : using electric means
• CPC Class Codes: B60R 25/25 : using biometry
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: E05F 15/431 : specially adapted for vehicle windows or roofs
• CPC Class Codes: E05F 2015/433 : using reflection from the obstruction
• CPC Class Codes: E05F 2015/434 : with cameras or optical sensors
• CPC Class Codes: E05Y 2900/516 : for trucks or trailers
• CPC Class Codes: E05Y 2900/55 : Windows
• CPC Class Codes: G01F 23/20 : by measurement of weight, e.g. to determine the level of stored liquefied gas
• CPC Class Codes: G01F 23/2962 : Measuring transit time of reflected waves
• CPC Class Codes: G01F 23/804 : containing circuits handling parameters other than liquid level
• CPC Class Codes: G01S 13/04 : Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 13/88 : Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 15/04 : Systems determining presence of a target
• CPC Class Codes: G01S 15/06 : Systems determining the position data of a target
• CPC Class Codes: G01S 15/87 : Combinations of sonar systems
• CPC Class Codes: G01S 15/88 : Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 15/931 : of land vehicles
• CPC Class Codes: G01S 17/88 : Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/89 : for mapping or imaging
• CPC Class Codes: G01S 7/417 : involving the use of neural networks
• CPC Class Codes: G01S 7/48 : of systems according to group G01S17/00
• CPC Class Codes: G01S 7/4802 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/487 : Extracting wanted echo signals , e.g. pulse detection
• CPC Class Codes: G01S 7/539 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08G 1/161 : Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 : involving continuous checking
• CPC Class Codes: G08G 1/164 : Centralised systems, e.g. external to vehicles
• CPC Class Codes: G10K 2210/1282 : Automobiles
• CPC Class Codes: G10K 2210/3219 : Geometry of the configuration

Abstract

A vehicle interior monitoring system to identify, locate and monitor occupants, including their parts, and other objects in the passenger compartment and objects outside of a motor vehicle, such as an automobile or truck, by illuminating the contents of the vehicle and objects outside of the vehicle with electromagnetic, and specifically infrared, radiation and using one or more lenses to focus images of the contents onto one or more arrays of charge coupled devices (CCD arrays). Outputs from the CCD arrays, are analyzed by appropriate computational means employing trained pattern recognition technologies, to classify, identify or locate the contents or external objects. In general, the information obtained by the identification and monitoring system is used to affect the operation of some other system in the vehicle. When system is installed in the passenger compartment of an automotive vehicle equipped with an airbag, the system determines the position of the vehicle occupant relative to the airbag and disables deployment of the airbag if the occupant is positioned so that he/she is likely to be injured by the deployment of the airbag.

Claims

1. In a motor vehicle having an interior passenger compartment containing at least one occupying item, an interior monitoring system comprising:

a) illumination means for illuminating with electromagnetic radiation a portion of said vehicle interior passenger compartment in which said at least one occupying item is likely situated;

b) receiver means for receiving electromagnetic illumination reflected from said at least one occupying item in said vehicle interior passenger compartment;

c) processor means coupled to said receiver means for processing said received electromagnetic illumination and generating an electronic signal characteristic of said at least one occupying item in said vehicle interior passenger compartment based thereon;

d) means for measuring the distance of said at least one occupying item from said receiver means;

e) categorization and identification means coupled to said processor means for categorizing said electronic signal to thereby identify said at least one occupying item, said categorization and identification means comprising trained pattern recognition means for processing said electronic signal based on said received electromagnetic illumination from said at least one occupying item to provide an identification of said at least one occupying item based thereon, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from data of possible occupying items of the vehicle and patterns of received electromagnetic illumination from the possible occupying items; and

f) output means coupled to said categorization and identification means and said distance measuring means for affecting another system in said vehicle in response to the identification of said at least one occupying item and the distance of said at least one occupying item from said receiver means.

2. The arrangement in accordance with claim 1 wherein said receiver means comprises two CCD arrays spaced apart from one another, said distance measurement means comprising means for analyzing images received by said two CCD arrays.

3. The arrangement in accordance with claim 1 wherein said another system is an inflatable passive restraint apparatus and said output means comprises circuitry to disable the deployment of said inflatable restraint apparatus.

4. The arrangement in accordance with claim 1 wherein said receiver means comprise at least one CCD array arranged to receive waves reflected off the vehicle windshield and without a direct line of sight between said at least one CCD array and said at least one occupying item.

6. The arrangement in accordance with claim 1, wherein said processor means are structured and arranged to generate an electronic signal characteristic of the form of said at least one occupying item in said vehicle interior passenger compartment based on said received electromagnetic radiation.

9. The system in accordance with claim 1 wherein said means for measuring distance comprises a focusable lens.

10. The system in accordance with claim 1 wherein said means for measuring distance comprises a pulsed laser.

11. The system in accordance with claim 1 wherein said means for measuring distance comprises structured light.

12. The system in accordance with claim 1 wherein said means for measuring distance comprises an ultrasonic transmitter and receiver.

13. The system in accordance with claim 12, wherein said ultrasonic transmitter comprises a phased array.

21. The arrangement in accordance with claim 1, wherein said receiver means comprise at least one CCD array.

5. In a motor vehicle having an interior passenger compartment containing at least one occupying item having surfaces, an interior monitoring system comprising:

(a) at least one infrared transmitter for transmitting infrared waves toward said at least one occupying item;

(b) modulation means for modulating said transmitted waves such that said transmitted waves constitute a first signal at a first frequency in the infrared range;

(c) generator means for generating a second signal having a second frequency in the infrared range slightly different from said first frequency;

(d) at least one infrared receiver for receiving infrared waves transmitted from said at least one transmitter and reflected from said at least one occupying item, said at least one receiver being structured and arranged to generate a third signal at the same frequency as said first signal but shifted in phase and based on the received infrared waves;

(e) first combining means for combining said first signal with said second signal to produce a first beat signal;

(f) second combining means for combining said third signal with said second signal to produce a second beat signal; and

(g) detection means for detecting a phase shift between said first beat signal and said second beat signal, said phase shift being a measure of the position of said at least one occupying item from said at least one transmitter.

19. The system of claim 5, wherein said generator means comprise a crystal oscillator.

7. In a motor vehicle having an interior passenger compartment containing at least one occupying item having surfaces, a system for determining the location of said at least one occupying item comprising:

a) illumination means for illuminating a portion of said vehicle interior passenger compartment with electromagnetic radiation;

b) receiver means for receiving reflected electromagnetic illumination from said surfaces of said at least one occupying item within said vehicle interior passenger compartment, said receiver means comprising at least one CCD array structured and arranged to obtain an image representative of the electromagnetic illumination reflected from said surfaces of said at least one occupying item and convert the image into electrical signals;

c) processor means coupled to said at least one CCD array for processing said electrical signals converted from said image obtained by said at least one CCD array, said processor means being structured and arranged to process said electrical signals from said at least one CCD array into a measurement of the distance from said at least one occupying item to said at least one CCD array;

(d) output means coupled to said processor means for affecting another system in the vehicle in response to the measured distance of said at least one occupying item from said at least one CCD array by said processor means.

8. The system in accordance with claim 7 wherein said CCD array comprises a fisheye lens.

14. The system in accordance with claim 7 wherein said processor means further comprises trained pattern recognition means for processing the electrical signals corresponding to the electromagnetic illumination received by said CCD array to obtain the distance from said at least one CCD array to said at least one occupying item, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from data of distances between occupying items and said at least one CCD array and patterns of received electromagnetic illumination from the occupying items.

15. The system in accordance with claim 7 wherein said at least one occupying item is a human part selected from the group consisting of the eyes, the ears, the head and the chest.

16. The system in accordance with claim 14, wherein said vehicle has a driver and rear view mirrors, said another vehicle system comprising positioning means for positioning at least one of the rear view mirrors relative to the position of the eyes of the driver.

17. The system in accordance with claim 7 wherein said occupying item is a human, said vehicle has at least one airbag and said another vehicle system comprises means to control the inflation of said at least one airbag.

18. In a motor vehicle having an occupying item, a vehicle occupant position sensor arrangement comprising:

(a) at least one transmitter for transmitting electromagnetic radiation toward said occupant, said transmitted radiation illuminating a substantial portion of said occupant;

(b) at least one receiver for receiving reflected radiation from said occupant, said receiver comprising means for measuring the position of said occupant, said position measurement means comprising a charge coupled device for converting the received radiation into electrical signals and trained pattern recognition means for processing said electronic signals based on the received radiation reflected from said occupant to obtain the position of said occupant based thereon, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from different positions of an occupant and patterns of received radiation reflected from the occupant in the different positions;

(c) mounting means for mounting said at least one transmitter and said at least one receiver onto said vehicle; and

(d) analysis means coupled to said position measuring means of said at least one receiver for responding to said position of said occupant to affect another system in said vehicle.

20. The arrangement of claim 18, wherein said vehicle has a driver and rear view mirrors, said another vehicle system comprising positioning means for positioning at least one of the rear view mirrors relative to the position of the eyes of the driver.

22. In a motor vehicle having an interior passenger compartment containing at least one occupying item having surfaces and at least one infrared transmitter for transmitting infrared waves toward said at least one occupying item, a method for determining the position of said at least one occupying item from said at least one transmitter comprising the steps of:

modulating said transmitted waves from said at least one transmitter such that said transmitted waves constitute a first signal at a first frequency in the infrared range;

generating a second signal having a second frequency in the infrared range slightly different from said first frequency;

arranging at least one infrared receiver to receive infrared waves transmitted from said at least one transmitter and reflected from said at least one occupying item and generate a third signal at the same frequency as said first signal but shifted in phase and based thereon;

combining said first signal with said second signal to produce a first beat signal;

combining said third signal with said second signal to produce a second beat signal; and

detecting a phase shift between said first beat signal and said second beat signal, said phase shift being a measure of the position of said at least one occupying item from said at least one transmitter.