Optical identification and monitoring system using pattern recognition for use with vehicles

• US 5,845,000 A
• Filed: 06/07/1995
• Issued: 12/01/1998
• Est. Priority Date: 05/05/1992
• Status: Expired due to Term

• Current Assignee: American Vehicular Sciences LLC (Acacia Research Corporation)
• Original Assignee: Automotive Technologies International Incorporated
• Inventors: Breed, David S., DuVall, Wilbur E., Johnson, Wendell C.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US08/474,786
• Time in Patent Office: 1,273 Days
• Field of Search: 340/436, 382/104, 382/103, 382/291, 382/100, 280/735, 348/143, 348/148
• US Class Current: 382/100
• CPC Class Codes: B60J 10/00 Sealing arrangements
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/02246 Electric motors therefor
• CPC Class Codes: B60N 2/0244 with logic circuits
• CPC Class Codes: B60N 2/0248 with memory of positions
• CPC Class Codes: B60N 2/0268 using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant
• CPC Class Codes: B60N 2/0272 using sensors or detectors for detecting the position of seat parts
• CPC Class Codes: B60N 2/0276 reaction to emergency situations, e.g. crash
• CPC Class Codes: B60N 2/28 Seats readily mountable on, and dismountable from, existing seats or other parts of the vehicle
• CPC Class Codes: B60N 2/2806 for securing the child seat to the vehicle
• CPC Class Codes: B60N 2/2863 backward facing
• CPC Class Codes: B60N 2/829 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/853 characterised by their adjusting mechanisms, e.g. electric motors
• CPC Class Codes: B60N 2/888 with arrangements for protecting against abnormal g-forces, e.g. by displacement of the head-rest
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 11/0241 for telephones
• CPC Class Codes: B60R 16/0231 Circuits relating to the driving or the functioning of the vehicle monitoring tyres B60C23/00; indicating overspeed B60K31/00; for dash boards B60K37/00, B60Q3/10; for indicating emergencies B60Q1/52; brake control systems B60T; registering or indicating the working of vehicles G07C5/00; measuring distance G01C, e.g. combinations of speed and distance G01C23/00; engine indicators G01L; measuring speed or acceleration G01P
• CPC Class Codes: B60R 16/037 : for occupant comfort , e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
• CPC Class Codes: B60R 19/205 : inflatable in the direction of an obstacle upon impending impact, e.g. using air bags
• CPC Class Codes: B60R 19/42 : extending primarily along the sides of, or completely encircling, a vehicle ornamental or guard strips B60R13/04
• CPC Class Codes: B60R 2001/1223 : with sensors or transducers
• CPC Class Codes: B60R 2021/0006 : Lateral collision
• CPC Class Codes: B60R 2021/0027 : Post collision measures, e.g. notifying emergency services
• CPC Class Codes: B60R 2021/01315 : monitoring occupant displacement
• CPC Class Codes: B60R 2021/23332 : using independent bags, one within the other
• CPC Class Codes: B60R 2021/26058 : using a combination of inflators
• CPC Class Codes: B60R 2021/2765 : comprising means to control the venting
• CPC Class Codes: B60R 2022/208 : by automatic or remote control means
• CPC Class Codes: B60R 2022/288 : with means to adjust or regulate the amount of energy to be absorbed
• CPC Class Codes: B60R 2022/4825 : for sensing amount of belt winded on retractor
• CPC Class Codes: B60R 21/013 : including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/01336 : using filtering
• CPC Class Codes: B60R 21/0134 : responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: B60R 21/0136 : responsive to actual contact with an obstacle , e.g. to vehicle deformation, bumper displacement or bumper velocity relative to the vehicle monitoring crash strength in combination with passenger seat detection B60R21/01558
• CPC Class Codes: B60R 21/01516 : using force or pressure sensing means
• CPC Class Codes: B60R 21/01534 : using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 : using ultrasonic waves
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/0154 : in combination with seat heating
• CPC Class Codes: B60R 21/01542 : detecting passenger motion
• CPC Class Codes: B60R 21/01544 : detecting seat belt parameters, e.g. length, tension or height-adjustment
• CPC Class Codes: B60R 21/01548 : sensing the amount of belt winded on retractor
• CPC Class Codes: B60R 21/01552 : detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01554 : Seat position sensors
• CPC Class Codes: B60R 21/21656 : Steering wheel covers or similar cup-shaped covers
• CPC Class Codes: B60R 21/23138 : specially adapted for side protection B60R21/23184, B60R21/232 take precedence
• CPC Class Codes: B60R 21/233 : comprising a plurality of individual compartments; comprising two or more bag-like members, one within the other B60R21/232 takes precedence
• CPC Class Codes: B60R 22/20 : adjustable in position, e.g. in height
• CPC Class Codes: B60R 22/26 : secured to the seat
• CPC Class Codes: B60R 22/321 : using electric means
• CPC Class Codes: B60R 25/25 : using biometry
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: E05F 15/431 : specially adapted for vehicle windows or roofs
• CPC Class Codes: E05F 2015/433 : using reflection from the obstruction
• CPC Class Codes: E05F 2015/434 : with cameras or optical sensors
• CPC Class Codes: E05Y 2900/516 : for trucks or trailers
• CPC Class Codes: E05Y 2900/55 : Windows
• CPC Class Codes: G01F 23/20 : by measurement of weight, e.g. to determine the level of stored liquefied gas
• CPC Class Codes: G01F 23/2962 : Measuring transit time of reflected waves
• CPC Class Codes: G01F 23/804 : containing circuits handling parameters other than liquid level
• CPC Class Codes: G01S 13/04 : Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 13/88 : Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 15/04 : Systems determining presence of a target
• CPC Class Codes: G01S 15/06 : Systems determining the position data of a target
• CPC Class Codes: G01S 15/87 : Combinations of sonar systems
• CPC Class Codes: G01S 15/88 : Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 15/931 : of land vehicles
• CPC Class Codes: G01S 17/88 : Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/89 : for mapping or imaging
• CPC Class Codes: G01S 7/417 : involving the use of neural networks
• CPC Class Codes: G01S 7/48 : of systems according to group G01S17/00
• CPC Class Codes: G01S 7/4802 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/487 : Extracting wanted echo signals , e.g. pulse detection
• CPC Class Codes: G01S 7/539 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G06V 20/593 : Recognising seat occupancy
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08G 1/161 : Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 : involving continuous checking
• CPC Class Codes: G08G 1/164 : Centralised systems, e.g. external to vehicles
• CPC Class Codes: G10K 2210/1282 : Automobiles
• CPC Class Codes: G10K 2210/3219 : Geometry of the configuration

Abstract

A vehicle interior monitoring system to identify, locate and monitor occupants, including their parts, and other objects in the passenger compartment and objects outside of a motor vehicle, such as an automobile or truck, by illuminating the contents of the vehicle and objects outside of the vehicle with electromagnetic, and specifically infrared, radiation and using one or more lenses to focus images of the contents onto one or more arrays of charge coupled devices (CCD arrays). Outputs from the CCD arrays, are analyzed by appropriate computational means employing trained pattern recognition technologies, to classify, identify or locate the contents or external objects. In general, the information obtained by the identification and monitoring system is used to affect the operation of some other system in the vehicle. When system is installed in the passenger compartment of an automotive vehicle equipped with an airbag, the system determines the position of the vehicle occupant relative to the airbag and disables deployment of the airbag if the occupant is positioned so that he/she is likely to be injured by the deployment of the airbag.

Claims

1. In a motor vehicle having an interior passenger compartment containing at least one occupying item, an interior monitoring system comprising:

a) illumination means for illuminating a portion of said vehicle interior passenger compartment with electromagnetic radiation in which said at least one occupying item is likely situated;

b) receiver means for receiving electromagnetic illumination reflected from said at least one occupying item in said vehicle interior passenger compartment;

c) processor means coupled to said receiver means for processing said received illumination and generating an electronic signal characteristic of said at least one occupying item in said passenger compartment based thereon;

d) categorization and identification means coupled to said processor means for categorizing said electronic signal to thereby identify said at least one occupying item, said categorization and identification means comprising trained pattern recognition means for processing said electronic signal based on said received illumination from said at least one occupying item to provide an identification of said at least one occupying item based thereon, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from data of possible occupying items of the vehicle and patterns of received electromagnetic illumination from the possible occupying items; and

e) output means coupled to said categorization and identification means for affecting another system in said vehicle in response to the identification of said at least one occupying item.

2. The system in accordance with claim 1, wherein said another system comprises an inflatable airbag and said at least one occupying item is an occupant, said pattern recognition means being structured and arranged to process said electronic signal based on said received illumination from said occupant into a positional categorization of said signal characteristic of the position of the occupant based on data corresponding to patterns of received electromagnetic illumination stored within said pattern recognition means and associated with occupants of the vehicle in different positions, said output means comprising means for modifying the time at which inflation of said inflatable airbag is initiated in response to the position of the occupant.

3. The system in accordance with claim 1, wherein said another system comprises an inflatable airbag and said at least one occupying item is an occupant, said pattern recognition means being structured and arranged to process said electronic signal based on said received illumination from said surfaces of the occupant into a positional categorization of said signal characteristic of the position of the occupant based on data corresponding to patterns of received electromagnetic illumination stored within said pattern recognition means and associated with occupants of the vehicle in different positions, said output means comprising means for modifying the rate at which said inflatable airbag is inflated in response to the position of the occupant.

4. The system in accordance with claim 1, wherein said another system comprises an inflatable airbag and said at least one occupying item is an occupant, said pattern recognition means being structured and arranged to process said electronic signal based on said received illumination from said occupant into a categorization of said signal characteristic of the presence of the occupant based on data corresponding to patterns of received electromagnetic illumination stored within said pattern recognition means and associated with the presence and absence of the occupant of the vehicle, said output means comprising means for suppressing deployment of said inflatable in response to the absence of the occupant.

5. The system in accordance with claim 1, wherein said at least one occupying item is on an occupant, said pattern recognition means being structured and arranged to process said electronic signal based on said received illumination from said occupant into a head-positional categorization of said signal characteristic of the position of the occupant'"'"'s head based on data corresponding to patterns of received electromagnetic illumination stored within said pattern recognition means and associated with the head of occupants of the vehicle in different positions, said trained pattern recognition means comprises means to categorize the motion of the occupant'"'"'s head over time, and said output means coupled to said pattern recognition means responds to the category of motion of the occupant'"'"'s head over time to affect said another system in said vehicle.

6. The sensor in accordance with claim 5, wherein said another system comprises an alarm.

7. The sensor in accordance with claim 5, wherein said another system comprises limiting means for limiting the speed of said vehicle.

8. The system in accordance with claim 1 wherein said trained pattern recognition means comprises a neural network.

9. The system in accordance with claim 1 wherein said vehicle further comprises a seat and said at least one occupying item is a rear facing child seat positioned on said seat of said vehicle, and said another vehicle system comprises an airbag.

25. The system of claim 1, wherein said receiver means comprise a CCD array.

10. In a motor vehicle having an interior and an exterior, a monitoring system for monitoring at least one object exterior to said vehicle comprising:

a) transmitter means for transmitting electromagnetic waves to illuminate the at least one exterior object;

b) reception means for receiving reflected electromagnetic illumination from the at least one exterior object;

c) processor means coupled to said reception means for processing said received illumination and creating an electronic signal characteristic of said exterior object based thereon;

d) categorization means coupled to said processor means for categorizing said electronic signal to identify said exterior object, said categorization means comprising trained pattern recognition means for processing said electronic signal based on said received illumination from said exterior object to provide an identification of said exterior object based thereon, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from data of possible exterior objects and patterns of received electromagnetic illumination from the possible exterior objects; and

e) output means coupled to said categorization means for affecting another system in the vehicle in response to the identification of said exterior object.

11. The system in accordance with claim 10, further comprising measurement means for measuring the distance from the at least one exterior object to said vehicle, said measurement means comprising radar.

12. The system in accordance with claim 10, further comprising measurement means for measuring the distance from the at least one exterior object to said vehicle, said measurement means comprising a pulsed laser.

13. The system in accordance with claim 10, wherein said another system is a warning system which is activated when the at least one exterior object is within a specified range of said vehicle and said vehicle driver begins to execute an action which increases the probability of a collision with the at least one exterior object.

14. The system in accordance with claim 10, wherein said another system is a side impact airbag system which is activated when the at least one exterior object is within a specified range of said vehicle and said processor means determines that there is a high probability of a collision with the at least one exterior object.

15. The system in accordance with claim 10 wherein said processor means comprises a neural network algorithm.

19. The system of claim 10, wherein said reception means comprise a CCD array.

16. In a motor vehicle having an interior and an exterior, an automatic headlight dimming system comprising:

a) reception means for receiving electromagnetic radiation from the exterior of the vehicle;

b) processor means coupled to said reception means for processing the received radiation and creating an electronic signal characteristic of the received radiation;

c) categorization means coupled to said processor means for categorizing said electronic signal to identify a source of the radiation, said categorization means comprising trained pattern recognition means for processing said electronic signal based on said received radiation to provide an identification of the source of the radiation based thereon, said pattern recognition means being structured and arranged to apply a pattern recognition algorithm generated from data of possible sources of radiation including lights of vehicles and patterns of received radiation from the possible sources; and

d) output means coupled to said categorization means for dimming the headlights in said vehicle in response to the identification of the source of the radiation.

17. The invention in accordance with claim 16 wherein said categories further comprise radiation from taillights of a vehicle-in-front.

18. The invention in accordance with claim 16 wherein said trained pattern recognition algorithm comprises a neural network.

20. The invention in accordance with claim 16, wherein said reception means comprise a CCD array.

21. In a motor vehicle having an interior passenger compartment containing at least one occupying item, an interior monitoring system comprising:

a) illumination means for illuminating a portion of said vehicle interior passenger compartment with electromagnetic radiation in which said at least one occupying item is likely situated;

b) receiver means for receiving electromagnetic illumination reflected from said at least one occupying item in said vehicle interior passenger compartment;

c) processor means coupled to said receiver means for processing said received illumination and generating an electronic signal characteristic of the form of said at least one occupying item in said passenger compartment based thereon;

d) categorization and identification means coupled to said processor means for categorizing said electronic signal to thereby identify said at least one occupying item based on the form of said at least one occupying item, said categorization and identification means comprising trained pattern recognition means for applying a pattern recognition algorithm; and

e) output means coupled to said categorization and identification means for affecting another system in said vehicle in response to the identification of said at least one occupying item based on the form of said at least one occupying item.

22. The system of claim 21, wherein said trained pattern recognition means are structured and arranged to process said electronic signal based on said received illumination from said at least one occupying item to provide an identification of said at least one occupying item based thereon by applying a pattern recognition algorithm generated from data of possible occupying items of the vehicle and patterns of received electromagnetic illumination from the possible occupying items.

23. A method for affecting a system in a vehicle based on an object exterior of the vehicle, comprising the steps of:

a) transmitting electromagnetic waves to illuminate the exterior object;

b) receiving reflected electromagnetic illumination from the object on an array;

c) processing the received illumination and creating an electronic signal characteristic of the exterior object based thereon;

d) processing the electronic signal based on the received illumination from the exterior object to identify the exterior object, said processing step comprising the steps of generating a pattern recognition algorithm from data of possible exterior objects and patterns of received electromagnetic illumination from the possible exterior objects, storing the algorithm within a pattern recognition system and applying the pattern recognition algorithm using the electronic signal as input to obtain the identification of the exterior object; and

e) affecting the system in the vehicle in response to the identification of the exterior object.

24. A method for monitoring an interior of a motor vehicle having an interior passenger compartment containing at least one occupying item, comprising the steps of:

a) illuminating a portion of said vehicle interior passenger compartment with electromagnetic radiation in which said at least one occupying item is likely situated;

b) receiving electromagnetic illumination reflected from said at least one occupying item in said vehicle interior passenger compartment on an array;

c) processing said received illumination and generating an electronic signal characteristic of said at least one occupying item in said passenger compartment based thereon;

d) processing the electronic signal based on the received illumination from said at least one occupying item to provide an identification of said at least one occupying item, said processing step comprising the steps of generating a pattern recognition algorithm from data of possible occupying items and patterns of received electromagnetic illumination from the possible occupying items, storing the algorithm within a pattern recognition system and applying the pattern recognition algorithm using the electronic signal as input to obtain the identification of the at least one occupying item; and

e) affecting another system in said vehicle in response to the identification of said at least one occupying item.