Vehicular monitoring systems using image processing

• US 20020116106A1
• Filed: 04/05/2002
• Published: 08/22/2002
• Est. Priority Date: 06/07/1995
• Status: Active Grant

• Current Assignee: American Vehicular Sciences LLC (Acacia Research Corporation)
• Original Assignee: Automotive Technologies International Incorporated
• Inventors: Breed, David S., Johnson, Wendell C., DuVall, Wilbur E.
• Granted Patent: US 6,856,873 B2
• Time in Patent Office: Days
• US Class Current: 701/45
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/02246 Electric motors therefor
• CPC Class Codes: B60N 2/0244 with logic circuits
• CPC Class Codes: B60N 2/0268 using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant
• CPC Class Codes: B60N 2/0272 using sensors or detectors for detecting the position of seat parts
• CPC Class Codes: B60N 2/0276 reaction to emergency situations, e.g. crash
• CPC Class Codes: B60N 2/28 Seats readily mountable on, and dismountable from, existing seats or other parts of the vehicle
• CPC Class Codes: B60N 2/2806 for securing the child seat to the vehicle
• CPC Class Codes: B60N 2/2863 backward facing
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 2022/208 by automatic or remote control means
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/0134 responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: B60R 21/0152 using strain gauges
• CPC Class Codes: B60R 21/01534 using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01536 using ultrasonic waves
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/0154 in combination with seat heating
• CPC Class Codes: B60R 25/255 : Eye recognition
• CPC Class Codes: G01S 15/87 : Combinations of sonar systems
• CPC Class Codes: G01S 15/88 : Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 17/88 : Lidar systems specially adapted for specific applications
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Vehicular monitoring arrangement for monitoring an environment of the vehicle including at least one active pixel camera for obtaining images of the environment of the vehicle and a processor coupled to the active pixel camera(s) for determining at least one characteristic of an object in the environment based on the images obtained by the active pixel camera(s). The active pixel camera can be arranged in a headliner, roof or ceiling of the vehicle to obtain images of an interior environment of the vehicle, in an A-pillar or B-pillar of the vehicle to obtain images of an interior environment of the vehicle, or in a roof, ceiling, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle behind a front seat of the vehicle. The determined characteristic can be used to enable optimal control of a reactive component, system or subsystem coupled to the processor. When the reactive component is an airbag assembly including at least one airbag, the processor can be designed to control at least one deployment parameter of the airbag(s).

Claims

1. A vehicle including a monitoring arrangement for monitoring an environment of the vehicle, the monitoring arrangement comprising:

at least one active pixel camera for obtaining images of the environment of the vehicle; and

a processor coupled to said at least one active pixel camera for determining at least one characteristic of an object in the environment based on the images obtained by said at least one active pixel camera.

2. The vehicle of claim 1, wherein said at least one active pixel camera is arranged in a headliner, roof or ceiling of the vehicle to obtain images of an interior environment of the vehicle.

3. The vehicle of claim 1, wherein said at least one active pixel camera is arranged in an A-pillar or B-pillar of the vehicle to obtain images of an interior environment of the vehicle.

4. The vehicle of claim 1, wherein said at least one active pixel camera is arranged in a roof, ceiling, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle behind a front seat of the vehicle.

5. The vehicle of claim 1, further comprising a reactive component, system or subsystem coupled to said processor, said processor being arranged to control said reactive component, system or subsystem based on the determined at least one characteristic of the object.

6. The vehicle of claim 5, wherein said reactive component, system or subsystem is an airbag assembly including at least one airbag, said processor being arranged to control at least one deployment parameter of said at least one airbag.

7. The vehicle of claim 1, wherein the at least one characteristic is selected from a group consisting of the presence of an object in the environment, the type of object in the environment, the position of an object in the environment and the velocity of an object in the environment.

8. The vehicle of claim 1, wherein said monitoring arrangement further comprises means for illuminating the environment.

9. The vehicle of claim 1, wherein said at least one active pixel camera is arranged to obtain images of an environment exterior of the vehicle.

10. A method for monitoring an environment of the vehicle, comprising the steps of:

obtaining images of the environment of the vehicle by means of at least one active pixel camera; and

determining at least one characteristic of an object in the environment based on the images obtained by the at least one active pixel camera.

11. The method of claim 10, further comprising the step of mounting the at least one active pixel camera in a headliner, roof or ceiling of the vehicle to obtain images of an interior environment of the vehicle.

12. The method of claim 10, further comprising the step of mounting the at least one active pixel camera in an A-pillar, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle.

13. The method of claim 10, further comprising the step of mounting the at least one active pixel camera in a roof, ceiling, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle behind a front seat of the vehicle.

14. The method of claim 10, further comprising the step of controlling a reactive component, system or subsystem based on the determined at lest one characteristic of the object.

15. The method of claim 10, further comprising the step of selecting the at least one characteristic from a group consisting of the presence of an object in the environment, the type of object in the environment, the position of an object in the environment and the velocity of an object in the environment.

16. The method of claim 10, further comprising the step of illuminating the environment.

17. The method of claim 10, further comprising the step of arranging the at least one active pixel camera to obtain images of an environment exterior of the vehicle.

19. The vehicle of claim 18, wherein said at least one pixel comprises an array of light-receiving pixels.

20. The vehicle of claim 18, wherein said light source is arranged to direct light into an interior environment of the vehicle.

21. The vehicle of claim 18, wherein said light source is arranged to direct light to an exterior environment of the vehicle.

22. The vehicle of claim 18, further comprising means for modulating a frequency of the light being directed by said light source into the environment.

23. The vehicle of claim 18, further comprising means for providing a correlation pattern in a form of code division modulation of the light being directed by said light source into the environment.

24. The vehicle of claim 18, wherein said at least one pixel is a photo diode.

25. The vehicle of claim 24, wherein said photo diode is a PIN or avalanche diode.

27. The vehicle of claim 26, wherein said imaging device is arranged in a headliner, roof or ceiling of the vehicle to obtain images of an interior environment of the vehicle.

28. The vehicle of claim 26, wherein said imaging device is arranged in an A-pillar or B-pillar of the vehicle to obtain images of an interior environment of the vehicle.

29. The vehicle of claim 26, wherein said imaging device is arranged in a roof, ceiling, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle behind a front seat of the vehicle.

30. The vehicle of claim 26, further comprising a reactive component, system or subsystem coupled to said processor, said processor being arranged to control said reactive component, system or subsystem based on the determined at least one characteristic of the object.

31. The vehicle of claim 30, wherein said reactive component, system or subsystem is an airbag assembly including at least one airbag, said processor being arranged to control at least one deployment parameter of said at least one airbag.

32. The vehicle of claim 26, wherein the at least one characteristic is selected from a group consisting of the presence of an object in the environment, the type of object in the environment, the position of an object in the environment and the velocity of an object in the environment.

33. The vehicle of claim 26, wherein said monitoring arrangement further comprises means for illuminating the environment.

34. The vehicle of claim 26, wherein said imaging device is arranged to obtain images of an environment exterior of the vehicle.

36. The vehicle of claim 35, wherein said light source is arranged to direct pulses of light into an interior environment of the vehicle.

37. The vehicle of claim 35, wherein said light source is arranged to direct pulses light into an exterior environment of the vehicle.

38. The vehicle of claim 35, wherein said light source is arranged in a headliner, roof or ceiling of the vehicle to obtain images of an interior environment of the vehicle.

39. The vehicle of claim 35, wherein said light source is arranged in an A-pillar or B-pillar of the vehicle to obtain images of an interior environment of the vehicle.

40. The vehicle of claim 35, wherein said light source is arranged in a roof, ceiling, B-pillar or C-pillar of the vehicle to obtain images of an interior environment of the vehicle behind a front seat of the vehicle.

41. The vehicle of claim 35, further comprising a reactive component, system or subsystem coupled to said processor, said processor being arranged to control said reactive component, system or subsystem based on the determined distance.

42. The vehicle of claim 41, wherein said reactive component system or subsystem is an airbag assembly including at least one airbag, said processor being arranged to control at least one deployment parameter of said at least one airbag.

43. The vehicle of claim 35, wherein the at least one characteristic is selected from a group consisting of the presence of an object in the environment, the type of object in the environment, the position of an object in the environment and the velocity of an object in the environment.

44. The vehicle of claim 35, wherein said monitoring arrangement further comprises means for illuminating the environment.

46. The vehicle of claim 45, wherein said phase difference is a measure of the distance between the object and said wave receiver and said wave transmitter.

47. The vehicle of claim 45, wherein said wave transmitter comprises an infrared driver and said receiver comprises an infrared diode.

49. The method of claim 48, wherein the phase difference is a measure of the distance between the object and the wave receiver and the wave transmitter.

50. The method of claim 48, further comprising,

generating a second signal based on the received waves at the same frequency as the first signal but shifted in phase;

generating a third signal having a third frequency different than the first frequency and in the same radio range as the first frequency;

combining the first signal and the third signal to produce a first beat signal; and

combining the second signal and the third signal to produce a second beat signal.

18. A vehicle including an arrangement for measuring position of an object in an environment of or about the vehicle, the position-measuring arrangement comprising:

a light source capable of directing modulated light into the environment;

at least one light-receiving pixel arranged to receive the modulated light after reflection by any objects in the environment; and

a processor for determining the distance between any objects from which the modulated light is reflected and said light source based on the reception of the modulated light by said at least one pixel.

26. A vehicle including a monitoring arrangement for monitoring an environment of the vehicle, the monitoring arrangement comprising:

an imaging device for obtaining three-dimensional images of the environment; and

a processor embodying a pattern recognition technique for processing the three-dimensional images to determine at least one characteristic of an object in the environment based on the three-dimensional images obtained by said imaging device.

35. A vehicle including an arrangement for measuring position of an object in an environment of or about the vehicle, the position-measuring arrangement comprising:

a light source capable of directing individual pulses of light into the environment;

at least one array of light-receiving pixels arranged to receive light after reflection by any objects in the environment; and

a processor for determining the distance between any objects from which any pulse of light is reflected and said light source based on a difference in time between the emission of a pulse of light by said light source and the reception of light by said at least one array.

45. A vehicle including an arrangement for monitoring objects in or about a vehicle, the arrangement comprising:

generating means generating a first signal having a first frequency in a specific radio range;

a wave transmitter arranged to receive the signal and transmit waves toward the objects;

a wave-receiver arranged relative to said wave transmitter for receiving waves transmitted by said wave transmitter after the waves have interacted with an object, said wave receiver being arranged to generate a second signal based on the received waves at the same frequency as said first signal but shifted in phase; and

detecting means for detecting a phase difference between said first and second signals, whereby said phase difference is a measure of a property of the object.

48. A method for monitoring objects in or about a vehicle, comprising the steps of:

generating a first signal having a first frequency in a specific radio range;

directing the first signal to a wave transmitter to cause the wave transmitter to transmit waves toward the objects;

arranging a wave receiver relative to the wave transmitter;

receiving waves transmitted by the wave transmitter at the wave receiver after the waves have interacted with an object;

detecting a phase difference between the first and second signals, the phase difference being a measure of a property of the object.