Imaging system for vehicle

• US 10,616,507 B2
• Filed: 06/18/2018
• Issued: 04/07/2020
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng, Higgins-Luthman, Michael J., Livengood, William R., Harris, James D.
• Primary Examiner(s): Nguyen, Phuoc H
• Application Number:
  US16/010,581
• Publication Number:
  US 20180302575A1
• Time in Patent Office: 659 Days
• Field of Search: None
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

A vehicular imaging system includes a camera with an imaging sensor. A red-pass spectral filter is disposed at each pixel of a first set of photosensing pixels, a blue-pass spectral filter is disposed at each pixel of a second set of photosensing pixels, a green-pass spectral filter is disposed at each pixel of a third set of photosensing pixels, and each pixel of a fourth set of photosensing pixels senses at least near infrared radiation. A near infrared illumination source is operable to emit non-visible near infrared illumination to enhance nighttime viewing by the camera. Outputs of the pixels of the first, second and third sets of photosensing pixels are processed for at least one of (i) a lane departure warning system and (ii) an object detection system. Output of the photosensing pixels of the fourth set of photosensing pixels is processed for a night vision system.

Claims

1. An imaging system suitable for use in a vehicle, said imaging system comprising:

a camera comprising an imaging sensor;

wherein said imaging sensor comprises a two-dimensional array of photosensing pixels;

wherein a red-pass spectral filter is disposed at each photosensing pixel of a first set of photosensing pixels of said two-dimensional array, whereby said first set of photosensing pixels primarily senses red visible light;

wherein a blue-pass spectral filter is disposed at each photosensing pixel of a second set of photosensing pixels of said two-dimensional array, whereby said second set of photosensing pixels primarily senses blue visible light;

wherein a green-pass spectral filter is disposed at each photosensing pixel of a third set of photosensing pixels of said two-dimensional array, whereby said third set of photosensing pixels primarily senses green visible light;

wherein each photosensing pixel of a fourth set of photosensing pixels of said two-dimensional array senses at least near infrared radiation;

wherein said camera is configured for attachment at an in-cabin side of a windshield of a vehicle equipped with said imaging system, and wherein, with said camera attached at the equipped vehicle, said camera has a field of view forward of the equipped vehicle through the windshield of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises output of said first set of photosensing pixels of said two-dimensional array, output of said second set of photosensing pixels of said two-dimensional array, output of said third set of photosensing pixels of said two-dimensional array and output of said fourth set of photosensing pixels of said two-dimensional array;

wherein said imaging system is associated with (i) a lane departure warning system of the equipped vehicle, (ii) an object detection system of the equipped vehicle and (iii) a night vision system of the equipped vehicle;

a near infrared illumination source disposed at a front portion of the equipped vehicle and operable to emit non-visible near infrared illumination to illuminate non-visibly forward of the equipped vehicle to enhance nighttime viewing by said camera, and wherein said near infrared illumination source, when operated, illuminates at least a portion of the field of view of said camera with non-visible near infrared illumination;

wherein outputs of said photosensing pixels of said first, second and third sets of photosensing pixels of said two-dimensional array of photosensing pixels are processed for at least one of (i) the lane departure warning system of the equipped vehicle and (ii) the object detection system of the equipped vehicle; and

wherein output of said photosensing pixels of said fourth set of photosensing pixels of said two-dimensional array of photosensing pixels is processed for the night vision system of the equipped vehicle.

2. The imaging system of claim 1, wherein said near infrared illumination source is disposed in a headlamp assembly of the equipped vehicle.

3. The imaging system of claim 2, wherein said near infrared illumination source, when operated, emits non-visible near infrared illumination that is modulated by pulse width modulation (PWM).

4. The imaging system of claim 3, wherein image data capture by said imaging sensor is synchronized with modulation of the non-visible near infrared illumination emitted by said near infrared illumination source of the headlamp assembly.

5. The imaging system of claim 3, wherein the non-visible near infrared illumination emitted by said near infrared illumination source of the headlamp assembly is encoded with a modulation sequence unique to the equipped vehicle to avoid other vehicles traveling ahead of the equipped vehicle interfering with said imaging system.

6. The imaging system of claim 1, wherein said imaging system is operable to generate color video images based on captured image data.

7. The imaging system of claim 1, wherein said near infrared illumination source comprises a near infrared light emitting diode.

8. The imaging system of claim 1, wherein a red filter and a blue filter are stacked one on top of the other to form a filter that blocks visible light at each photosensing pixel of said fourth set of photosensing pixels of said two-dimensional array.

9. The imaging system of claim 1, wherein said imaging system is associated with an intelligent headlamp control system of the equipped vehicle.

10. The imaging system of claim 1, wherein said near infrared illumination source comprises a near infrared emitting laser.

11. The imaging system of claim 1, wherein, responsive at least in part to processing by said image processor of output of said first set of photosensing pixels and with said camera attached at the equipped vehicle and viewing forward through the windshield of the equipped vehicle, said imaging system is operable to detect traffic signs.

12. The imaging system of claim 1, wherein, responsive at least in part to processing by said image processor of image data captured by said imaging sensor and with said camera attached at the equipped vehicle and viewing forward of the equipped vehicle, said imaging system is operable to detect objects located external of the equipped vehicle.

13. An imaging system suitable for use in a vehicle, said imaging system comprising:

a camera comprising an imaging sensor;

wherein said imaging sensor comprises a two-dimensional array of photosensing pixels;

wherein a red-pass spectral filter is disposed at each photosensing pixel of a first set of photosensing pixels of said two-dimensional array, whereby said first set of photosensing pixels primarily senses red visible light;

wherein a blue-pass spectral filter is disposed at each photosensing pixel of a second set of photosensing pixels of said two-dimensional array, whereby said second set of photosensing pixels primarily senses blue visible light;

wherein a green-pass spectral filter is disposed at each photosensing pixel of a third set of photosensing pixels of said two-dimensional array, whereby said third set of photosensing pixels primarily senses green visible light;

wherein each photosensing pixel of a fourth set of photosensing pixels of said two-dimensional array senses at least near infrared radiation;

wherein said camera is configured for attachment at an in-cabin surface of a windshield of a vehicle equipped with said imaging system, and wherein, with said camera attached at the equipped vehicle, said camera has a field of view forward of the equipped vehicle through the windshield of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises output of said first set of photosensing pixels of said two-dimensional array, output of said second set of photosensing pixels of said two-dimensional array, output of said third set of photosensing pixels of said two-dimensional array and output of said fourth set of photosensing pixels of said two-dimensional array;

wherein said imaging system is associated with at least one of (i) a lane departure warning system of the equipped vehicle, (ii) an object detection system of the equipped vehicle and (iii) an intelligent headlamp control system of the equipped vehicle;

wherein said imaging system is associated with a night vision system of the equipped vehicle;

a near infrared illumination source disposed at a portion of the equipped vehicle and operable to emit non-visible near infrared illumination to illuminate non-visibly forward of the equipped vehicle to enhance nighttime viewing by said camera, and wherein said near infrared illumination source, when operated, illuminates at least a portion of the field of view of said camera with non-visible near infrared illumination;

wherein outputs of said photosensing pixels of said first, second and third sets of photosensing pixels of said two-dimensional array of photosensing pixels are processed for the at least one of (i) the lane departure warning system of the equipped vehicle, (ii) the object detection system of the equipped vehicle and (iii) the intelligent headlamp control system of the equipped vehicle;

wherein output of said photosensing pixels of said fourth set of photosensing pixels of said two-dimensional array of photosensing pixels is processed for the night vision system of the equipped vehicle; and

wherein said near infrared illumination source, when operated, emits non-visible near infrared illumination that is encoded by modulation.

14. The imaging system of claim 13, wherein the non-visible near infrared illumination emitted by said near infrared illumination source, when said near infrared illumination source is operated, is modulated by pulse width modulation (PWM).

15. The imaging system of claim 14, wherein image data capture by said imaging sensor is synchronized with modulation of the non-visible near infrared illumination emitted by said near infrared illumination source.

16. The imaging system of claim 13, wherein the non-visible near infrared illumination emitted by said near infrared illumination source is encoded with a modulation sequence unique to the equipped vehicle to avoid other vehicles traveling ahead of the equipped vehicle interfering with said imaging system.

17. The imaging system of claim 16, wherein said imaging system is operable to generate color video images based on captured image data.

18. The imaging system of claim 17, wherein said near infrared illumination source comprises a near infrared light emitting diode.

19. The imaging system of claim 17, wherein said near infrared illumination source comprises a near infrared emitting laser.

20. The imaging system of claim 13, wherein said near infrared illumination source is disposed at a front portion of the equipped vehicle.

21. The imaging system of claim 13, wherein said near infrared illumination source is disposed in a headlamp assembly of the equipped vehicle.

22. The imaging system of claim 13, wherein said near infrared illumination source is disposed at the in-cabin side of the windshield of the equipped vehicle.

23. An imaging system suitable for use in a vehicle, said imaging system comprising:

a camera comprising an imaging sensor;

wherein said imaging sensor comprises a two-dimensional array of photosensing pixels;

wherein a red-pass spectral filter is disposed at each photosensing pixel of a first set of photosensing pixels of said two-dimensional array, whereby said first set of photosensing pixels primarily senses red visible light;

wherein a blue-pass spectral filter is disposed at each photosensing pixel of a second set of photosensing pixels of said two-dimensional array, whereby said second set of photosensing pixels primarily senses blue visible light;

wherein a green-pass spectral filter is disposed at each photosensing pixel of a third set of photosensing pixels of said two-dimensional array, whereby said third set of photosensing pixels primarily senses green visible light;

wherein each photosensing pixel of a fourth set of photosensing pixels of said two-dimensional array senses at least near infrared radiation;

wherein said camera is configured for attachment at an in-cabin surface of a windshield of a vehicle equipped with said imaging system, and wherein, with said camera attached at the equipped vehicle, said camera has a field of view forward of the equipped vehicle through the windshield of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises output of said first set of photosensing pixels of said two-dimensional array, output of said second set of photosensing pixels of said two-dimensional array, output of said third set of photosensing pixels of said two-dimensional array and output of said fourth set of photosensing pixels of said two-dimensional array;

wherein said imaging system is associated with at least one of (i) a lane departure warning system of the equipped vehicle, (ii) an object detection system of the equipped vehicle and (iii) an intelligent headlamp control system of the equipped vehicle;

wherein said imaging system is associated with a night vision system of the equipped vehicle;

a near infrared illumination source disposed at a portion of the equipped vehicle and operable to emit non-visible near infrared illumination to illuminate non-visibly forward of the equipped vehicle to enhance nighttime viewing by said camera, and wherein said near infrared illumination source, when operated, illuminates at least a portion of the field of view of said camera with non-visible near infrared illumination;

wherein outputs of said photosensing pixels of said first, second and third sets of photosensing pixels of said two-dimensional array of photosensing pixels are processed for the at least one of (i) the lane departure warning system of the equipped vehicle, (ii) the object detection system of the equipped vehicle and (iii) the intelligent headlamp control system of the equipped vehicle;

wherein output of said photosensing pixels of said fourth set of photosensing pixels of said two-dimensional array of photosensing pixels is processed for the night vision system of the equipped vehicle;

wherein said near infrared illumination source, when operated, emits non-visible near infrared illumination that is modulated; and

wherein the non-visible near infrared illumination emitted by said near infrared illumination source is encoded with a modulation sequence unique to the equipped vehicle to avoid other vehicles traveling ahead of the equipped vehicle interfering with said imaging system.

24. The imaging system of claim 23, wherein the non-visible near infrared illumination emitted by said near infrared illumination source, when said near infrared illumination source is operated, is modulated by pulse width modulation (PWM).

25. The imaging system of claim 24, wherein image data capture by said imaging sensor is synchronized with modulation of the non-visible near infrared illumination emitted by said near infrared illumination source.

26. The imaging system of claim 23, wherein said near infrared illumination source comprises a near infrared light emitting diode.

27. The imaging system of claim 23, wherein said near infrared illumination source comprises a near infrared emitting laser.

28. The imaging system of claim 23, wherein said near infrared illumination source is disposed at a front portion of the equipped vehicle.

29. The imaging system of claim 28, wherein said near infrared illumination source is disposed in a headlamp assembly of the equipped vehicle.

30. The imaging system of claim 23, wherein said near infrared illumination source is disposed at the in-cabin side of the windshield of the equipped vehicle.