Imaging system for vehicle

• US 10,003,755 B2
• Filed: 12/08/2014
• Issued: 06/19/2018
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng, Higgins-Luthman, Michael J., Livengood, William R., Harris, James D.
• Primary Examiner(s): Nguyen, Phuoc
• Application Number:
  US14/562,826
• Publication Number:
  US 20150092059A1
• Time in Patent Office: 1,289 Days
• Field of Search: 348148
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

An imaging system suitable for use in a vehicle includes an imaging sensor having a two-dimensional array of photosensing pixels that includes at least one sub-array having a first photosensing pixel, a second photosensing pixel, a third photosensing pixel and a fourth photosensing pixel. Spectral filtering is disposed at the photosensing pixels whereby the first photosensing pixel primarily senses red visible light, the second photosensing primarily senses blue visible light, the third photosensing primarily senses green visible light and the fourth photosensing primarily senses infrared radiation. The imaging sensor is disposed at a vehicle and has a field of view external of the vehicle. An image processor is operable for processing image data captured by the imaging sensor. The image data includes outputs of the first, second, third and fourth photosensing pixels. The processing of image data by the image processor may include de-mosaicing that reduces infrared color wash-out.

Claims

1. An imaging system suitable for use in a vehicle, said imaging system comprising:

an imaging sensor comprising a two-dimensional array of photosensing pixels, said two-dimensional array comprising a plurality of sub-arrays, each consisting of four photosensing pixels arranged in two rows and two columns, wherein each of said sub-arrays comprises a first photosensing pixel, a second photosensing pixel, a third photosensing pixel and a fourth photosensing pixel;

wherein spectral filtering is disposed at said first photosensing pixel whereby said first photosensing pixel primarily senses red visible light;

wherein spectral filtering is disposed at said second photosensing pixel whereby said second photosensing primarily senses blue visible light;

wherein spectral filtering is disposed at said third photosensing pixel whereby said third photosensing primarily senses green visible light;

wherein spectral filtering is disposed at said fourth photosensing pixel whereby said fourth photosensing primarily senses infrared radiation;

wherein said imaging sensor is disposed at a vehicle equipped with said imaging system and wherein, with said imaging sensor disposed at the equipped vehicle, said imaging sensor has a field of view forward of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises an output of said first photosensing pixel of each sub-array, an output of said second photosensing pixel of each sub-array, an output of said third photosensing pixel of each sub-array and an output of said fourth photosensing pixel of each sub-array;

wherein said imaging system is associated with (i) a lane departure warning system of the equipped vehicle, (ii) an object detection system of the equipped vehicle and (iii) a night vision system of the equipped vehicle;

an infrared illumination source disposed at the equipped vehicle and operable to emit at least near infrared illumination to illuminate forward of the equipped vehicle to enhance nighttime viewing by said imaging sensor, and wherein said infrared illumination source, when activated, illuminates at least a portion of said field of view of said imaging sensor with infrared illumination;

wherein the outputs of said first, second and third photosensing pixels of sub-arrays are processed for at least one of (i) the lane departure warning system of the equipped vehicle and (ii) the object detection system of the equipped vehicle; and

wherein the output of at least said fourth photosensing pixel of sub-arrays is processed for the night vision system of the equipped vehicle.

2. The imaging system of claim 1, wherein said processing of image data by said image processor comprises de-mosaicing, and wherein said de-mosaicing reduces infrared color wash-out, and wherein said de-mosaicing comprises rolling window de-mosaicing.

3. The imaging system of claim 2, wherein said rolling window de-mosaicing comprises 2×

2 window de-mosaicing.

4. The imaging system of claim 2, wherein said rolling window de-mosaicing comprises 3×

3 window de-mosaicing.

5. The imaging system of claim 1, wherein said imaging system is associated with an intelligent headlamp control system of the equipped vehicle.

6. The imaging system of claim 1, wherein said near infrared illumination source comprises at least one of (i) a near infrared light-emitting light emitting diode and (ii) a near infrared light-emitting laser.

7. The imaging system of claim 1, wherein, with said imaging sensor disposed at the equipped vehicle, said imaging sensor has a field of view forward through a windshield of the equipped vehicle, and wherein, responsive at least in part to processing by said image processor of the output of said first pixel of said at least one sub-array and with said imaging sensor disposed at the equipped vehicle and viewing forward through the windshield of the equipped vehicle, said imaging system is operable to detect traffic signs.

8. The imaging system of claim 1, wherein, responsive at least in part to processing by said image processor of image data captured by said imaging sensor and with said imaging sensor disposed at the equipped vehicle and viewing forward of the equipped vehicle, said imaging system is operable to detect reflections by objects located external of the equipped vehicle.

9. The imaging system of claim 1, wherein said two-dimensional array of photosensing pixels comprises at least 262,144 photosensing pixels.

10. The imaging system of claim 1, wherein said imaging sensor comprises part of a rear viewing color camera that is operable to capture video image data.

11. An imaging system suitable for use in a vehicle, said imaging system comprising:

an imaging sensor comprising a two-dimensional array of photosensing pixels, said two-dimensional array comprising a plurality of sub-arrays, each consisting of four photosensing pixels arranged in two rows and two columns, wherein each of said sub-arrays comprises a first photosensing pixel, a second photosensing pixel, a third photosensing pixel and a fourth photosensing pixel;

wherein spectral filtering is disposed at said first photosensing pixel whereby said first photosensing pixel primarily senses red visible light;

wherein spectral filtering is disposed at said second photosensing pixel whereby said second photosensing primarily senses blue visible light;

wherein spectral filtering is disposed at said third photosensing pixel whereby said third photosensing primarily senses green visible light;

wherein spectral filtering is disposed at said fourth photosensing pixel whereby said fourth photosensing primarily senses infrared radiation;

wherein said imaging sensor is disposed at a vehicle equipped with said imaging system and wherein, with said imaging sensor disposed at the equipped vehicle, said imaging sensor has a field of view rearward of the equipped vehicle, and wherein said imaging system is associated with a rear backup system of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises an output of said first photosensing pixel of each sub-array, an output of said second photosensing pixel of each sub-array, an output of said third photosensing pixel of each sub-array and an output of said fourth photosensing pixel of each sub-array;

wherein said two-dimensional array of photosensing pixels comprises at least 262,144 photosensing pixels;

wherein said imaging sensor comprises part of a rear viewing color camera that is operable to capture video image data;

wherein the equipped vehicle is equipped with ultrasonic sensors, and wherein an indication is given to a driver of the equipped vehicle of a presence of an object adjacent the equipped vehicle based, at least in part, on sensing by said ultrasonic sensors and on image processing by said image processor of video image data captured by said rear viewing color camera;

an infrared illumination source disposed at the equipped vehicle and operable to emit at least near infrared illumination to illuminate rearward of the equipped vehicle to enhance nighttime viewing by said imaging sensor, and wherein said infrared illumination source, when activated, illuminates at least a portion of said field of view of said imaging sensor with infrared illumination;

wherein the outputs of said first, second and third photosensing pixels of sub-arrays are processed for object detection; and

wherein the output of at least said fourth photosensing pixel of sub-arrays is processed for a night vision system of the equipped vehicle.

12. The imaging system of claim 11, wherein said processing of image data by said image processor comprises de-mosaicing, and wherein said de-mosaicing reduces infrared color wash-out, and wherein said de-mosaicing comprises rolling window de-mosaicing.

13. The imaging system of claim 11, wherein said near infrared illumination source comprises at least one of (i) a near infrared light-emitting light emitting diode and (ii) a near infrared light-emitting laser.

14. An imaging system suitable for use in a vehicle, said imaging system comprising:

an imaging sensor comprising a two-dimensional array of photosensing pixels, said two-dimensional array comprising a plurality of sub-arrays, each consisting of four photosensing pixels arranged in two rows and two columns, wherein each of said sub-arrays comprises a first photosensing pixel, a second photosensing pixel, a third photosensing pixel and a fourth photosensing pixel;

wherein spectral filtering is disposed at said first photosensing pixel whereby said first photosensing pixel primarily senses red visible light;

wherein spectral filtering is disposed at said second photosensing pixel whereby said second photosensing primarily senses blue visible light;

wherein spectral filtering is disposed at said third photosensing pixel whereby said third photosensing primarily senses green visible light;

wherein spectral filtering is disposed at said fourth photosensing pixel whereby said fourth photosensing primarily senses infrared radiation;

wherein said imaging sensor is disposed at a vehicle equipped with said imaging system and wherein, with said imaging sensor disposed at the equipped vehicle, said imaging sensor has a field of view forward of the equipped vehicle;

wherein said imaging system is associated with (i) a lane departure warning system of the equipped vehicle, (ii) an object detection system of the equipped vehicle and (iii) a night vision system of the equipped vehicle;

an image processor operable for processing image data captured by said imaging sensor;

wherein said image data comprises an output of said first photosensing pixel of each sub-array, an output of said second photosensing pixel of each sub-array, an output of said third photosensing pixel of each sub-array and an output of said fourth photosensing pixel of each sub-array;

wherein said two-dimensional array of photosensing pixels comprises at least 262,144 photosensing pixels;

an infrared illumination source disposed at the equipped vehicle and operable to emit at least near infrared illumination to illuminate forward of the equipped vehicle to enhance nighttime viewing by said imaging sensor, and wherein said infrared illumination source, when activated, illuminates at least a portion of said field of view of said imagine sensor with infrared illumination;

wherein the outputs of said first, second and third photosensing pixels of sub-arrays are processed for at least one of (i) the lane departure warning system of the equipped vehicle and (ii) the object detection system of the equipped vehicle; and

wherein the output of at least said fourth photosensing pixel of sub-arrays is processed for the night vision system of the equipped vehicle.

15. The imaging system of claim 14, wherein said imaging system is associated with an automatic headlamp control system of the equipped vehicle, and wherein, responsive at least in part to processing by said image processor of the output of said first pixel of said at least one sub-array and with said imaging sensor disposed at the equipped vehicle and viewing forwardly through a windshield of the equipped vehicle, said imaging system is operable to detect a taillight of a vehicle ahead of the equipped vehicle.

16. The imaging system of claim 15, wherein, responsive at least in part to processing of image data captured by said imaging sensor and with said imaging sensor disposed at the equipped vehicle and viewing forward of the equipped vehicle, said imaging system is operable to detect reflections by objects located external of the equipped vehicle.

17. The imaging system of claim 14, wherein said near infrared illumination source comprises at least one of (i) a near infrared light-emitting light emitting diode and (ii) a near infrared light-emitting laser.

18. The imaging system of claim 17, wherein said processing of image data by said image processor comprises de-mosaicing, and wherein said de-mosaicing reduces infrared color wash-out, and wherein said de-mosaicing comprises rolling window de-mosaicing.

19. The imaging system of claim 18, wherein said rolling window de-mosaicing comprises 2×

2 window de-mosaicing.