Method for providing image data from a camera system, camera system and motor vehicle

• US 10,692,192 B2
• Filed: 10/20/2015
• Issued: 06/23/2020
• Est. Priority Date: 10/21/2014
• Status: Active Grant

• Current Assignee: Connaught Electronics Limited (Valeo SE)
• Original Assignee: Connaught Electronics Limited (Valeo SE)
• Inventors: Denny, Patrick Eoghan, Deegan, Brian Michael Thomas
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US15/520,659
• Publication Number:
  US 20170308999A1
• Time in Patent Office: 1,708 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

The invention relates to a method for providing image data (24) from a camera system (3) for a motor vehicle (1), wherein the camera system (3) includes at least one camera, in particular a plenoptic camera (4), including a lens (6) and a sensor array (7), in which electromagnetic radiation (15, 17, 19, 21) is captured by means of the sensor array (7) and image data (24) of an environmental region (11) of the motor vehicle (1) is provided based on the captured electromagnetic radiation (15, 17, 19, 21) and the image data (24) is evaluated by means of an evaluation device (5), wherein a direction of incidence of the electromagnetic radiation (15, 17, 19, 21) on the sensor array (7) is determined by the evaluation device (5) based on the image data (24) provided by the sensor array (7) and the image data (24) is adapted by means of the evaluation device (5) depending on the determined direction of incidence.

Claims

1. A method for providing image data from a camera system for a motor vehicle, the method comprising:

capturing electromagnetic radiation by a sensor array;

capturing first sensor data from an infrared wavelength range of an electromagnetic radiation by a first sensor of the sensor array;

capturing further sensor data in a visible wavelength range of the electromagnetic radiation by a second sensor of the sensor array;

providing image data of an environmental region of the motor vehicle based on the electromagnetic radiation captured from the first sensor and the second sensor;

evaluating the image data by an evaluation device;

determining a direction of incidence of the electromagnetic radiation on the sensor array by the evaluation device based on the image data provided by the sensor array; and

adapting the image data by the evaluation device based on the determined direction of incidence; and

adapting the image data based on at least a first approximation image provided by low-pass filtering of first sensor data, and/or adapting the image data based on at least a second approximation image provided by low-pass filtering of the further sensor data; and

weighting portions of the sensor data in the respective infrared wavelength range and visible wavelength range based on brightness of the environmental region to obtain optimally exposed image data, wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decreased with increasing brightness of the environmental region,

wherein the image data is additionally adapted depending on the first sensor data,

wherein the camera system comprises a single plenoptic camera that comprises a lens and the sensor array,

wherein the sensor array comprises the first sensor and the second sensor,

and wherein the camera system is mounted on the vehicle,

wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decrease with increasing brightness of the environmental region in accordance with the equation;

Image data=f((g*IR)+(1−

g)*C), and

wherein f is a function for generating the image data, g is a parameter of an automatic gain control, IR is first sensor data of the first sensor from the infrared wavelength range and C is further sensor data from other sensors from the visible wavelength range.

2. The method according to claim 1, wherein hazy and/or misty areas of the image data are adapted depending on the determined direction of incidence.

3. The method according to claim 1, wherein a near infrared range of the electromagnetic radiation is captured as the infrared wavelength range.

4. The method according to claim 1, wherein the camera system has at least two cameras, the method further comprising:

determining a baseline describing a distance between the at least two cameras; and

adapting the image data depending on the determined baseline.

5. The method according to claim 4, wherein an image depth value of the image data is determined based on the baseline, by which a distance to an object in the environmental region is described.

6. The method according to claim 5, wherein:

a first contrast image determining a foreground of the first approximation image is provided based on the first approximation image,

a first contrast image determining a background of the first approximation image is provided based on the first approximation image,

a second contrast image describing a foreground of the second approximation image is provided based on the second approximation image, or

a second contrast image describing a background of the second approximation image is provided based on the second approximation image.

7. The method according to claim 6, wherein at least one selected from a group consisting of:

the first approximation image, the second approximation image, the first contrast image, and the second contrast image is/are provided in different resolutions.

8. The method according to claim 1, wherein the image data is adapted depending on a current GNSS position of the camera system.

9. The method according to claim 1, further comprising

determining a current distance between the camera system and an object in the environmental region;

and

adapting the image data depending on the determined distance.

10. A camera system for a motor vehicle comprising:

a single plenoptic camera that comprises a lens and a sensor array for capturing electromagnetic radiation and for providing image data of an environmental region of the motor vehicle based on the captured electromagnetic radiation from a first sensor and at least one second sensor; and

an evaluation device that evaluates the image data,

wherein the camera system is mounted on the vehicle;

wherein the sensor array comprises the first sensor and the at least one second sensor;

wherein the first sensor captures first sensor data from an infrared wavelength range of the electromagnetic radiation;

wherein the at least one second sensor captures further sensor data in a visible wavelength range of the electromagnetic radiation; and

wherein the evaluation device;

evaluates the image data,

determines a direction of incidence of the electromagnetic radiation on the sensor array based on the image data provided by the sensor array,

adapts the image data based on the determined direction of incidence,

adapts the image data based on at least a first approximation image provided by low-pass filtering of first sensor data, and/or adapts the image data based on at least a second approximation image provided by low-pass filtering of the further sensor data,

adapts the image data based on the first sensor data, and

weighs portions of the sensor data in the respective infrared wavelength range and visible wavelength range based on brightness of the environmental region to obtain optimally exposed image data, wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decreased with increasing brightness of the environmental region,

wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decrease with increasing brightness of the environmental region in accordance with the equation;

Image data=f((g*IR)+(1−

g)*C), and

wherein f is a function for generating the image data, g is a parameter of an automatic gain control, IR is first sensor data of the first sensor from the infrared wavelength range and C is further sensor data from other sensors from the visible wavelength range.

11. The camera system according to claim 10, wherein the lens is formed as a microlens array.

12. A motor vehicle comprising a camera system, the camera system being mounted on the motor vehicle the camera system comprises:

a single plenoptic camera that comprises a lens and a sensor array for capturing electromagnetic radiation and for providing image data of an environmental region of the motor vehicle based on the captured electromagnetic radiation from a first sensor and at least one second sensor; and

an evaluation device that evaluates the image data,

wherein the sensor array comprises a first sensor and at least one second sensor;

wherein the first sensor captures first sensor data from an infrared wavelength range of the electromagnetic radiation;

wherein the at least one second sensor captures further sensor data in a visible wavelength range of the electromagnetic radiation; and

wherein the evaluation device;

evaluates the image data,

determines a direction of incidence of the electromagnetic radiation on the sensor array based on the image data provided by the sensor array,

adapts the image data based on the determined direction of incidence,

adapts the image data based on at least a first approximation image provided by low-pass filtering of first sensor data, and/or adapts the image data based on at least a second approximation image provided by low-pass filtering of the further sensor data, and

adapts the image data based on the first sensor data, and

weighs portions of the sensor data in the respective infrared wavelength range and visible wavelength range based on brightness of the environmental region to obtain optimally exposed image data, wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decreased with increasing brightness of the environmental region,

wherein the portion of the infrared wavelength range in the image data is increased with decreasing brightness of the environmental region and decrease with increasing brightness of the environmental region in accordance with the equation;

Image data=f((g*IR)+(1−

g)*C), and

wherein f is a function for generating the image data, g is a parameter of an automatic gain control, IR is first sensor data of the first sensor from the infrared wavelength range and C is further sensor data from other sensors from the visible wavelength range.