Object Detection Enhancement of Reflection-Based Imaging Unit

• US 20170270375A1
• Filed: 12/03/2015
• Published: 09/21/2017
• Est. Priority Date: 12/07/2014
• Status: Active Grant

• Current Assignee: BrightWay Vision Ltd.
• Original Assignee: BrightWay Vision Ltd.
• Inventors: Grauer, Yoav
• Granted Patent: US 9,904,859 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G02B 23/12 with means for image conversion or intensification
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces

Abstract

Imaging system and method, the system including a main detection unit, an auxiliary detection unit, an image processor, and a controller. The main detection unit includes a light source that emits light pulses and a gated image sensor that receives reflections of the light pulses reflected from a selected depth of field in the environment and converts the reflections into a reflection-based image. The auxiliary detection unit includes a thermal sensor that detects infrared radiation emitted from the environment and generates an emission-based image. The image processor processes and detects at least one region of interest in the acquired reflection-based image and/or acquired emission-based image. The controller adaptively controls at least one detection characteristic of a detection unit based on information obtained from the other detection unit. The image processor detects at least one object of interest in the acquired reflection-based image and/or acquired emission-based image.

Claims

1. An imaging system comprising:

a main detection unit, configured to acquire reflection-based images of an environment to be imaged, said main detection unit comprising;

at least one light source, configured to emit light pulses toward said environment; and

at least one gated image sensor, configured to receive reflections of said light pulses reflected from a selected depth of field (DOF) in said environment, and to convert said reflections into a reflection-based image,

an auxiliary detection unit, configured to acquire emission-based images of said environment, said auxiliary detection unit comprising;

at least one thermal sensor configured to detect infrared radiation emitted from said environment, and to generate an emission-based image,

an image processor, configured to process and to detect at least one region of interest (ROI) in at least one acquired reflection-based image and at least one acquired emission-based image; and

a controller, configured to adaptively control at least one of detection characteristic of at least one of;

said main detection unit; and

said auxiliary detection unit, based on information obtained from the other one of;

said main detection unit; and

said auxiliary detection unit,

wherein said image processor is further configured to detect at least one object of interest in at least one of;

said reflection-based image; and

said emission-based image.

2. The imaging system of claim 1, wherein said controller is configured to direct said main detection unit toward at least one ROI detected in said emission-based image.

3. The imaging system of claim 1, wherein said controller is configured to selectively modify at least one operating parameter of said main detection unit, said operating parameter selected from the group consisting of:

a gating parameter;

a pulse width;

a pulse intensity;

a pulse shape;

a cycle duration of at least one of said gating cycles;

a delay time of at least one of said gating cycles;

at least one depth of field (DOF);

a maximum range to be imaged;

the field of view (FOV) of said image sensor;

the field of illumination (FOI) of said light source;

the sensitivity of said image sensor;

the frame rate of said image sensor; and

any combination thereof.

4. The imaging system of claim 1, wherein said controller is configured to selectively modify at least one operating parameter of said auxiliary detection unit, said operating parameter selected from the group consisting of:

the direction of said thermal sensor;

the FOV of said thermal sensor;

the sensor gain of said thermal sensor; and

any combination thereof.

5. The imaging system of claim 1, wherein said image processor is further configured to generate a merged image by fusing at least a portion of said emission-based image with at least a portion of said reflection-based image.

6. The imaging system of claim 5, further comprising a display, configured to display at least one image selected from the group consisting of:

said reflection-based image;

said emission-based image; and

said merged image.

7. The imaging system of claim 1, further comprising a supplementary detector, configured to obtain supplementary information related to said environment.

8. The imaging system of claim 7, wherein said supplementary information is selected from the group consisting of:

ambient light in said environment;

weather or climate conditions in said environment;

time or date;

location or driving route of a vehicle on which said system is mounted;

the velocity or acceleration of said vehicle; and

the relative distance to potential objects in said environment.

9. The imaging system of claim 8, wherein said controller is configured to selectively modify at least one operating parameter of at least one of:

said main detection unit; and

said auxiliary detection unit, further based on said supplementary information.

10. The imaging system of claim 1, further comprising a database comprising thermal signature data associated with different objects, wherein said image processor is configured to detect said object of interest in accordance with said thermal signature data.

11. The imaging system of claim 1, further configured to provide an indication of at least one detected object of interest.

12. The imaging system of claim 11, wherein said indication is selected from the group consisting of:

a visual notification;

a tactile notification;

an audio notification;

visually indicating said object of interest on a displayed image;

illuminating said object of interest in said environment;

an indication of a potential road hazard of a vehicle on which said system is mounted;

a forward collision warning (FCW);

a lane departure warning (LDW);

a traffic sign recognition (TSR); and

any combination thereof.

13. The imaging system of claim 1, wherein said main detection unit is operative in a spectral range selected from the group consisting of:

visible;

near infrared (NIR);

short wave infrared (SWIR); and

any combination thereof.

14. The imaging system of claim 1, wherein said auxiliary detection unit is operative in a spectral range selected from the group consisting of:

short wave infrared (SWIR);

medium wave infrared (MWIR);

far infrared (FIR); and

any combination thereof.

15. The imaging system of claim 1, wherein the resolution of said thermal sensor is lower than the resolution of said image sensor.

16. The imaging system of claim 1, mounted on a vehicle.

17. A vehicle, comprising the imaging system of claim 1.

18. An imaging method comprising the procedures of:

acquiring reflection-based images of an environment with a main detection unit, by;

emitting light pulses toward said environment, using at least one light source; and

receiving reflections of said pulses reflected from a selected depth of field (DOF) in said environment and converting said reflections into a reflection-based image, using at least one gated image sensor;

acquiring emission-based images of said environment with an auxiliary detection unit, by detecting infrared radiation emitted from said environment and generating an emission-based image, using a thermal sensor;

processing and detecting at least one region of interest (ROI) in at least one acquired reflection-based image and at least one acquired emission-based image;

adaptively controlling at least one detection characteristic of at least one of;

said main detection unit; and

said auxiliary detection unit, based on information obtained from the other one of;

said main detection unit; and

said auxiliary detection unit; and

detecting at least one object of interest in at least one of;

said reflection-based image; and

said emission-based image.

19. The imaging method of claim 18, wherein said procedure of adaptively controlling at least one detection characteristic comprises directing said main detection unit toward at least one ROI detected in said emission-based image.

20. The imaging method of claim 18, wherein said procedure of adaptively controlling at least one detection characteristic comprises selectively modifying at least one operating parameter of said main detection unit, said operating parameter selected from the group consisting of:

a gating parameter;

a pulse width;

a pulse intensity;

a pulse shape;

a cycle duration of at least one of said gating cycles;

a delay time of at least one of said gating cycles;

at least one depth of field (DOF);

a maximum range to be imaged;

the field of view (FOV) of said image sensor;

the field of illumination (FOI) of said light source;

the sensitivity of said image sensor;

the frame rate of said image sensor; and

any combination thereof.

21. The imaging method of claim 18, wherein said procedure of adaptively controlling at least one detection characteristic comprises selectively modifying at least one operating parameter of said auxiliary detection unit, said operating parameter selected from the group consisting of:

the direction of said thermal sensor;

the FOV of said thermal sensor;

the sensor gain of said thermal sensor; and

any combination thereof.

22. The imaging method of claim 18, further comprising the procedure of generating a merged image by fusing at least a portion of said emission-based image with at least a portion of said reflection-based image.

23. The imaging method of claim 22, further comprising the procedure of displaying at least one image selected from the group consisting of:

said reflection-based image;

said emission-based image; and

said merged image.

24. The imaging method of claim 18, further comprising the procedure of obtaining supplementary information related to said environment.

25. The imaging method of claim 24, wherein said supplementary information is selected from the group consisting of:

ambient light in said environment;

weather or climate conditions in said environment;

time or date;

location or driving route of a vehicle on which said system is mounted;

the velocity or acceleration of said vehicle; and

the relative distance to potential objects in said environment.

26. The imaging method of claim 24, wherein said procedure of adaptively controlling at least one detection characteristic, is further performed based on said supplementary information.

27. The imaging method of claim 18, further comprising the procedure of providing an indication of at least one detected object of interest.

28. The imaging method of claim 18, wherein said indication is selected from the group consisting of:

a visual notification;

a tactile notification;

an audio notification;

visually indicating said object of interest on a displayed image;

illuminating said object of interest in said environment;

an indication of a potential road hazard of a vehicle on which said system is mounted;

a forward collision warning (FCW);

a lane departure warning (LDW);

a traffic sign recognition (TSR); and

any combination thereof.