System and method for image presentation by a vehicle driver assist module

• US 10,504,214 B2
• Filed: 11/16/2018
• Issued: 12/10/2019
• Est. Priority Date: 02/03/2017
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Chakraborty, Pranjal, Biswal, Rajesh, Mukherjee, Debashis
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US16/193,650
• Publication Number:
  US 20190087944A1
• Time in Patent Office: 389 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/60 : using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/73 : Deblurring; Sharpening
• CPC Class Codes: G06V 10/454 : Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

The present disclosure relates to systems, devices and methods for image presentation for vehicle driver assistance. In one embodiment, a method includes receiving image data representing an outside of a vehicle and enhancing the image data. Image enhancement can include inverting the image data, running a dehazing operation based on scene radiance, medium transmission and atmospheric light of the image data. The method may also include generating reconstructed image data from the enhanced image data by performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation. The method also includes outputting the reconstructed image data to a display unit of the vehicle. Vehicle driver assist systems and devices are provided to enhance image data detected by vehicle cameras and wearable devices.

Claims

1. A method for image presentation by a vehicle driver assist module, the method comprising:

receiving, by a driver assist module, image data representing an outside of a vehicle, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle;

inverting, by the driver assist module, the image data to form inverted image data;

running, by the driver assist module, a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;

inverting, by the driver assist module, the enhanced inverted image data to generate enhanced image data;

generating, by the driver assist module, reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by

performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and

performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and

outputting, by the driver assist module, the reconstructed image data to a display unit of the vehicle.

2. The method of claim 1, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

3. The method of claim 1, wherein the device includes a first camera and a second camera mounted to an eyewear frame.

4. The method of claim 3, wherein the first camera and the second camera provide real time video capture.

5. The method of claim 1, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

6. The method of claim 1, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

7. The method of claim 1, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.

8. A vehicle driver assist module comprising:

an input module to receive image data representing an outside of a vehicle, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle;

an output module; and

an image enhancer module coupled to the input module and output module, the image enhancer module including a processor configured to

invert the image data to form inverted image data;

run a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;

invert the enhanced inverted image data to generate enhanced image data;

generate reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by

performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and

performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and

control the output module to output the reconstructed image data to a display unit of the vehicle.

9. The device of claim 8, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

10. The device of claim 8, wherein the device includes a first camera and a second camera mounted to an eyewear frame.

11. The device of claim 10, wherein the first camera and the second camera provide real time video capture.

12. The device of claim 8, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

13. The device of claim 8, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

14. The device of claim 8, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.

15. A method for image presentation by a vehicle driver assist module, the method comprising:

receiving, by a driver assist module, image data representing an outside of a vehicle;

inverting, by the driver assist module, the image data to form inverted image data;

running, by the driver assist module, a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;

inverting, by the driver assist module, the enhanced inverted image data to generate enhanced image data;

generating, by the driver assist module, reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by

performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and

performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and

outputting, by the driver assist module, the reconstructed image data to a display unit of the vehicle, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

16. The method of claim 15, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

17. The method of claim 15, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle, and wherein the device includes a first camera and a second camera mounted to an eyewear frame.

18. The method of claim 17, wherein the first camera and the second camera provide real time video capture.

19. The method of claim 15, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

20. The method of claim 15, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.