Apparatus and method for combining lane information with a far-infrared night vision image

• US 9,128,290 B2
• Filed: 04/19/2012
• Issued: 09/08/2015
• Est. Priority Date: 11/25/2011
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Kim, Seok Gyu
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): EDWARDS, TYLER B
• Application Number:
  US13/450,924
• Publication Number:
  US 20130135478A1
• Time in Patent Office: 1,237 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: G02B 23/12 with means for image conversion or intensification

Abstract

The present invention provides an apparatus and method for combining lane information with a far-infrared image, by combining an image taken by a far-infrared image capturing device with a lane information image, and displaying the combined image such that a driver can visualize the location and position of the road lanes on a night vision image that allows visualization of objects such as, for example, animals, people, vehicles, and the like, in the road at night, or in inclement weather conditions, thereby ensuring safe driving while preventing the driver from veering out of a road lane.

Claims

1. A night vision system of a vehicle comprising:

a lane recognition module comprising a lane detection image capturing device and a first controller configured to receive lane detection image data from the lane detection image capturing device and output the lane detection image data to a second controller;

a far-infrared night vision module comprising a far-infrared image capturing device and the second controller, wherein the second controller is configured to receive the lane detection image data from the first controller, receive a far-infrared image from the far-infrared image capturing device, set the far-infrared image as a reference image with reference coordinates and correct the lane detection image data to match the reference coordinates, and then, combine the corrected lane detection image data from the first controller with the far-infrared image such that lanes from the lane detection image data are matched to the far-infrared image based on the reference coordinates of the far-infrared image, and output the combined image to a display means; and

a thermal sensor and a far-infrared lamp installed around a fog light or head light to carry out an imaging operation of the far-infrared image capturing device such that the far-infrared lamp emits far-infrared rays toward a subject area in front of the vehicle and, at the same time, the thermal sensor detects heat of an object in front of the vehicle,

wherein the far-infrared image capturing device takes the far-infrared image that is an image of a road in front of the vehicle, and the image is transmitted to the second controller, and at the same that the far-infrared image capturing device takes the image, the lane detection image capturing device takes a lane information image of the road, which is transmitted to the first controller.

2. The system of claim 1, wherein the far-infrared image capturing device is located on the front bumper, the front spoiler, the front grill, or the windshield edge of the vehicle.

3. The system of claim 1, wherein the far-infrared image capturing device is located on the upper windshield edge of the vehicle.

4. A method for combining a lane information image with a far-infrared image comprising:

capturing a lane information image with a lane recognition module comprising a lane detection image capturing device and a first controller configured to receive lane detection image data from the lane detection image capturing device and output the lane detection image data to a second controller;

capturing a far-infrared image with a far-infrared night vision module comprising a far-infrared image capturing device and the second controller, wherein the second controller is configured to receive the lane detection image data from the first controller and a far-infrared image from the far-infrared image capturing device;

setting the far-infrared image as a reference image with reference coordinates and correcting the lane information image to match the reference coordinates, and then, combining the image data having the corrected lane information image from the first controller with the far-infrared image from the second controller such that lanes from the lane detection image data are matched to the far-infrared image based on the reference coordinates of the far-infrared image; and

outputting the combined image to a display means,

wherein a thermal sensor and a far-infrared lamp are installed around a fog light or head light to carry out an imaging operation of the far-infrared image capturing device such that the far-infrared lamp emits far-infrared rays toward a subject area in front of the vehicle and, at the same time, the thermal sensor detects heat of an object in front of the vehicle,

wherein the far-infrared image capturing device takes the far-infrared image that is an image of a road in front of the vehicle, and the image is transmitted to the second controller, and at the same that the far-infrared image capturing device takes the image, the lane detection image capturing device takes the lane information image of the road, which is transmitted to the first controller.

5. The method of claim 4, further comprising storing the reference coordinates of the far-infrared image and the corrected lane information image in the first controller of the lane recognition module or in the second controller of the far-infrared night vision module.

6. A non-transitory computer readable medium containing program instructions executed by at least one controller, the computer readable medium comprising:

program instructions that request at least one lane information image from a lane recognition module comprising a lane detection image capturing device and a first controller configured to receive lane detection image data from the lane detection image capturing device and output the lane detection image data to a second controller;

program instructions that request at least one far-infrared image from a far-infrared night vision module comprising a far-infrared image capturing device and the second controller, wherein the second controller is configured to receive the lane detection image data from the first controller and a far-infrared image from the far-infrared image capturing device;

program instructions that transmit the at least one image from the lane detection module to the far-infrared night vision module;

program instructions that data process the at least one image from the lane detection module and the at least one image from the far-infrared detection image capturing device, set the far-infrared image as a reference image with reference coordinates and correct the lane information image to match the reference coordinates, and then, combine the image data having the corrected lane information image from the first controller with the far-infrared image from the second controller such that lanes from the lane detection image data are matched to the far-infrared image based on the reference coordinates of the far-infrared image, and create at least one combined image; and

program instructions that display the at least one combined image to a driver,

wherein a thermal sensor and a far-infrared lamp are installed around a fog light or head light to carry out an imaging operation of the far-infrared image capturing device such that the far-infrared lamp emits far-infrared rays toward a subject area in front of the vehicle and, at the same time, the thermal sensor detects heat of an object in front of the vehicle,

wherein the far-infrared image capturing device takes the far-infrared image that is an image of a road in front of the vehicle, and the image is transmitted to the second controller, and at the same that the far-infrared image capturing device takes the image, the lane detection image capturing device takes the lane information image of the road, which is transmitted to the first controller.

7. The computer readable media of claim 6, further comprising:

program instructions that data process the at least one image from the far-infrared night vision module to create a reference coordinate image; and

program instructions that compare the at least one image from the lane detection module to the reference coordinate image and data process the at least one image from the lane detection module to correct the lane detection image to match the reference coordinate image.