Image processing apparatus for reducing effects of fog on images obtained by vehicle-mounted camera and driver support apparatus which utilizies resultant processed images

• US 20080317287A1
• Filed: 06/12/2008
• Published: 12/25/2008
• Est. Priority Date: 06/13/2007
• Status: Active Grant

• Current Assignee: DENSO Corporation, National University Corporation Hokkaido University, National University Corporation (University of Fukui)
• Original Assignee: DENSO Corporation, National University Corporation Hokkaido University
• Inventors: Haseyama, Miki
• Granted Patent: US 8,098,890 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Kalman filter processing is applied to each of successive images of a scene obscured by fog, captured by an onboard camera of a vehicle. The measurement matrix for the Kalman filter is established based on currently estimated characteristics of the fog, and intrinsic luminance values of a scene portrayed by a current image constitute the state vector for the Kalman filter. Adaptive filtering for removing the effects of fog from the images is thereby achieved, with the filtering being optimized in accordance with the degree of image deterioration caused by the fog.

Claims

1. An image processing apparatus coupled to receive successive periodically obtained data sets expressing respective input images, including fog-affected images captured under conditions of fog, said image processing apparatus operating on said input images during respective processing cycles to generate corresponding restored images having effects of fog substantially removed;

wherein;

said image processing apparatus comprises image restoration circuitry configured to convert a current pixel block of a current one of said input images to a corresponding restored pixel block, by applying Kalman filter processing to obtain predicted values of a state vector which expresses intrinsic luminance values of said current pixel block;

said Kalman filter processing is executed by applying

a measurement vector expressing measured luminance values of said current pixel block,

a measurement matrix expressing respective extents of deterioration of said measured luminance values, and

a state transition matrix expressing relationships between said measured luminance values of said current pixel block and respective luminance values of a corresponding pixel block in an estimated image, said estimated image having been derived beforehand based on applying said Kalman filter processing to a preceding one said input images;

and wherein said restored pixel block is obtained as a restored vector constituted by said predicted values of the state vector, and each of said restored images is obtained as an array of restored pixel blocks.

2. An image processing apparatus according to claim 1, comprising circuitry configured to derive a deterioration parameter value as a product of an estimated distance of an external location represented by said current pixel block and an estimated value of an extinction coefficient of the atmosphere, and wherein said measurement matrix is determined based on said deterioration parameter.

3. An image processing apparatus according to claim 1, comprising circuitry configured to perform a sequence of processing operations during each of said processing cycles, comprising:

deriving an estimated pixel block as an estimated vector by a calculation based on said measurement vector, and storing said estimated pixel block as a part of an estimated image, for use in a succeeding processing cycle;

deriving a provisionally restored pixel block as a provisional restored vector by a calculation based on multiplying said measurement vector by an inverse of said measurement matrix; and

deriving said state transition matrix by comparing said provisionally restored pixel block with a corresponding motion-compensated pixel block, said corresponding motion-compensated pixel block being extracted from an estimated image which was derived and stored in a preceding processing cycle.

4. An image processing apparatus according to claim 3, wherein said image processing apparatus comprises circuitry configured to judge a condition of luminance variations within said current input image and to calculate respective values of a first deterioration parameter and a second deterioration parameter based on results of said judgement, and wherein:

said first deterioration parameter expresses a dispersed-light component of each of said measured values of luminance in said current image, where said dispersed-light component results from dispersion of light by said fog;

a specific values of said second deterioration parameter is calculated corresponding to said current pixel block, said specific value expressing an estimated deterioration factor of said measured luminance values of said corresponding pixel block;

said measurement matrix is established based on said second deterioration parameter; and

in processing said current pixel block, said provisionally restored vector is derived by operating on each measured luminance value of said current pixel block by a calculation based upon said first deterioration parameter and said corresponding value of the second deterioration parameter.

5. An image processing apparatus according to claim 4, wherein said second deterioration parameter is obtained as the product of an estimated distance of an external location portrayed by said pixel block and an estimated value of an extinction coefficient of the atmosphere, and wherein said measurement matrix is a scalar matrix comprising a scalar value that is an inverse exponential function of said second deterioration parameter.

6. An image processing apparatus according to claim 5, wherein said estimated distance is calculated based on a position of said pixel block within said current input image and wherein said estimated value of the extinction coefficient of the atmosphere is calculated based upon said luminance variations within said current input image.

7. An image processing apparatus according to claim 3, wherein:

said state transition matrix comprises a plurality of rows each corresponding to a specific pixel in said current pixel block and a plurality of columns each corresponding to a specific pixel of said corresponding motion-compensated pixel block; and

each of said rows comprises a single 1 digit with remaining elements of said row being 0 digits, and with respective positions of said 1 digits within said rows being indicative of a relationship between luminance values of pixels in said current pixel block and luminance values of pixels in said motion-compensated corresponding pixel block.

8. A driver support apparatus comprising:

an image processing apparatus as claimed in claim 1;

a camera apparatus installed in a vehicle for capturing said input images as successive images of a region external to said vehicle and for supplying data expressing said input images to said image processing apparatus; and

a display apparatus coupled to receive data expressing said restored images from said image processing apparatus, for generating a visible display of said restored images.

9. A driver support apparatus according to claim 8, comprising:

target object detection circuitry configured to detect objects located external to said vehicle, based on said restored images, and

control circuitry configured to control equipment of said vehicle in accordance with results obtained by said target object detection circuitry, for providing support to a driver of said vehicle.