METHOD FOR IMAGE RESTORATION IN A COMPUTER VISION SYSTEM

• US 20110273582A1
• Filed: 01/08/2010
• Published: 11/10/2011
• Est. Priority Date: 01/20/2009
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Gayko, Jens, Halimeh, Jad Camille, Roser, Martin Benjamin
• Granted Patent: US 8,797,417 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

A vehicle is equipped with a camera (which may be a stereoscopic camera) and a computer for processing the image data acquired by the camera. The image acquired by the camera is processed by the computer, and features are extracted therefrom. The features are further processed by various techniques such as object detection/segmentation and object tracking/classification. The acquired images are sometimes contaminated by optical occlusions such as raindrops, stone-chippings and dirt on the windshield. In such a case, the occluded parts of the image are reconstructed by optical flow estimation or stereo disparity estimation. The fully reconstructed image is then used for intended applications.

Claims

1. A method for image restoration in a computer vision system, comprising:

capturing a first image of an object that is optically occluded by a contamination;

capturing a second image of the object from a different view point; and

reconstructing an optically occluded part of the first image by using information in the second image.

2. The method according to claim 1, wherein the first and second image are captured through a substantially transparent sheet that is optically occluded by the contamination.

3. The method according to claim 2, wherein the contamination includes a member selected from a group consisting of a raindrop, a stone-chipping and a smear deposit on the transparent sheet.

4. The method according to claim 1, wherein the first image is captured by a camera placed at a first position, and the second image is captured by the same or a different camera placed at a second position which is displaced from the first position in a direction perpendicular to an optical axis of the camera at the first position.

5. The method according to claim 1, wherein the object is stationary and the first image is captured by a camera carried by a moving platform at a first time point, the second image being captured by the cameral at a second time point which is temporally displaced from the first time point.

6. The method according to claim 1, wherein the image of the object is captured as a plurality of frames corresponding to successive time points, and upon detection of an optical occlusion in any one of the frames, the optically occluded part of the image in the incident frame is reconstructed from a corresponding part of the image in a preceding frame.

7. The method according to claim 6, wherein the image of the object is captured through a substantially transparent sheet that can be optically occluded by contamination and is equipped with a wiper configured to wipe a surface of the transparent sheet to remove such contamination at a regular interval, and the reconstruction of the optically occluded part of the image is performed by using a frame from which contamination has just been removed by the wiper.