Image restoration method in computer vision system, including method and apparatus for identifying raindrops on a windshield

• US 8,797,417 B2
• Filed: 01/08/2010
• Issued: 08/05/2014
• Est. Priority Date: 01/20/2009
• Status: Expired due to Fees

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Gayko, Jens, Roser, Martin Benjamin, Halimeh, Jad Camille
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): Nazrul, Shahbaz
• Application Number:
  US13/144,985
• Publication Number:
  US 20110273582A1
• Time in Patent Office: 1,670 Days
• Field of Search: 348/148, 348/47
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

A vehicle is equipped with a camera (which may be a stereoscopic camera) and a computer for processing the image data acquired by the camera. The image acquired by the camera is processed by the computer, and features are extracted therefrom. The features are further processed by various techniques such as object detection/segmentation and object tracking/classification. The acquired images are sometimes contaminated by optical occlusions such as raindrops, stone-chippings and dirt on the windshield. In such a case, the occluded parts of the image are reconstructed by optical flow estimation or stereo disparity estimation. The fully reconstructed image is then used for intended applications.

Claims

1. A method for image restoration in a computer vision system, comprising:

capturing a first image of an object that is optically occluded by a contamination comprising at least one raindrop;

capturing a second image of the object from a different view point; and

reconstructing an optically occluded part of the first image by using information in the second image,

wherein the first and second images of the object are captured as a plurality of frames corresponding to successive time points, and upon detection of an optical occlusion in any one of the frames, the optically occluded part of the image in the incident frame is reconstructed from a refracted image of the at least one raindrop, and

wherein the image of the object is captured through a substantially transparent sheet that can be optically occluded by contamination and is equipped with a wiper configured to wipe a surface of the transparent sheet to remove such contamination at a regular interval, and the reconstruction of the optically occluded part of the image is performed by using a frame from which contamination has just been removed by the wiper.

2. The method according to claim 1, wherein the object is stationary and the first image is captured by a camera carried by a moving platform at a first time point, the second image being captured by the cameral at a second time point which is temporally displaced from the first time point.