OUTSIDE RECOGNITION SYSTEM, VEHICLE AND CAMERA DIRTINESS DETECTION METHOD

• US 20160364620A1
• Filed: 01/30/2015
• Published: 12/15/2016
• Est. Priority Date: 02/18/2014
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: AKIYAMA, Yasuhiro, NAKAMURA, Katsuyuki, IRIE, Kota
• Granted Patent: US 10,268,902 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An outside recognition system configured to include a camera installed on a vehicle, to analyze an image from the camera, and to recognize surroundings of the vehicle based on an analysis result of the image from the camera comprises a dirtiness detection unit which determines, as at least one reference line, at least one line on a part of at least one object that does not change in the image from the camera even if the vehicle travels, determines an observation line that corresponds to each of the at least one reference line with the image from the camera, calculates an amount by which the observation line is displaced from each of the at least one reference line, and detects whether dirt adheres to the camera based on the amount of displacement.

Claims

1. An outside recognition system configured to include a camera installed on a vehicle, analyze an image from the camera, and recognize surroundings of the vehicle based on an analysis result of the image from the camera comprising:

a dirtiness detection unit configured to detect whether dirt adheres to the camera,

wherein the dirtiness detection unit determines, as at least one reference line, at least one line on a part of at least one object that does not change in the image from the camera even if the vehicle travels,

wherein the dirtiness detection unit determines an observation line that corresponds to each of the at least one reference line with the image from the camera,

wherein the dirtiness detection unit calculates an amount by which the observation line is displaced from each of the at least one reference line, and

wherein the dirtiness detection unit detects whether dirt adheres to the camera based on the amount of displacement.

2. The outside recognition system according to claim 1, wherein the dirtiness detection unit calculates the amount of displacement by calculating an area of a region between one of the at least one reference line and an observation line which corresponds to the one of the at least one reference line.

3. The outside recognition system according to claim 2,

wherein the at least one reference line includes a first reference line on a part of a body of the vehicle and a second reference line on a part of a bumper of the vehicle,

wherein the dirtiness detection unit calculates a first observation line corresponding to the first reference line with the image from the camera,

wherein the dirtiness detection unit calculates a second observation line corresponding to the second reference line with the image from the camera,

wherein the dirtiness detection unit calculates a first area between the first observation line and the first reference line,

wherein the dirtiness detection unit calculates a second area between the second observation line and the second reference line, and

wherein the dirtiness detection unit detects whether dirt adheres to the camera based on a magnitude relationship between the first area and a predetermined first threshold and a magnitude relationship between the second area and a predetermined second threshold, or a magnitude relationship between a sum of the first area and the second area and a predetermined third threshold.

4. The outside recognition system according to claim 1,

wherein the dirtiness detection unit determines the first reference line and the second reference line as the at least one reference line,

wherein the dirtiness detection unit calculates a first observation line corresponding to the first reference line with the image from the camera,

wherein the dirtiness detection unit calculates a second observation line corresponding to the second reference line with the image from the camera, and

wherein the dirtiness detection unit calculates, by calculating an area of a region between the first observation line and the second observation line, an amount of displacement of the first observation line from the first reference line and an amount of displacement of the second observation line from the second reference line.

5. The outside recognition system according to claim 4,

wherein the dirtiness detection unit includes:

an observation point setting unit configured to determine a plurality of observation points on each of the first reference line and the second reference line; and

a displacement observation unit configured to observe a plurality of displacement points corresponding to the plurality of observation points in the image from the camera, and

wherein the displacement observation unit unifies, in a case where direction of displacement of the displacement points corresponding to the plurality of observation points on the first reference line includes a non-uniform direction and direction of displacement of the displacement points corresponding to the plurality of observation points on the second reference line includes a non-uniform direction, all of the directions of displacement for an area between the first observation line and the second observation line to be larger or an area between the first observation line and the second observation line to be smaller, and corrects the displacement points corresponding to the plurality of observation points on the first reference line and the displacement points corresponding to the plurality of observation points on the second reference line.

6. The outside recognition system according to claim 4,

wherein the first reference line is a line on a part of a body of the vehicle,

wherein the second reference line is a line on a part of a bumper of the vehicle, and

wherein the dirtiness detection unit, based on a magnitude relationship between an area of a region between the first observation line and the second observation line and a predetermined fourth threshold, detects whether dirt adheres to the camera.

7. The outside recognition system according to claim 1,

wherein the dirtiness detection unit includes:

an observation point setting unit configured to determines a plurality of observation points on the at least one reference line; and

a displacement observation unit configured to observe the plurality of displacement points corresponding to the plurality of observation points in the image from the camera, and

wherein the displacement observation unit calculates the observation line by interpolation with the plurality of observed displacement points.

8. The outside recognition system according to claim 7,

wherein the displacement observation unit acquires a luminance value of a plurality of pixels of the observation point whose direction is orthogonal to the reference line, and

wherein the displacement observation unit calculates, as the displacement point corresponding to the observation line, a pixel whose gradient of luminance value to an adjacent pixel is largest.

9. The outside recognition system according to claim 1 wherein the dirtiness detection unit determines the at least one reference line based on design information on the vehicle, installation position information on the camera and parameter information on the camera.

10. The outside recognition system according to claim 1,

wherein the dirtiness detection unit suspends, in a case it is determined that dirt adheres to the camera, an operation to recognize surroundings of the vehicle based on analysis result of the image from the camera, and

wherein the dirtiness detection unit notifies a driver of the vehicle of detection of dirt which adheres to the camera.

11. A vehicle including an outside recognition system according to claim 1.

12. A dirtiness detection method of a camera for an outside recognition system configured to include a camera installed in a vehicle, analyze an image from the camera, and recognize surroundings of the vehicle based on an analysis result of the image from the camera, the method comprising:

determining, by outside recognition system, as at least one reference line, at least one line on a part of at least one object that does not change in the image from the camera even if the vehicle travels,

determining, by outside recognition system, an observation line that corresponds to each of the at least one reference line with the image from the camera,

calculating, by outside recognition system, an amount by which the observation line is displaced from the each of the at least one reference line, and

detecting, by outside recognition system, whether dirt adheres to the camera on the basis of the amount of displacement.