Malfunction diagnosis apparatus

• US 10,635,910 B2
• Filed: 07/05/2016
• Issued: 04/28/2020
• Est. Priority Date: 07/06/2015
• Status: Active Grant

• Current Assignee: Transtron Inc. (Fujitsu Limited)
• Original Assignee: Transtron Inc. (Fujitsu Limited)
• Inventors: Hishinuma, Yoshiaki, Urushizaki, Naoyuki, Sugita, Hirokazu, Kobayashi, Kazuhiko
• Primary Examiner(s): Anderson, II, James M
• Application Number:
  US15/202,527
• Publication Number:
  US 20170011271A1
• Time in Patent Office: 1,393 Days
• Field of Search: None
• CPC Class Codes: B60W 2050/021 Means for detecting failure or malfunction
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 40/00 Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 5/20 Circuitry for controlling amplitude response

Abstract

This malfunction diagnosis apparatus detects a line on a road surface by distinguishing the line from an asphalt surface other than the line. The malfunction diagnosis apparatus sets a first area corresponding to the line and a second area corresponding to the asphalt surface using a first image captured while the line is detected. The malfunction diagnosis apparatus calculates a first brightness in the first area and a second brightness in the second area using a second image captured while the vehicle is cruising and the line is not detected. Further, the malfunction diagnosis apparatus diagnoses the malfunction based on at least one of the first brightness and the second brightness.

Claims

1. A malfunction diagnosis apparatus diagnosing a malfunction of a system for processing an image captured by an on-board camera, and including a processor configured to cause performance of operations, the operations comprising:

detecting, in a first image captured by a vehicle and using a line recognition algorithm, a plurality of lines printed on a road surface;

detecting, in the first image, a portion of the road surface with no line printed thereon;

setting, in response to the plurality of lines being detected in the first image in which the plurality of lines are detected, a plurality of first areas that each include a corresponding line of the plurality of detected lines, the plurality of first areas being arranged in parallel with a gap therebetween in a horizontal direction;

setting a second area within the gap that corresponds to the detected portion with no line printed thereon;

transmitting information about positions of the second area and the plurality of first areas; and

in response to the plurality of lines being detected in the first image and in response to one or more of the plurality of lines not being detected in a second image that is captured by the vehicle after the first image is captured and that is captured while the vehicle is cruising;

calculating, using the information about the positions of the second area and the plurality of first areas, a first brightness in a plurality of first areas of the second image having positions corresponding to the positions of the plurality of first areas of the first image by calculating an average value of brightness of each pixel included in the plurality of the first areas of the second image, and a second brightness in a second area of the second image having a position corresponding to the position of the second area of the first image by calculating an average value of brightness of each pixel included in the second area of the second image; and

diagnosing the malfunction using the first brightness and the second brightness.

2. The malfunction diagnosis apparatus according to claim 1, wherein the operations further comprise:

diagnosing that the malfunction has occurred if the first brightness is less than a first threshold or the second brightness is more than a second threshold, the second threshold being set larger than the first threshold.

3. The malfunction diagnosis apparatus according to claim 2, wherein the first threshold is set using a minimum brightness of the second area in the first image.

4. The malfunction diagnosis apparatus according to claim 2, wherein the second threshold is set using a maximum brightness of one or more of the plurality of first areas in the first image.

5. The malfunction diagnosis apparatus according to claim 1, wherein, the operations further comprise:

storing information representing whether or not the line was detected in immediately close past;

in response to the plurality of lines printed on the road being detected in immediately close past with reference to the information representing whether or not the line was detected in immediately close pas and in response to the plurality of lines not being detected in the second image,

calculating, using the information about positions of the second area and the plurality of first areas, the first brightness and the second brightness;

diagnosing the malfunction using the first brightness and the second brightness; and

in response to the plurality of lines printed on the road not being detected in immediately close past;

calculating an average brightness of a third area using the second image, the third area being set to include both the plurality of first areas and the second area, and

diagnosing the malfunction using the average brightness.

6. The malfunction diagnosis apparatus according to claim 5, wherein, the operations further comprise:

diagnosing that the malfunction has occurred if the average brightness is less than a third threshold or the average brightness is more than a fourth threshold, the fourth threshold being set larger than the third threshold.

7. The malfunction diagnosis apparatus according to claim 6, wherein the third threshold and the fourth threshold are set, using the first image immediately before, using a maximum brightness of one or more of the plurality of first areas, a minimum brightness of the second area, and the average brightness of the third area.

8. The malfunction diagnosis apparatus according to claim 1, wherein:

an implementation condition of the malfunction diagnosis is that a vehicle speed is equal to or more than a predetermined speed.

9. The malfunction diagnosis apparatus according to claim 1, wherein:

an implementation condition of the malfunction diagnosis is that a magnitude of a steering angle or a yaw rate of the vehicle is equal to or more than a predetermined value.

10. The malfunction diagnosis apparatus according to claim 1, wherein the operations further comprise:

determining whether one or more lines of the plurality of lines have been detected or not previously.

11. A non-transitory computer-readable recording medium storing a malfunction diagnosis program for diagnosing a malfunction of a system for processing an image captured by an on-board camera, the program causing a computer to execute operations comprising:

detecting, in a first image captured by a vehicle and using a line recognition algorithm, a plurality of lines printed on a road surface;

detecting, in the first image, a portion of the road surface with no line printed thereon;

setting, in response to the plurality of lines being detected in the first image in which the plurality of lines are detected, a plurality of first areas that each include a corresponding line of the plurality of detected lines, the plurality of first areas being arranged in parallel with a gap therebetween in a horizontal direction;

setting a second area within the gap that corresponds to the detected portion with no line printed thereon;

transmitting information about positions of the second area and the plurality of first areas;

in response to the plurality of lines being detected in the first image and in response to one or more of the plurality of lines not being detected in a second image that is captured by the vehicle after the first image is captured and that is captured while the vehicle is cruising;

calculating, using the information about the positions of the second area and the plurality of first areas, a first brightness in a plurality of first areas of the second image having positions corresponding to the positions of the plurality of first areas of the first image by calculating an average value of brightness of each pixel included in the plurality of first areas of the second image, and a second brightness in a second area of the second image having a position corresponding to the position of the second area of the first image by calculating an average value of brightness of each pixel included in the second area of the second image; and

diagnosing the malfunction using at least one of the first brightness and the second brightness.