Vehicle front-view monitoring system

• US 7,619,649 B2
• Filed: 07/12/2001
• Issued: 11/17/2009
• Est. Priority Date: 07/12/2000
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Kogure, Masaru, Hiwatashi, Yutaka
• Primary Examiner(s): CZEKAJ, DAVID J
• Application Number:
  US09/902,576
• Publication Number:
  US 20020018119A1
• Time in Patent Office: 3,050 Days
• Field of Search: 348/148, 348/116, 348/118, 340/937, 382/103, 382/109, 382/106
• US Class Current: 348/148
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A vehicle front-view monitoring system determines whether there is a fail occurring on the monitoring system based on luminance data on an image of a front view taken by camera and takes fail-safe measures if the fail is occurring. The luminance data indicates luminance-distribution characteristic values indicating horizontal luminance-distribution on the image. A fail can be determined based on a parameter obtained by normalizing the luminance-distribution characteristic values by a shutter speed for the camera equipment.

Claims

1. A vehicle front-view monitoring system for taking fail-safe measures comprising:

a camera device for taking an image of a view in front;

a calculator for calculating luminance data on the image, wherein the luminance data provides a source for a luminance-distribution characteristic value indicating a horizontal luminance-distribution on the image; and

a determination section for determining whether there is a fail occurring on the image taken by the camera,

wherein the determination section determines the fail based on a parameter obtained by normalizing the luminance-distribution characteristic value by a shutter speed for the camera device, and

wherein the parameter is obtained by dividing the luminance-distribution characteristic value by the shutter speed for the camera device.

2. The vehicle front-view monitoring system according to claim 1, wherein the luminance data provides a source for the luminance-distribution characteristic value that includes the maximum value of addition of luminance on the image.

3. The vehicle front-view monitoring system according to claim 1, wherein the luminance data provides a source for the luminance-distribution characteristic value that includes a luminance-addition variance on the image.

4. The vehicle front-view monitoring system according to claim 1, wherein the luminance data include data related to luminance edges in a predetermined monitoring area on the image.

5. The vehicle front-view monitoring system according to claim 4, wherein the number of related to luminance edges data is the number of luminance edges.

6. The vehicle front-view monitoring system according to claim 4, wherein the number of data related to luminance edges is the number of distance data obtained by a pair of cameras of the camera device.

7. The vehicle front-view monitoring system according to claim 4, wherein the monitoring area is set on the upper section of the image where a vehicle running ahead is displayed.

8. The vehicle front-view monitoring system according to claim 1, wherein the calculator further calculates the number of data as another parameter related to luminance edges in a predetermined monitoring area on the image, the determination section determining the fail based on the parameters.

9. The vehicle front-view monitoring system for taking fail-safe measures according to claim 1,

wherein the calculator calculates the luminance data on the basis of luminance values of sample pixels in preset monitoring areas on the image, the preset monitoring areas are arranged on the image in a horizontal direction.

10. The vehicle front-view monitoring system for taking fail-safe measures according to claim 9,

wherein the calculator calculates sums of the luminance values in each preset monitoring areas and calculates the maximum value of the sums as the luminance-distribution characteristic value, and

wherein the determination section determines the fail based on the parameter which is obtained by normalizing the maximum value by the shutter speed.

11. The vehicle front-view monitoring system for taking fail-safe measures according to claim 9,

wherein the calculator calculates a variance value of the luminance values as the luminance-distribution characteristic value, and

wherein the determination section determines the fail based on the parameter which is obtained by normalizing the variance value by the shutter speed.

12. A vehicle front-view monitoring system for taking a fail-safe measure comprising:

a camera device for taking an image of a view in front;

a calculator for calculating luminance data on the image; and

a determination section for determining whether there is a fail occurring on the monitoring system based on the luminance data, the fail-safe measure being taken if the fail is occurring,

wherein the calculator calculates a luminance center as the luminance data, the luminance center corresponding to a horizontal position on the image at which there is a convergence of luminance, and the calculator calculates a luminance moment indicating a horizontal luminance distribution based on the luminance center and normalizes the luminance moment by dividing the luminance moment by a total of luminance values of sample pixels on the image,

the determination section determines the fail by evaluating the horizontal luminance distribution on the image based on the luminance center normalized luminance moment.

13. The monitoring system of claim 12, further comprising:

an image recognition device which receives image data from said camera device; and

a vehicle state alteration device which is in communication with said image recognition device and alters a vehicle condition based on image data criteria;

wherein said fail-safe measures prevents the vehicle state alteration device from activating an image recognition based vehicle control change in said vehicle.

14. The monitoring system of claim 13 further comprising requirement means for determining whether one or more of the fail safe interruption criteria is present, which requirement means is associated with one or more of the following criteria;

a) that an upper luminance saturation factor LR1 is larger than a lower luminance saturation factor LR2;

b) that a number of luminance edges EN of an area of the vehicle in front is smaller than a reference number;

c) that a number of the calculation of the distance data is smaller than a reference number;

d) that a luminance-distribution characteristics parameter normalized by a shutter speed VAR′

is greater than a reference value;

e) that a maximum addition of luminance normalized by a shutter speed Amax′

is larger than a reference value,

f) that a normalized luminance moment LMt is smaller than center corresponding to a horizontal position on the image which there is a convergence of luminance that is smaller than a reference value.

15. The monitoring system of claim 14 wherein the requirement means determines that a fail safe mode interruption is appropriate if one or more of criteria a to f is not satisfied.

16. The monitoring system of claim 14 wherein the requirement means determines that a fail safe mode interruption is appropriate if just one of criteria a to f is not satisfied.

17. The monitoring system of claim 14 wherein the requirement means determines that fail safe mode activation is appropriate if one or more of criteria a to f is satisfied.

18. The monitoring system of claim 14 wherein the requirement means determines that fail safe mode activation is appropriate if two or more of criteria a to fare satisfied.

19. The monitoring system of claim 14 wherein the requirement means determines that fail safe mode activation is appropriate only if all of criteria a to fare satisfied.

20. The monitoring system of claim 13 wherein said determination section prevents said fail-safe measures from activating if a vehicle is recognized ahead or a distance to a vehicle is detected based on said image recognition device.

21. The monitoring system of claim 13 wherein said determination section includes means for preventing said fail-safe measures from activating based on camera shutter speed.

22. The monitoring system of claim 12, wherein said fail-safe measures precludes activation of an image recognition based vehicle control change until conditions are sensed by said monitoring system that a luminance state is acceptable for proper monitoring system functioning.

23. A vehicle front-view monitoring system, comprising:

a camera device which is mounted on the vehicle for taking an image of a view in front of the vehicle;

an image recognition device which receives image data from said camera device;

a vehicle state alteration device which is in communication with said image recognition device and alters a vehicle condition based upon image data criteria when there is deemed not to be a monitoring system failure;

a determination section for determining whether there is a monitoring system failure based on a luminance state condition; and

a fail safe implementation device which places said monitoring system in a fail safe mode based on the determination section determining a luminance based monitoring system failure has occurred, which fail safe mode precludes an activity in said vehicle state alteration device; and

requirement means for determining whether one or more fail safe interruption criteria is present, which requirement means is associated with one or more of the following criteria;

a) that an upper luminance saturation factor LR1 is larger than a lower luminance saturation factor LR2;

b) that a number of luminance edges EN of an area of the vehicle in front is smaller than a reference number;

c) that a number of the calculation of the distance data is smaller than a reference number;

d) that a luminance-distribution characteristics parameter normalized by a shutter speed VAR′

is greater than a reference value;

e) that a maximum addition of luminance normalized by a shutter speed Amax′

is larger than a reference value; and

f) that a normalized luminance moment LM′

is smaller than center corresponding to a horizontal position on the image which luminance are converged is smaller than a reference value,

wherein said requirement means determining that fail safe mode activation is appropriate only if all of criteria a to fare satisfied,

wherein said luminance state condition is a luminance condition deemed to present a potential for a lowered monitoring accuracy condition, and wherein said fail safe implementation device precludes any activity in said vehicle state alteration device until a restatement condition, wherein monitoring accuracy is judged acceptable, is established.

24. The monitoring system of claim 23, further comprising

a calculator for calculating luminance data on the image;

wherein said determination section for determining whether there is a monitoring system failure occurring on the monitoring system based on a luminance state condition determines whether there is a monitoring system failure based on luminance calculations derived from said calculator.

25. The monitoring system of claim 23 wherein said vehicle condition to be altered is a warning system and said fail-safe implementation device prevents said warning system from activating.

26. The monitoring system of claim 23 wherein said fail safe implementation device prevents the vehicle state alteration device from activating a vehicle control change in said vehicle.

27. The monitoring system of claim 26 wherein the vehicle control change of said vehicle state alteration device, which is precluded from activating in the fail safe mode, is a vehicle slow down control change.

28. The monitoring system of claim 23, wherein said determination section for determining whether there is a monitoring system failure occurring on the monitoring system based on a luminance state condition determines whether there is a luminance state condition failure based on luminance calculations, and said fail safe implementation device prevents said vehicle state alteration device from activating an image recognition based vehicle control change in said vehicle.

29. The monitoring system of claim 23, wherein said determination section for determining whether there is a monitoring system failure occurring on the monitoring system based on a luminance state condition determines whether there is a monitoring system failure based on optical irregularities occurring on monitored images, and said fail safe implementation device prevents said vehicle state alteration device from activating an image recognition based vehicle control change in said vehicle.