Travel lane marking recognition system

• US 9,669,829 B2
• Filed: 10/27/2015
• Issued: 06/06/2017
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Shimomura, Osamu, Suzuki, Shunsuke, Tsuruta, Tomohiko, Kawasaki, Naoki
• Primary Examiner(s): PAIGE, TYLER D
• Application Number:
  US14/924,539
• Publication Number:
  US 20160121889A1
• Time in Patent Office: 588 Days
• Field of Search: 701 41, 382195, 348148
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/05 Type of road, e.g. motorways, local streets, paved or unpaved roads
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An on-board camera captures an image of a travel lane ahead of an own vehicle. At least one state acquisition apparatus acquires an acquired value indicating a state of the own vehicle. A travel lane marking recognition system recognizes a travel lane marking that demarcates the travel lane from an image captured by the on-board camera. The travel lane marking recognition system sets a reliability level of the recognized travel lane marking based on the acquired value of the at least one state acquisition apparatus. When the reliability level is higher than a first predetermined threshold, the travel lane marking recognition system performs driving control of the own vehicle based on the recognized travel lane marking. When the reliability level is lower than the first predetermined threshold, the travel lane marking recognition system performs a deviation warning of deviation from the travel lane based on the recognized travel lane marking.

Claims

1. A travel lane marking recognition system comprising:

an on-board camera that captures an image of a travel lane ahead of an own vehicle;

at least one state acquisition apparatus other than the on-board camera that acquires an acquired value indicating a state of the own vehicle;

a recognizing unit that recognizes a travel lane marking that demarcates the travel lane from an image captured by the on-board camera;

a reliability level setting unit that sets a reliability level of the travel lane marking recognized by the recognizing unit based on the acquired value acquired by the state acquisition apparatus; and

a driving assisting unit that performs driving control of the own vehicle based on the travel lane marking recognized by the recognizing unit when the reliability level set by the reliability level setting unit is higher than a first predetermined threshold, and performs a deviation warning of deviation from the travel lane based on the travel lane marking recognized by the recognizing unit when the reliability level is lower than the first predetermined threshold, wherein

the at least one state acquisition apparatus includes a solid object detection apparatus that detects a solid object in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking in a periphery of the solid object when the solid object is detected, by the solid object detection apparatus, in a position corresponding to a periphery of the travel lane marking recognized by the recognizing unit.

2. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes a position acquisition apparatus that acquires a current position of the own vehicle on a map, and

the reliability level setting unit reduces the reliability level of the travel lane marking in a periphery of a crosswalk when the current position of the own vehicle acquired by the position acquisition apparatus is before the crosswalk on the map.

3. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an attribute acquisition apparatus that acquires an attribute of a road on which the own vehicle is traveling, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the attribute acquired by the attribute acquisition apparatus indicates a city street.

4. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an attribute acquisition apparatus that acquires an attribute of a road on which the own vehicle is traveling, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the attribute acquired by the attribute acquisition apparatus indicates a parking lot.

5. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an other vehicle detection apparatus that detects an other vehicle in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when a difference between a direction in which the travel lane marking recognized by the recognizing unit extends and a moving direction of the other vehicle detected by the other vehicle detection apparatus is greater than a predetermined angle.

6. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes a vehicle behavior sensor that detects at least one of a pitch angle of the own vehicle and a roll angle of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when at least one of when the pitch angle detected by the vehicle behavior sensor is greater than a first predetermined angle and when the roll angle detected by the vehicle behavior sensor is greater than a second predetermined angle.

7. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an environmental condition acquisition apparatus that acquires an environmental condition in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the environmental condition acquired by the environmental condition acquisition apparatus indicates being in backlighting in relation to the on-board camera.

8. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an environmental condition acquisition apparatus that acquires an environmental condition in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the environmental condition acquired by the environmental condition acquisition apparatus indicates being at night.

9. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an environmental condition acquisition apparatus that acquires an environmental condition in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the environmental condition acquired by the environmental condition acquisition apparatus indicates being during rainfall or during fog.

10. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an environmental condition acquisition apparatus that acquires an environmental condition in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the environmental condition acquired by the environmental condition acquisition apparatus indicates being on a snowy road.

11. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes an environmental condition acquisition apparatus that acquires an environmental condition in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when the environmental condition acquired by the environmental condition acquisition apparatus indicates being in a desert.

12. The travel lane marking recognition system according to claim 1, further comprising

a pitch angle calculator unit that calculates a pitch angle of the own vehicle from travel lane markings on both sides of the travel lane when travel lane markings on both sides of the travel lane are recognized by the recognizing unit,

wherein

the at least one state acquisition apparatus includes a vehicle behavior sensor that detects a pitch angle of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking when a difference between the pitch angle calculated by the pitch angle calculator unit and the pitch angle detected by vehicle behavior sensor is greater than a third predetermined angle.

13. The travel lane marking recognition system according to claim 12, wherein

the pitch angle calculator unit sets, as a calculated pitch angle, the pitch angle detected by vehicle behavior sensor when a travel lane marking on one side of the travel lane is recognized by the recognizing unit.

14. The travel lane marking recognition system according to claim 1, further comprising:

a branching road determining unit that performs a determination of whether or not a travel lane marking candidate detected from the image by the recognizing unit is a branching road; and

a branching reliability level calculator unit that calculates a reliability level of the determination performed by the branching road determining unit,

wherein

the at least one state acquisition apparatus includes a gradient detection apparatus that detects a gradient of the travel lane, and

the branching reliability level calculator unit reduces the reliability level of the determination performed by the branching road determining unit when the branching road determining unit determines that the travel lane marking candidate is a branching road and when the gradient of the travel lane detected by the gradient detection apparatus is greater than a predetermined gradient.

15. The travel lane marking recognition system according to claim 1, further comprising

a curvature calculator unit that calculates a curvature of the travel lane and a curvature change rate of the travel lane from the travel lane marking recognized by the recognizing unit,

wherein

the at least one state acquisition apparatus includes a roadside object detection apparatus that detects a roadside object in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking, i) when at least one of a) the curvature calculated by the curvature calculator unit is greater than a first predetermined value and b) the curvature change rate calculated by the curvature calculator unit is greater than a second predetermined value, and ii) when the roadside object is detected, by the roadside object detection apparatus, in a position corresponding to the periphery of the travel lane marking recognized by the recognizing unit.

16. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus is a plurality of state acquisition apparatuses,

the travel lane marking recognition system further comprises an acquired value reliability level calculator unit that calculates reliability levels of acquired values acquired by the plurality of state acquisition apparatuses, and

the reliability level setting unit reduces the reliability level of the travel lane marking when each of the reliability levels of acquired values acquired by the plurality of state acquisition apparatuses is higher than a second predetermined threshold and when states of the own vehicle indicated by the acquired values do not match with one another.

17. The travel lane marking recognition system according to claim 1, wherein

the reliability level setting unit corrects the reliability level of the travel lane marking using information detected from the image.

18. The travel lane marking recognition system according to claim 17, wherein

the reliability level setting unit reduces the reliability level of the travel lane marking when a travel lane marking candidate detected on one side of the travel lane from the image by the recognizing unit is a plurality of travel lane markings.

19. The travel lane marking recognition system according to claim 17, wherein

the reliability level setting unit reduces the reliability level of the travel lane marking when the travel lane marking recognized by the recognizing unit is a composite line.

20. The travel lane marking recognition system according to claim 17, further comprising

a curvature calculator unit that calculates a curvature of the travel lane and a curvature change rate of the travel lane from the travel lane marking recognized by the recognizing unit,

wherein

the reliability level setting unit reduces the reliability level of the travel lane marking, when at least one of i) the curvature calculated by the curvature calculator unit is greater than a third predetermined value and ii) the curvature change rate calculated by the curvature calculator unit is greater than a fourth predetermined value.

21. The travel lane marking recognition system according to claim 17, wherein

the reliability level setting unit reduces the reliability level of the travel lane marking when edge points exceeding a predetermined quantity are extracted in a periphery of the travel lane marking recognized by the recognizing unit.

22. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus acquires, as an acquired value, map information of a periphery of the own vehicle.

23. The travel lane marking recognition system according to claim 1, wherein:

the at least one state acquisition apparatus includes an inter-vehicle communication apparatus that receives, from an other vehicle, a position of the other vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking in a periphery of the other vehicle when the position of the other vehicle received by the inter-vehicle communication apparatus is a position corresponding to a periphery of the travel lane marking recognized by the recognizing unit.

24. The travel lane marking recognition system according to claim 1, wherein

the at least one state acquisition apparatus includes a road-vehicle communication apparatus that receives, from a roadside apparatus, a position of a solid object in relation to the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking in a periphery of the position of the solid object when the position of the solid object received by the inter-vehicle communication apparatus lies ahead of the own vehicle in the travel lane.

25. The travel lane marking recognition system according to claim 1, wherein

the reliability level setting unit reduces the reliability level of the travel lane marking when an acquired value indicating a state in which the white line is difficult to recognize the travel lane marking is acquired, by the at least one state acquisition apparatus unit, over a predetermined amount of time so as to amount to more than predetermined percentage.

26. A travel lane marking recognition system comprising:

an on-board camera that captures an image of a travel lane ahead of an own vehicle;

at least one state acquisition apparatus other than the on-board camera that acquires an acquired value indicating a state of the own vehicle;

a recognizing unit that recognizes a travel lane marking that demarcates the travel lane from an image captured by the on-board camera;

a reliability level setting unit that sets a reliability level of the travel lane marking recognized by the recognizing unit based on the acquired value acquired by the state acquisition apparatus; and

a display unit that displays the travel lane marking recognized by the recognizing unit such as to differentiate between a portion of which the reliability level set by the reliability level setting unit is higher than the first predetermined threshold and a portion of which the reliability level is lower than the first threshold, wherein

the at least one state acquisition apparatus includes a solid object detection apparatus that detects a solid object in a periphery of the own vehicle, and

the reliability level setting unit reduces the reliability level of the travel lane marking in a periphery of the solid object when the solid object is detected, by the solid object detection apparatus, in a position corresponding to a periphery of the travel lane marking recognized by the recognizing unit.