Top-down refinement in lane marking navigation

• US 10,317,231 B2
• Filed: 06/10/2015
• Issued: 06/11/2019
• Est. Priority Date: 06/10/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Ferencz, Andras, Reisman, Pini
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/736,057
• Publication Number:
  US 20150354976A1
• Time in Patent Office: 1,462 Days
• Field of Search: None
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, top-down refinement in lane marking navigation is provided. The system may include one or more memories storing instructions and one or more processors configured to execute the instructions to cause the system to receive from one or more cameras one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line, update a model of the lane marking based on odometry of the one or more cameras relative to the roadway, refine the updated model of the lane marking based on an appearance of dashes derived from the received one or more images and a spacing between dashes derived from the received one or more images, and cause one or more navigational responses in the vehicle based on the refinement of the updated model.

Claims

1. A computer system comprising:

one or more memories storing instructions; and

one or more processors configured to execute the instructions to cause the system to;

receive, from one or more cameras, one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

obtain a multi-frame model of the lane marking, wherein the multi-frame model of the lane marking includes a location for the lane marking at a first time;

update the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identify a plurality of candidate dashes from the received one or more images;

calculate spacing information relating to spaces between the plurality of candidate dashes;

perform a top-down modeling process to refine the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

cause one or more navigational responses in the vehicle based on the refinement of the updated multi-frame model.

2. The computer system of claim 1, wherein the refinement of the updated multi-frame model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.

3. The computer system of claim 1, wherein the refinement of the updated multi-frame model is further based on existence of a parallel lane marking on one side of dashes derived from the received one or more images.

4. The computer system of claim 1, wherein the appearance of the dashes is based on a width of the dashes.

5. The computer system of claim 1, wherein the appearance of the dashes is based on a brightness of the dashes.

6. The computer system of claim 1, wherein the appearance of the dashes is based on an edge sharpness of the dashes.

7. The computer system of claim 1, wherein refining the updated multi-frame model of the lane marking based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information comprises:

refining the updated multi-frame model using a dynamic programming technique.

8. The computer system of claim 1, wherein the one or more processors are configured to execute the instructions to cause the system to:

perform a bottom-up modeling process to generate a single-frame model of the lane marking based on a single frame of the received one or more images; and

match the generated single-frame model to the refined multi-frame model to approve the lane marking.

9. The computer system of claim 1, wherein the one or more one navigational responses comprises a steering of the vehicle.

10. The computer system of claim 1, wherein the one or more one navigational responses comprises an audible alert.

11. A vehicle, comprising:

a body;

one or more cameras configured to acquire one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

one or more memories storing instructions; and

one or more processors configured to execute the instructions to cause the vehicle to;

receive, from one or more cameras, the one or more acquired images of the roadway;

obtain a multi-frame model of the lane marking, wherein the multi-frame model of the lane marking includes a location for the lane marking at a first time;

update the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identify a plurality of candidate dashes from the received one or more images;

calculate spacing information relating to spaces between the plurality of candidate dashes;

perform a top-down modeling process to refine the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

cause one or more navigational responses in the vehicle based on the refinement of the multi-frame updated model.

12. The vehicle of claim 11, wherein the refinement of the multi-frame updated model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.

13. The vehicle of claim 11, wherein the refinement of the multi-frame updated model is further based on existence of a parallel lane marking on one side of dashes derived from the received one or more images.

14. The vehicle of claim 11, wherein the appearance of the dashes is based on a width of the dashes.

15. The vehicle of claim 11, wherein the appearance of the dashes is based on a brightness of the dashes.

16. The vehicle of claim 11, wherein the appearance of the dashes is based on an edge sharpness of the dashes.

17. The vehicle of claim 11, wherein refining the updated multi-frame model of the lane marking based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information comprises:

refining the updated multi-frame model using a dynamic programming technique.

18. The vehicle of claim 11, wherein the one or more processors are configured to execute the instructions to cause the vehicle to:

generate a model of the lane marking based on the received one or more images; and

match the generated model to the refined model to approve the lane marking.

19. The vehicle of claim 11, wherein the one or more one navigational responses comprises a steering of the vehicle.

20. The vehicle of claim 11, wherein the one or more one navigational responses comprises an audible alert.

21. A method comprising:

receiving, by one or more processors, from one or more cameras, one or more images of a roadway in a vicinity of a vehicle, the roadway comprising a lane marking comprising a dashed line;

obtaining a multi-frame model of the lane marking, wherein the model of the lane marking includes a location for the lane marking at a first time;

updating, by the one or more processors, the multi-frame model of the lane marking based on odometry of the one or more cameras relative to the roadway to obtain an updated multi-frame model of the lane marking, wherein the updated multi-frame model of the lane marking includes an estimated updated location for the lane marking at a second time;

identifying a plurality of candidate dashes from the received one or more images;

calculating spacing information relating to spaces between the plurality of candidate dashes;

performing a top-down modeling process to refine, by the one or more processors, the updated multi-frame model of the lane marking at the updated location at the second time based on an appearance of dashes derived from the received one or more images and based on the calculated spacing information; and

causing, by the one or more processors, one or more navigational responses in the vehicle based on the refinement of the updated multi-frame model.

22. The method of claim 21, wherein the refinement of the updated multi-frame model is further based on a lateral position, relative to the roadway, of dashes derived from the received one or more images.