Lane boundary detection using images

• US 9,081,385 B1
• Filed: 12/21/2012
• Issued: 07/14/2015
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David Ian, Senlet, Turgay
• Primary Examiner(s): Trammell, James
• Assistant Examiner(s): LANG, MICHAEL DEAN
• Application Number:
  US13/723,358
• Time in Patent Office: 935 Days
• Field of Search: 701/28
• US Class Current: 1/1
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Methods and systems for lane boundary detection using images are described. A computing device may be configured to receive, from an image-capture device coupled to a vehicle, an image of a road of travel of the vehicle. The computing device may be configured to identify a pixel in the image based on an intensity of the pixel and a comparison of the intensity of the pixel to respective intensities of neighboring pixels. Based on the intensity of the pixel and the comparison, the computing device may be configured to determine a likelihood that the pixel belongs to a portion of the image depicting a lane marker on the road. Based at least on the likelihood, the computing device may be configured to and provide instructions to control the vehicle.

Claims

1. A method comprising:

receiving, from an image-capture device coupled to a vehicle, an image of a road of travel of the vehicle;

determining that a pixel in the image meets a first condition related to whether an intensity of the pixel exceeds a threshold intensity, wherein the threshold intensity is associated with a predetermined intensity of lane markers;

determining that the pixel meets a second condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a lateral direction are substantially equal to the intensity of the pixel, wherein the lateral direction is perpendicular to a direction of travel of the vehicle on the road;

determining that the pixel meets a third condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a longitudinal direction are substantially equal to the intensity of the pixel, wherein the longitudinal direction is parallel to the direction of travel of the vehicle on the road;

determining that the pixel meets a fourth condition related to whether respective intensities of pixels beyond a respective number of pixels from the pixel in the lateral and longitudinal directions are lower than the intensity of the pixel;

determining, based on the pixel meeting the first condition, the second condition, the third condition, and the fourth condition, a likelihood that the pixel belongs to a portion of the image depicting a lane marker on the road; and

controlling, using a computing device, the vehicle based at least on the likelihood.

2. The method of claim 1, wherein determining the likelihood comprises:

determining a first likelihood based on the pixel meeting the first condition;

determining a second likelihood based on the pixel meeting the second condition;

determining a third likelihood based on the pixel meeting the third condition;

determining a fourth likelihood based on the pixel meeting the fourth condition;

determining the likelihood based on the first, second, third, and fourth likelihoods.

3. The method of claim 1, further comprising:

identifying a group of pixels based on intensities of pixels of the group of pixels and comparison of the intensities to given intensities of given pixels of neighboring groups of pixels in the image;

determining a given likelihood that the group of pixels form an edge of a lane marker; and

estimating a location of a lane boundary on the road based on the given likelihood, wherein controlling the vehicle comprises is based on the estimated location of the lane boundary.

4. The method of claim 1, wherein the intensity of the pixel is indicative of a level of brightness of the pixel in the image.

5. The method of claim 1, wherein the computing device is configured to control the vehicle in an autonomous operation mode.

6. A non-transitory computer readable medium having stored thereon executable instructions that, upon execution a computing device, cause the computing device to perform functions comprising:

receiving, from an image-capture device coupled to a vehicle, an image of a road of travel of the vehicle;

determining that a pixel in the image meets a first condition related to whether an intensity of the pixel exceeds a threshold intensity, wherein the threshold intensity is associated with a predetermined intensity of lane markers;

determining that the pixel meets a second condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a lateral direction are substantially equal to the intensity of the pixel, wherein the lateral direction is perpendicular to a direction of travel of the vehicle on the road;

determining that the pixel meets a third condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a longitudinal direction are substantially equal to the intensity of the pixel, wherein the longitudinal direction is parallel to the direction of travel of the vehicle on the road;

determining that the pixel meets a fourth condition related to whether respective intensities of pixels beyond a respective number of pixels from the pixel in the lateral and longitudinal directions are lower than the intensity of the pixel;

determining, based on the pixel meeting the first condition, the second condition, the third condition, and the fourth condition, a likelihood that the pixel belongs to a portion of the image depicting a lane marker on the road; and

controlling the vehicle based at least on the likelihood.

7. The non-transitory computer readable medium of claim 6, wherein the function of determining the likelihood comprises:

determining a first likelihood based on the pixel meeting the first condition;

determining a second likelihood based on the pixel meeting the second condition;

determining a third likelihood based on the pixel meeting the third condition;

determining a fourth likelihood based on the pixel meeting the fourth condition;

determining the likelihood based on the first, second, third, and fourth likelihoods.

8. The non-transitory computer readable medium of claim 6, wherein the intensity of the pixel is indicative of a level of brightness of the pixel in the image.

9. The non-transitory computer readable medium of claim 6, wherein the computing device is configured to control the vehicle in an autonomous operation mode.

10. A control system comprising:

at least one processor; and

a memory having stored thereon executable instructions that, upon execution by the at least one processor, cause the control system to perform functions comprising;

receiving, from an image-capture device coupled to a vehicle, an image of a road of travel of the vehicle;

determining that a pixel in the image meets a first condition related to whether an intensity of the pixel exceeds a threshold intensity, wherein the threshold intensity is associated with a predetermined intensity of lane markers;

determining that the pixel meets a second condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a lateral direction are substantially equal to the intensity of the pixel, wherein the lateral direction is perpendicular to a direction of travel of the vehicle on the road;

determining that the pixel meets a third condition related to whether respective intensities of pixels within a threshold number of pixels from the pixel in a longitudinal direction are substantially equal to the intensity of the pixel, wherein the longitudinal direction is parallel to the direction of travel of the vehicle on the road;

determining that the pixel meets a fourth condition related to whether respective intensities of pixels beyond a respective number of pixels from the pixel in the lateral and longitudinal directions are lower than the intensity of the pixel;

determining, based on the pixel meeting the first condition, the second condition, the third condition, and the fourth condition, a likelihood that the pixel belongs to a portion of the image depicting a lane marker on the road; and

controlling the vehicle based at least on the likelihood.

11. The control system of claim 10, wherein function of determining the likelihood comprises:

determining a first likelihood based on the pixel meeting the first condition;

determining a second likelihood based on the pixel meeting the second condition;

determining a third likelihood based on the pixel meeting the third condition;

determining a fourth likelihood based on the pixel meeting the fourth condition;

determining the likelihood based on the first, second, third, and fourth likelihoods.

12. The control system of claim 10, wherein the functions further comprise:

identifying a group of pixels based on intensities of pixels of the group of pixels and comparison of the intensities to given intensities of given pixels of neighboring groups of pixels in the image;

determining a given likelihood that the group of pixels form an edge of a lane marker; and

estimating a location of a lane boundary on the road based on the given likelihood, wherein controlling the vehicle is based on the estimated location of the lane boundary.

13. The control system of claim 10, wherein the intensity of the pixel is indicative of a level of brightness of the pixel in the image.

14. The control system of claim 10, wherein the control system is configured to control the vehicle in an autonomous operation mode.