Construction zone object detection using light detection and ranging

• US 8,996,228 B1
• Filed: 09/05/2012
• Issued: 03/31/2015
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David Ian, Haehnel, Dirk, Mahon, Ian
• Primary Examiner(s): Jabr, Fadey
• Assistant Examiner(s): INGRAM, THOMAS P
• Application Number:
  US13/603,618
• Time in Patent Office: 937 Days
• Field of Search: 701/23, 701/28, 701/300, 348/118, 901/47
• US Class Current: 701/28
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: B60W 60/00184 related to infrastructure
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/3461 Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types, segments such as motorways, toll roads, ferries
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/93185 : Controlling the brakes
• CPC Class Codes: G01S 2013/9319 : Controlling the accelerator
• CPC Class Codes: G01S 2013/9327 : Sensor installation details
• CPC Class Codes: G01S 2013/93271 : in the front of the vehicles
• CPC Class Codes: G01S 2013/93273 : on the top of the vehicles
• CPC Class Codes: G01S 2013/93275 : in the bumper area
• CPC Class Codes: G01S 2013/93277 : in the lights
• CPC Class Codes: G01S 7/4808 : Evaluating distance, position or velocity data
• CPC Class Codes: G05D 1/021 : specially adapted to land vehicles
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 : extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/027 : comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 : of traffic signs
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

Methods and systems for construction zone object detection are described. A computing device may be configured to receive, from a LIDAR, a 3D point cloud of a road on which a vehicle is travelling. The 3D point cloud may comprise points corresponding to light reflected from objects on the road. Also, the computing device may be configured to determine sets of points in the 3D point cloud representing an area within a threshold distance from a surface of the road. Further, the computing device may be configured to identify construction zone objects in the sets of points. Further, the computing device may be configured to determine a likelihood of existence of a construction zone, based on the identification. Based on the likelihood, the computing device may be configured to modify a control strategy of the vehicle; and control the vehicle based on the modified control strategy.

Claims

1. A method, comprising:

receiving, at a computing device configured to control a vehicle, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from one or more objects on the road;

selecting, using the computing device, a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road;

identifying one or more construction zone objects in the selected portion;

determining, using the computing device, a number and locations of the one or more construction zone objects;

determining, using the computing device, a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects;

in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects;

modifying, using the computing device, a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and

controlling, using the computing device, the vehicle based on the modified control strategy.

2. The method of claim 1, wherein the vehicle is in an autonomous operation mode.

3. The method of claim 1, wherein the one or more construction zone objects are construction cones or construction barrels.

4. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.

5. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.

6. The method of claim 1, wherein identifying the one or more construction zone objects in the selected portion comprises determining, for each identified construction zone object, a respective likelihood of the identification.

7. The method of claim 6, wherein determining the respective likelihood of the identification comprises:

identifying a shape in the selected portion; and

matching the shape to one or more shapes of standard construction zone objects.

8. The method of claim 6, wherein determining the likelihood of the existence of the construction zone comprises processing information relating to the respective likelihoods, the number, and locations of the one or more construction zone objects by a classifier trained by previously collected training data, wherein training the classifier comprises:

receiving training data for a plurality of driving situations of the vehicle, wherein the training data includes respective LIDAR-based information relating to a respective 3D point cloud of a respective road;

receiving positive or negative indication of respective existence of a respective construction zone corresponding to respective training data for each of the driving situations;

correlating, for each driving situation, the positive or negative indication with the respective training data; and

determining parameters of the classifier based on the correlations for the plurality of driving situations.

9. The method of claim 1, wherein controlling the vehicle based on the modified control strategy comprises one or more of:

(i) utilizing sensor information received from on-board or off-board sensors in making a navigation decision rather than preexisting map information, (ii) utilizing the sensor information to estimate lane boundaries rather than the preexisting map information, (iii) determining locations of construction zone markers rather than lane markers on the road to estimate and follow the lane boundaries, (iv) activating one or more sensors for detection of construction workers and making the navigation decision based on the detection, (v) following another vehicle, (vi) maintaining a predetermined safe distance with other vehicles, (vii) turning-on lights, (viii) reducing a speed of the vehicle, (ix) stopping the vehicle.

10. A non-transitory computer readable medium having stored thereon instructions executable by a computing device of a vehicle to cause the computing device to perform functions comprising:

receiving, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from objects on the road;

selecting a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road;

identifying one or more construction zone objects in the selected portion;

determining, for each identified construction zone object, a respective likelihood of the identification;

determining, based on the respective likelihoods, a number and locations of the one or more construction zone objects;

determining a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects;

in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects;

modifying a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and

controlling the vehicle based on the modified control strategy.

11. The non-transitory computer readable medium of claim 10, wherein the vehicle is in an autonomous operation mode.

12. The non-transitory computer readable medium of claim 10, wherein the one or more construction zone objects are construction cones or construction barrels.

13. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.

14. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.

15. The non-transitory computer readable medium of claim 10, wherein the function of determining the respective likelihood of the identification comprises:

identifying a shape in the selected portion; and

matching the shape to one or more shapes of standard construction zone objects.

16. A control system for a vehicle, comprising:

a light detection and ranging (LIDAR) sensor configured to provide LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR sensor and reflected from objects on the road; and

a computing device in communication with the LIDAR sensor and configured to;

receive the LIDAR-based information;

select a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road;

identify one or more construction zone objects in the selected portion;

determine, for each identified construction zone object, a respective likelihood of the identification;

determine, based on the respective likelihoods, a number and locations of the one or more construction zone objects;

determine a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects;

in response to the likelihood exceeding a threshold likelihood, determine a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects;

modify a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and

control the vehicle based on the modified control strategy.

17. The system of claim 16, wherein the computing device is further configured to control the vehicle in an autonomous operation mode.

18. The system of claim 16, wherein the one or more construction zone objects are construction cones or construction barrels.

19. The system of claim 16, wherein, to determine the likelihood of the existence of the construction zone, the computing device is configured to determine, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.