Construction zone detection using a plurality of information sources

• US 9,221,461 B2
• Filed: 09/05/2012
• Issued: 12/29/2015
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David Ian, Haehnel, Dirk, Fairfield, Nathaniel
• Primary Examiner(s): Beaulieu, Yonel
• Assistant Examiner(s): KLEINMAN, LAIL A
• Application Number:
  US13/603,613
• Publication Number:
  US 20140067187A1
• Time in Patent Office: 1,210 Days
• Field of Search: 701/23, 701/26, 701/28, 701/301
• US Class Current: 1/1
• CPC Class Codes: B60W 2050/0091 of control inputs
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B60W 60/0015 specially adapted for safety

Abstract

Methods and systems for detection of a construction zone using information from a plurality of sources are described. In an example, a computing device, configured to control the vehicle, may be configured to receive information, from a plurality of sources, relating to detection of a construction zone on the road on which the vehicle is travelling. Also, the computing device may be configured to determine a likelihood of existence of the construction zone on the road, based on the information. Further the computing device may be configured to modify a control strategy associated with a driving behavior of the vehicle, based on the likelihood; and control the vehicle based on the modified control strategy.

Claims

1. A method, comprising:

receiving map information related to a prior map of a road of travel of a vehicle, wherein the map information includes road traffic sign information that indicates locations and types of traffic signs that existed on the road;

detecting, by a computing device using one or more sensors coupled to the vehicle, a traffic sign, and determining that the traffic sign is missing from the map information;

in response to at least determining that the detected traffic sign is missing from the map information determining, using the computing device, a likelihood of existence of a construction zone on the road;

in response to the likelihood exceeding a threshold likelihood, detecting one or more construction zone objects;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

providing, by the computing device, instructions to control the vehicle based on the likelihood and the severity of road changes.

2. The method of claim 1, further comprising:

based on detecting the traffic sign, receiving, from a plurality of sources of information, information relating to detection of the construction zone, wherein determining the likelihood of existence of the construction zone is further based on the information.

3. The method of claim 2, wherein each source of information of the plurality of sources of information is assigned a respective reliability metric indicative of a level of confidence of the detection of the construction zone based on previously collected data from a plurality of driving situations involving other construction zones, and wherein determining the likelihood of existence of the construction zone is further based on the respective reliability metrics of the plurality of sources of information.

4. The method of claim 2, wherein the information received from the plurality of sources of information includes one or more of:

(i) image-based information;

(ii) LIDAR-based information;

(iii) RADAR-based information; and

(iv) traffic information.

5. The method of claim 4, wherein the image-based information is received from one or more image-capture devices coupled to the computing device, and wherein the image-based information is indicative of (i) location of one or more static objects with respect to the road, and (ii) road geometry.

6. The method of claim 4, wherein the traffic information is indicative of behavior of other vehicles on the road and includes respective information received from one or more of:

(i) Global Positioning Satellite (GPS) devices in the other vehicles on the road, (ii) vehicle-to-infrastructure communication, (iii) vehicle-to-vehicle communication, (iv) traffic report broadcast, and (v) on-board or off-board sensors in communication with the computing device.

7. The method of claim 4, wherein the information includes the traffic information, and wherein determining the likelihood comprises determining a change in nominal speed and flow of traffic of other vehicles on the road based on the traffic information.

8. The method of claim 1, wherein the one or more sensors coupled to the vehicle include an image-capture device and a light detection and ranging (LIDAR) sensor, the method further comprising:

receiving, from the image-capture device, an image depicting the traffic sign missing from the map information;

receiving, from the LIDAR, LIDAR-based information comprising a three-dimensional (3D) point cloud corresponding to the image depicting the traffic sign, wherein the 3D point cloud comprises a set of points based on light emitted from the LIDAR sensor and reflected from a surface of the traffic sign;

determining, based on the image, a first likelihood that traffic sign is a candidate construction zone sign; and

determining a second likelihood that the traffic sign is a candidate construction zone sign, based on the LIDAR-based information, wherein determining the likelihood of existence of the construction zone on the road is based on the first likelihood and the second likelihood.

9. A non-transitory computer readable medium having stored thereon instructions that, when executed by a computing device, cause the computing device to perform functions comprising:

receiving map information related to a prior map of a road of travel of a vehicle, wherein the map information includes road traffic sign information that indicates locations and types of traffic signs that existed on the road;

detecting, using one or more sensors coupled to the vehicle, a traffic sign, and determining that the traffic sign is missing from the map information;

in response to at least determining that the detected traffic sign is missing from the map information determining a likelihood of existence of a construction zone on the road;

in response to the likelihood exceeding a threshold likelihood, detecting one or more construction zone objects;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

controlling the vehicle based on the likelihood and the severity of road changes.

10. The non-transitory computer readable medium of claim 9, wherein the functions further comprise:

based on detecting the traffic sign, receiving, from a plurality of sources of information, information relating to detection of the construction zone, wherein the function of determining the likelihood of existence of the construction zone is further based on the information.

11. The non-transitory computer readable medium of claim 10, wherein each source of information of the plurality of sources of information is assigned a respective reliability metric indicative of a level of confidence of the detection of the construction zone based on previously collected data from a plurality of driving situations involving other construction zones, and wherein the function of determining the likelihood of existence of the construction zone is further based on the respective reliability metrics of the plurality of sources of information.

12. A control system for a vehicle, comprising:

one or more sensors coupled to the vehicle;

one or more processors in communication with the one or more sensors; and

memory configured to store instructions, that when executed by the one or more processors, cause the control system to perform functions comprising;

receiving map information related to a prior map of a road of travel of a vehicle, wherein the map information includes road traffic sign information that indicates locations and types of traffic signs that existed on the road;

detecting, using one or more sensors coupled to the vehicle, a traffic sign, and determining that the traffic sign is missing from the map information;

in response to at least determining that the detected traffic sign is missing from the map information, determining a likelihood of existence of a construction zone on the road;

in response to the likelihood exceeding a threshold likelihood, detecting one or more construction zone objects;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

controlling the vehicle based on the likelihood and the severity of road changes.

13. The control system of claim 12, wherein the one or more sensors coupled to the vehicle comprise an image-capture device and a light detection and ranging (LIDAR) sensor, and wherein the functions further comprise:

receiving, from the image-capture device, an image depicting the traffic sign missing from the map information;

receiving, from the LIDAR, LIDAR-based information comprising a three-dimensional (3D) point cloud corresponding to the image depicting the traffic sign, wherein the 3D point cloud comprises a set of points based on light emitted from the LIDAR sensor and reflected from a surface of the traffic sign;

determining, based on the image, a first likelihood that the traffic sign is a candidate construction zone sign; and

determining a second likelihood that the traffic sign is a candidate construction zone sign, based on the LIDAR-based information, wherein the function of determining the likelihood of existence of the construction zone on the road is based on the first likelihood and the second likelihood.

14. The control system of claim 12, wherein the functions further comprise:

based on detecting the traffic sign, receiving, from a plurality of sources of information, information relating to detection of the construction zone, wherein;

each source of information of the plurality of sources of information is assigned a respective reliability metric indicative of a level of confidence of the detection of the construction zone based on previously collected data from a plurality of driving situations involving other construction zones, and

the function of determining the likelihood of existence of the construction zone is further based on the information and respective reliability metrics of the plurality of sources of information.

15. The control system of claim 12, wherein the control system is configured to control the vehicle in an autonomous mode of operation.