Construction zone sign detection using light detection and ranging

• US 9,193,355 B2
• Filed: 05/21/2015
• Issued: 11/24/2015
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David Ian, Haehnel, Dirk
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): INGRAM, THOMAS P
• Application Number:
  US14/718,237
• Publication Number:
  US 20150266471A1
• Time in Patent Office: 187 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/408 Traffic behavior, e.g. swarm
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 60/00184 related to infrastructure
• CPC Class Codes: G01S 17/02 Systems using the reflection of electromagnetic waves other than radio waves G01S17/66 takes precedence
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 : of land vehicles
• CPC Class Codes: G01S 7/4802 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 : extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/027 : comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G06V 20/10 : Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/09623 : Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Methods and systems for construction zone sign detection are described. A computing device may be configured to receive a 3D point cloud of a vicinity of a road on which a vehicle is travelling. The 3D point cloud may include points corresponding to light reflected from objects in the vicinity of the road. The computing device may be configured to determine a set of points representing an area at a given height from a surface of the road, and estimate a shape associated with the set of points. Further, the computing device may be configured to determine a likelihood that the set of points represents a construction zone sign, based on the estimated shape. Based on the likelihood, the computing device may be configured to modify a control strategy associated with a driving behavior of the vehicle; and control the vehicle based on the modified control strategy.

Claims

1. A method, comprising:

selecting, by a computing device of a vehicle, a portion of a three-dimensional (3D) point cloud representing an area at a height greater than a threshold height from a surface of a road of travel of the vehicle;

identifying a shape in the selected portion;

determining, using the computing device, that the identified shape represents a construction zone sign based at least on an outline of the shape;

determining a type of the construction zone sign based at least on the identified shape;

detecting one or more construction zone objects on the road;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

controlling, using the computing device, the vehicle based on the type of the construction zone sign and the severity of road changes.

2. The method of claim 1, wherein the vehicle is in an autonomous operation mode.

3. The method of claim 1, wherein the predetermined threshold height is related to a given height of a typical construction zone sign.

4. The method of claim 1, further comprising:

receiving, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information comprising (i) the 3D point cloud, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from one or more objects in a vicinity of the road, and (ii) intensity values of the reflected light for the points.

5. The method of claim 4, wherein determining that the identified shape represents a construction zone sign comprises:

comparing the respective intensity values to a threshold intensity value, wherein the threshold intensity value is related to intensity of light reflected from a reflective surface of a typical construction zone sign.

6. The method of claim 1, wherein determining that the identified shape represents a construction zone sign comprises:

matching the identified shape to one or more shapes of typical construction zone signs.

7. The method of claim 1, wherein the computing device has access to map information that includes road traffic sign information and indicates locations and types of traffic signs on the road, the method further comprising:

determining that the identified shape is associated with a traffic sign; and

determining that the traffic sign is missing from the map information, wherein determining that the identified shape represents a construction zone sign is further based on the traffic sign being missing from the map information.

8. The method of claim 1, wherein controlling the vehicle comprises:

utilizing sensor information received from on-board or off-board sensors in making a navigation decision rather than preexisting map information.

9. A non-transitory computer readable medium having stored thereon instructions that, when executed by a computing device of a vehicle, cause the computing device to perform operations comprising:

selecting a portion of a three-dimensional (3D) point cloud representing an area at a height greater than a threshold height from a surface of a road of travel of the vehicle;

identifying a shape in the selected portion;

determining that the identified shape represents a construction zone sign based at least on an outline of the shape;

determining a type of the construction zone sign based at least on the identified shape;

detecting one or more construction zone objects on the road;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

controlling the vehicle based on the type of the construction zone sign and the severity of road changes.

10. The non-transitory computer readable medium of claim 9, wherein the operation of controlling the vehicle comprises:

controlling the vehicle in an autonomous operation mode.

11. The non-transitory computer readable medium of claim 9, wherein the predetermined threshold height is related to a given height of a typical construction zone sign.

12. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

receiving, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information comprising (i) the 3D point cloud, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from one or more objects in a vicinity of the road, and (ii) intensity values of the reflected light for the points.

13. The non-transitory computer readable medium of claim 12, wherein the operation of determining that the identified shape represents a construction zone sign comprises:

comparing the respective intensity values to a threshold intensity value, wherein the threshold intensity value is related to intensity of light reflected from a reflective surface of a typical construction zone sign.

14. The non-transitory computer readable medium of claim 9, wherein the computing device has access to map information that includes road traffic sign information and indicates locations and types of traffic signs on the road, and wherein the operations further comprise:

determining that the identified shape is associated with a traffic sign; and

determining that the traffic sign is missing from the map information, wherein the operation of determining that the identified shape represents a construction zone sign is further based on the traffic sign being missing from the map information.

15. The non-transitory computer readable medium of claim 9, wherein the operation of controlling the vehicle comprises:

utilizing sensor information received from on-board or off-board sensors to estimate lane boundaries rather than utilizing preexisting map information.

16. A control system for a vehicle, comprising:

a light detection and ranging (LIDAR) device configured to capture a three-dimensional (3D) point cloud of a vicinity of a road of travel of the vehicle;

a computing device in communication with the LIDAR device; and

data storage comprising instructions that, when executed by the computing device, cause the control system to perform operations comprising;

selecting a portion of the 3D point cloud representing an area at a height greater than a threshold height from a surface of the road of travel of the vehicle;

identifying a shape in the selected portion;

determining that the identified shape represents a construction zone sign based at least on an outline of the shape;

determining a type of the construction zone sign based at least on the identified shape;

detecting one or more construction zone objects on the road;

determining a severity of road changes based on a number and locations of the one or more construction zone objects; and

controlling the vehicle based on the type of the construction zone sign and the severity of road changes.

17. The control system of claim 16, wherein the operation of controlling the vehicle comprises:

controlling the vehicle in an autonomous operation mode.

18. The control system of claim 16, wherein the predetermined threshold height is related to a given height of a typical construction zone sign.

19. The control system of claim 16, wherein the computing device has access to map information that includes road traffic sign information and indicates locations and types of traffic signs on the road, and wherein the operations further comprise:

determining that the identified shape is associated with a traffic sign; and

determining that the traffic sign is missing from the map information, wherein the operation of determining that the identified shape represents a construction zone sign is further based on the traffic sign being missing from the map information.

20. The control system of claim 16, wherein the operation of controlling the vehicle comprises:

activating one or more sensors for detection of construction workers and making the navigation decision based on the detection.