Construction zone sign detection

• US RE48,322 E1
• Filed: 11/07/2018
• Issued: 11/24/2020
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Fairfield, Nathaniel, Ferguson, David I., Ogale, Abhijit S., Wang, Matthew, Yee, Yangli Hector
• Primary Examiner(s): Tarae, Catherine M
• Application Number:
  US16/183,534
• Time in Patent Office: 748 Days
• CPC Class Codes: N/A

Abstract

Methods and systems for detection of a construction zone sign are described. A computing device, configured to control the vehicle, may be configured to receive, from an image-capture device coupled to the computing device, images of a vicinity of the road on which the vehicle is travelling. Also, the computing device may be configured to determine image portions in the images that may depict sides of the road at a predetermined height range. Further, the computing device may be configured to detect a construction zone sign in the image portions, and determine a type of the construction zone sign. Accordingly, the computing device may be configured to modify a control strategy associated with a driving behavior of the vehicle; and control the vehicle based on the modified control strategy.

Claims

1. A method comprising:

receiving, at a computing device configured to control a vehicle, from an image-capture device coupled to the vehicle, an image of a vicinity of a road on which the vehicle is travelling;

selecting, from the image, using the computing device, an image portion corresponding to a side of the road at a predetermined height;

detecting, using the computing device, a construction zone sign in the image portion;

determining, using the computing device, a type of the construction zone sign;

detecting one or more construction zone objects on the road;

determining, based on a number and locations of the one or more construction zone objects, a severity of road changes; and

controlling, using the computing device, the vehicle based on (i) the type of the construction zone sign, and (ii) the construction zone sign and the severity of road changes.

2. The method of claim 1, wherein controlling the vehicle comprises controlling the vehicle in an autonomous operation mode.

3. The method of claim 1, wherein detecting the construction zone sign comprises:

detecting a candidate construction zone sign in the image portion; and

determining that the candidate construction zone sign relates to a construction zone based on one or more of shape, color, and pattern of the candidate construction zone sign.

4. The method of claim 1, further comprising:

detecting a candidate construction zone sign in the image portion;

determining a first likelihood that the candidate construction zone sign relates to a construction zone based on one or more of shape, color, and pattern of the candidate construction zone sign;

receiving, at the computing device, from a light detection and ranging (LIDAR) device coupled to the vehicle, LIDAR-based information comprising a three-dimensional (3D) point cloud corresponding to the image portion, wherein the 3D point cloud comprises a set of points based on light emitted from the LIDAR and reflected from a surface of the candidate construction zone sign; and

determining a second likelihood that the candidate construction zone sign relates to the construction zone based on the LIDAR-based information, wherein detecting the construction zone sign is based on the first likelihood and the second likelihood.

5. The method of claim 1, further comprising:

detecting a candidate construction zone sign in the image portion;

determining a first likelihood that the candidate construction zone sign relates to a construction zone, based on one or more of shape, color, and pattern of the candidate construction zone sign;

receiving, at the computing device, from a radio detection and ranging (RADAR) device coupled to the vehicle, RADAR-based information relating to location and one or more characteristics of the candidate construction zone sign; and

determining a second likelihood that the candidate construction zone sign relates to the construction zone based on the RADAR-based information, wherein detecting the construction zone sign is based on the first likelihood and the second likelihood.

6. The method of claim 5, wherein the RADAR-based information includes dimensional characteristics of the candidate construction zone sign and indicates that the candidate construction zone sign is stationary.

7. The method of claim 1, wherein determining the type of the construction zone sign comprises further comprising:

determining a type of the construction zone sign by determining a shape of the construction zone sign; and

matching the shape to one or more of shapes of typical construction zone signs, and wherein controlling the vehicle is further based on the determined type of construction sign.

8. The method of claim 1, wherein the one or more so construction zone objects comprise a cone, a barrel, a worker, a barrier, or construction equipment.

21. The method of claim 1, further comprising selecting a control strategy for controlling the vehicle based on the severity of road changes, and wherein controlling the vehicle is further based on the control strategy.

22. The method of claim 21, wherein the control strategy comprises default driving behavior.

23. The method of claim 21, wherein the control strategy comprises defensive driving behavior.

24. The method of claim 1, wherein the controlling the vehicle is further based on traffic information.

25. The method of claim 24, wherein the traffic information is received from one or more of (i) Global Positioning Satellite (GPS) device, (ii) vehicle-to-infrastructure communication, (iii) vehicle-to-vehicle communication, and (iv) a traffic report broadcast.

26. The method of claim 1, further comprising:

determining a type of the construction zone sign by determining a shape of the construction zone sign; and

matching the shape to one or more of shapes of typical construction zone signs, and wherein controlling the vehicle is further based on the determined type of construction sign.

9. A non-transitory computer readable medium having stored thereon instructions that, when executed by a computing device of a vehicle, cause the computing device to perform operations comprising:

receiving, from an image-capture device coupled to the vehicle, an image of a vicinity of a road of travel of the vehicle;

selecting, from the image, an image portion corresponding to a side of the road;

detecting a construction zone sign in the image portion;

determining a type of the construction zone sign;

detecting one or more construction zone objects on the road;

determining, based on a number and locations of the one or more construction zone objects, a severity of road changes; and

controlling the vehicle based on (i) the type of the construction zone sign, and (ii) the construction zone sign and the severity of road changes.

10. The non-transitory computer readable medium of claim 9, wherein the one or more construction zone objects comprise a cone, a barrel, a worker, a barrier, or construction equipment.

11. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

identifying a lane boundary defined by the one or more constructions zone objects, wherein determining the severity of road changes is further based on the identified lane boundary.

12. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

detecting a candidate construction zone sign in the image portion;

determining a first likelihood that the candidate construction zone sign relates to a construction zone, based on one or more of shape, color, and pattern of the candidate construction zone sign;

receiving, from a light detection and ranging (LIDAR) device coupled to the vehicle, LIDAR-based information comprising a three-dimensional (3D) point cloud corresponding to the image portion, wherein the 3D point cloud comprises a set of points based on light emitted from the LIDAR and reflected from a surface of the candidate construction zone sign; and

determining a second likelihood that the candidate construction zone sign relates to the construction zone based on the LIDAR-based information, wherein detecting the construction zone sign is based on the first likelihood and the second likelihood.

13. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

detecting a candidate construction zone sign in the image portion;

determining a first likelihood that the candidate construction zone sign relates to a construction zone, based on one or more of shape, color, and pattern of the candidate construction zone sign;

receiving from a radio detection and ranging (RADAR) device coupled to the vehicle, RADAR-based information relating to location and one or more characteristics of the candidate construction zone sign; and

determining a second likelihood that the candidate construction zone sign relates to the construction zone based on the RADAR-based information, wherein detecting the construction zone sign is based on the first likelihood and the second likelihood.

14. The non-transitory computer readable medium of claim 13, wherein the RADAR-based information includes dimensional characteristics of the candidate construction zone sign and indicates that the candidate construction zone sign is stationary.

15. The non-transitory computer readable medium of claim 9, wherein determining the type of the construction zone sign comprises further comprising:

determining a type of the construction sign by determining a shape of the construction zone sign in the image portion; and

matching the shape to one or more of shapes of typical construction zone signs, and wherein controlling the vehicle is further based on the determined type of construction sign.

16. A control system for a vehicle, comprising:

an image-capture device;

a computing device in communication with the image-capture device; and

data storage comprising instructions that, when executed by the computing device, cause the control system to perform operations comprising;

receiving, from an image-capture device coupled to the vehicle, an image of a vicinity of a road of travel of the vehicle,

selecting, from the image, an image portion corresponding to a side of the road;

detecting a construction zone sign in the image portion,

determining a type of the construction zone sign,

detecting one or more construction zone objects on the road,

determining, based on a number and locations of the one or more construction zone objects, a severity of road changes, and

controlling the vehicle based on (i) the type of the construction zone sign, and (ii) the construction zone sign and the severity of road changes.

17. The control system of claim 16, wherein controlling the vehicle comprises controlling the vehicle in an autonomous operation mode.

18. The control system of claim 16, wherein detecting the construction zone sign comprises:

detecting a candidate construction zone sign in the image portion; and

determining that the candidate construction zone sign relates to a construction zone based on one or more of shape, color, and pattern of the candidate construction zone sign.

19. The control system of claim 16, wherein the one or more construction zone objects comprise a cone, a barrel, a worker, a barrier, or construction equipment.

20. The control system of claim 16, wherein the operations further comprise:

identifying a lane boundary defined by the one or more constructions zone objects, wherein determining the severity of road changes is further based on the identified lane boundary.