Road flare detection

• US 9,195,236 B1
• Filed: 09/05/2012
• Issued: 11/24/2015
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Fairfield, Nathaniel, Ferguson, David Ian
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): JHA, ABDHESH K
• Application Number:
  US13/603,609
• Time in Patent Office: 1,175 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/08 including control of electric propulsion units, e.g. motors or generators
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 22/00 Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees

Abstract

Methods and systems for road flare detection are described. A computing device configured to control the vehicle, may be configured to receive, from an image-capture device, images; image portions of the images may depict an area within a predetermined distance from a surface of a road on which the vehicle is travelling. Also, the computing device may be configured to determine, based on characteristics of an object depicted in the image portions such as color spectrum of the object, brightness of the object, and whether the object is stationary or moving with respect to the road, a likelihood that the object represents a road flare. Based on the likelihood, the computing device may be configured to modify a control strategy associated with a driving behavior of the vehicle; and control the vehicle based on the modified control strategy.

Claims

1. A method, comprising:

detecting, by computing device configured to control a vehicle, a bright object on a road of travel of the vehicle;

receiving, at the computing device, from an image-capture device coupled to the vehicle, an image of the road;

selecting, using the computing device, a portion of the image depicting an area within a predetermined threshold distance from a surface of the road proximate to the bright object, wherein the predetermined threshold distance is based on a height that encompasses a typical road flare;

determining, using the computing device, based on (a) whether the portion of the image includes the bright object, and (b) one or more characteristics of the bright object, a likelihood that the bright object represents a road flare, wherein the one or more characteristics include at least one of (i) a color spectrum of the bright object, (ii) a brightness level of the bright object, and (iii) whether the bright object is stationary or moving with respect to the road;

in response to the likelihood exceeding a predetermined threshold likelihood, identifying a hazardous condition indicated by the road flare and road changes resulting from the hazardous condition indicated by the road flare;

modifying, using the computing device, a control strategy associated with a driving behavior of the vehicle, based on the likelihood, the hazardous condition, and the road changes resulting from the hazardous condition; and

controlling, using the computing device, the vehicle based on the modified control strategy.

2. The method of claim 1, wherein the vehicle is in an autonomous operation mode.

3. The method of claim 1, further comprising;

receiving information indicative of a location of the surface of the road proximate to the bright object from one or more other sensors;

determining the location of the surface of the road based on the information indicative of the location of the surface of the road received from the one or more other sensors; and

determining the predetermined threshold distance from the surface of the road based on the determined location of the surface.

4. The method of claim 3, wherein the information includes at least one of:

(i) preexisting map of surface elevation of the road;

(ii) pitch and roll angles indicative of a driving attitude of the vehicle; and

(iii) LIDAR-based information corresponding to light emitted from a light detection and ranging (LIDAR) sensor and reflected from the surface of the road.

5. The method of claim 1, wherein receiving, from the image-capture device coupled to the computing device, the image comprises receiving a sequence of images, wherein image capturing parameters of the image-capture device are varied while capturing the sequence of images, and wherein the image capturing parameters include one or more of speed of exposure, aperture, resolution, and focal depth.

6. A non-transitory computer readable medium having stored thereon instructions executable by a computing device of a vehicle to cause the computing device to perform functions comprising:

detecting a bright object on a road of travel of the vehicle;

receiving, from an image-capture device coupled to the vehicle, an image of the road;

selecting, using the computing device, a portion of the image depicting an area within a predetermined threshold distance from a surface of the road proximate to the bright object, wherein the predetermined threshold distance is based on a height that encompasses a typical road flare;

determining, based on (a) whether the portion of the image includes the bright object, and (b) one or more characteristics of the bright object, a likelihood that the bright object represents a road flare, wherein the one or more characteristics include at least one of (i) a color spectrum of the bright object, (ii) a brightness level of the bright object, and (iii) whether the bright object is stationary or moving with respect to the road;

in response to the likelihood exceeding a predetermined threshold likelihood, identifying a hazardous condition indicated by the road flare and road changes resulting from the hazardous condition indicated by the road flare;

modifying a control strategy associated with a driving behavior of the vehicle, based on the likelihood, the hazardous condition, and the road changes resulting from the hazardous condition; and

controlling the vehicle based on the modified control strategy.

7. The non-transitory computer readable medium of claim 6, wherein the vehicle is in an autonomous operation mode.

8. The non-transitory computer readable medium of claim 6, wherein the instructions are further executable by the computing device to cause the computing device to perform functions comprising:

receiving information indicative of a location of the surface of the road proximate to the bright object from one or more other sensors;

determining the location of the surface of the road based on the information indicative of the location of the surface of the road received from the one or more other sensors; and

determining the predetermined threshold distance from the surface of the road based on the determined location of the surface.

9. The non-transitory computer readable medium of claim 8, wherein the information includes at least one of:

(i) preexisting map of surface elevation of the road;

(ii) pitch and roll angles indicative of a driving attitude of the vehicle; and

(iii) LIDAR-based information corresponding to light emitted from a light detection and ranging (LIDAR) sensor and reflected from the surface of the road.

10. The non-transitory computer readable medium of claim 6, the function of receiving the image comprises receiving a sequence of images, wherein image capturing parameters of the image-capture device are varied while capturing the sequence of images, and wherein the image capturing parameters include one or more of speed of exposure, aperture, resolution, and focal depth.

11. The non-transitory computer readable medium of claim 6, wherein the function of controlling the vehicle based on the modified control strategy comprises one or more of:

(i) utilizing sensor information received from on-board or off-board sensors in making a navigation decision rather than preexisting map information, (ii) utilizing the sensor information to estimate lane boundaries rather than the preexisting map information, (iii) determining locations of hazard condition markers rather than lane markers on the road to estimate and follow the lane boundaries, (iv) activating one or more sensors for detection of humans and making the navigation decision based on the detection, (v) following another vehicle, (vi) maintaining a predetermined safe distance with other vehicles, (vii) turning-on lights, and (viii) reducing a speed of the vehicle.

12. The non-transitory computer readable medium of claim 6, wherein the instructions are further executable by the computing device to cause the computing device to perform functions comprising:

determining the one or more characteristics of the object; and

determining, for each characteristic of the one or more characteristics, a respective likelihood based on a comparison of that characteristic to a corresponding characteristic of a typical road flare, wherein determining the likelihood that the object represents the road flare is based on a combination of the respective likelihoods of the one or more characteristics.

13. A control system for a vehicle, comprising:

an image-capture device; and

a computing device in communication with the image capture-device and configured to;

detect a bright object on a road of travel of the vehicle;

receive, from the image-capture device, an image of the road;

selecting, using the computing device, a portion of the image depicting an area within a predetermined threshold distance from a surface of the road proximate to the bright object, wherein the predetermined threshold distance is based on a height that encompasses a typical road flare;

determine, based on (a) whether the portion of the image includes the bright object, and (b) one or more characteristics of the bright object depicted in the one or more image portions, a likelihood that the bright object represents a road flare, wherein the one or more characteristics include at least one of (i) a color spectrum of the bright object, (ii) a brightness level of the bright object, and (iii) whether the bright object is stationary or moving with respect to the road;

in response to the likelihood exceeding a predetermined threshold likelihood, identifying a hazardous condition indicated by the road flare and road changes resulting from the hazardous condition indicated by the road flare;

modify a control strategy associated with a driving behavior of the vehicle, based on the likelihood, the hazardous condition, and the road changes resulting from the hazardous condition; and

control the vehicle based on the modified control strategy.

14. The system of claim 13, wherein the computing device is further configured to control the vehicle in an autonomous operation mode.

15. The system of claim 13, wherein the computing device is further configured to:

receive information indicative of a location of the surface of the road proximate to the bright object from one or more other sensors coupled to the vehicle;

determine the location of the surface of the road based on the information indicative of the location of the surface of the road received from the one or more other sensors; and

determine the predetermined threshold distance from the surface of the road based on the determined location of the surface.

16. The system of claim 15, wherein the information includes at least one of:

(i) preexisting map of surface elevation of the road;

(ii) pitch and roll angles indicative of a driving attitude of the vehicle; and

(iii) LIDAR-based information corresponding to light emitted from a light detection and ranging (LIDAR) sensor and reflected from the surface of the road.

17. The system of claim 13, wherein the computing device is further configured to receive a sequence of images, wherein image capturing parameters of the image-capture device are varied while capturing the sequence of images, and wherein the image capturing parameters include one or more of speed of exposure, aperture, resolution, and focal depth.