Vision-based indicator signal detection using spatiotemporal filtering

• US 9,305,223 B1
• Filed: 06/26/2013
• Issued: 04/05/2016
• Est. Priority Date: 06/26/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ogale, Abhijit, Ferguson, David Ian Franklin
• Primary Examiner(s): Sun, Yulin
• Application Number:
  US13/927,248
• Time in Patent Office: 1,014 Days
• Field of Search: 348/149
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/80 characterised by the intended use of the viewing arrangement
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

An autonomous vehicle is configured to detect an active turn signal indicator on another vehicle. An image-capture device of the autonomous vehicle captures an image of a field of view of the autonomous vehicle. The autonomous vehicle captures the image with a short exposure to emphasize objects having brightness above a threshold. Additionally, a bounding area for a second vehicle located within the image is determined. The autonomous vehicle identifies a group of pixels within the bounding area based on a first color of the group of pixels. The autonomous vehicle also calculates an oscillation of an intensity of the group of pixels. Based on the oscillation of the intensity, the autonomous vehicle determines a likelihood that the second vehicle has a first active turn signal. Additionally, the autonomous vehicle is controlled based at least on the likelihood that the second vehicle has a first active turn signal.

Claims

1. A method comprising:

receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;

determining a bounding area within the image including a depiction of a second vehicle;

identifying a first group of pixels in the image based on a first color of the first group of pixels, wherein the first group of pixels is located within a region of the bounding area;

identifying a second group of pixels in the image based on a second color of the second group of pixels, wherein the second group of pixels is located within the bounding area;

determining a correlation between an intensity of the first color of the first group of pixels and an intensity of the second color of the second group of pixels;

determining, based on the correlation, a likelihood that the second vehicle has an active indicator signal; and

providing instructions to control, using a computing device, the first vehicle based at least on the likelihood that the second vehicle has an active indicator signal.

2. The method of claim 1, wherein the bounding area is determined based on a previously stored image.

3. The method of claim 1, wherein determining the correlation comprises calculating an oscillation of an intensity of the first group of pixels, and wherein the oscillation is between 0.75 and 2.5 Hertz.

4. The method of claim 1, further comprising:

determining, based on a second oscillation of the second group of pixels, a likelihood that the second vehicle has another active indicator signal; and

determining, based on both the likelihood that the second vehicle has an active indicator signal and the likelihood that the second vehicle has another active indicator signal, a likelihood that the second vehicle has active emergency lights.

5. The method of claim 1, wherein the bounding area defines a region of the image in which the second vehicle is located.

6. The method of claim 1, further comprising identifying a sun glare based on the correlation.

7. A control system comprising:

at least one processor; and

a memory having stored thereon instructions that, upon execution by the at least one processor, cause the control system to perform functions comprising;

receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;

determining a bounding area within the image including a depiction of a second vehicle located within the image;

identifying a first group of pixels in the image based on a first color of the first group of pixels, wherein the first group of pixels is located within a region of the bounding area;

identifying a second group of pixels in the image based on a second color of the second group of pixels, wherein the second group of pixels is located within the bounding area;

determining a correlation between an intensity of the first color of the first group of pixels and an intensity of the second color of the second group of pixels; and

determining, based on the correlation, a likelihood that the second vehicle has an active indicator signal; and

providing instructions to control, using a computing device, the first vehicle based at least on the likelihood that the second vehicle has an active indicator signal.

8. The control system of claim 7, wherein the bounding area is determined based on a previously stored image.

9. The control system of claim 7, wherein determining the correlation comprises calculating an oscillation of an intensity of the first group of pixels, and wherein the oscillation is between 0.75 and 2.5 Hertz.

10. The control system of claim 7, further comprising:

determining, based on a second oscillation of the second group of pixels, a likelihood that the second vehicle has another active indicator signal; and

determining, based on both (i) the likelihood that the second vehicle has an active indicator signal and (ii) the likelihood that the second vehicle has another active indicator signal, a likelihood that the second vehicle has active emergency lights.

11. The control system of claim 7, wherein the bounding area defines a region of the image in which the second vehicle is located.

12. A non-transitory computer readable medium having stored thereon instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;

determining a bounding area for a second vehicle located within the image;

identifying a first group of pixels in the image based on a first color of the first group of pixels, wherein the first group of pixels is located within a region of the bounding area;

identifying a second group of pixels in the image based on a second color of the second group of pixels, wherein the second group of pixels is located within the bounding area;

determining a correlation between an intensity of the first color of the first group of pixels and an intensity of the second color of the second group of pixels; and

determining, based on the correlation, a likelihood that the second vehicle has a first active turn signal; and

providing instructions to control, using a computing device, the vehicle based at least on the likelihood that the second vehicle has a first active turn signal.

13. The non-transitory computer readable medium of claim 12, wherein the bounding area is determined based a previously stored image.

14. The non-transitory computer readable medium of claim 12, wherein determining the correlation comprises calculating an oscillation of an intensity of the first group of pixels, and wherein the oscillation is between 0.75 and 2.5 Hertz.

15. The non-transitory computer readable medium of claim 12, further comprising:

determining, based on a second oscillation of the second group of pixels, a likelihood that the second vehicle has a second active turn signal; and

determining, based on both (i) the likelihood that the second vehicle has a first active turn signal and (ii) the likelihood that the second vehicle has a second active turn signal, a likelihood that the second vehicle has active emergency lights.

16. The non-transitory computer readable medium of claim 12, wherein the bounding area defines a region of the image in which the second vehicle is located.

17. The non-transitory computer readable medium of claim 12, further comprising identifying a sun glare based on the correlation.