Vison-based object detection using a polar grid

• US 10,168,712 B1
• Filed: 08/08/2017
• Issued: 01/01/2019
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Lo, Wan-Yen, Ferguson, David Ian Franklin, Ogale, Abhijit
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Rivera-Martinez, Guillermo M
• Application Number:
  US15/671,316
• Time in Patent Office: 511 Days
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A computing device of a first vehicle may receive a first image and a second image of a second vehicle having flashing light signals. The computing device may determine, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle. The computing device may determine a polar grid that partitions the image region in the first image and the second image into polar bins, and identify portions of image data exhibiting a change in color and a change in brightness between the first image and the second image. The computing device may determine a type of the flashing light signals and a type of the second vehicle; and accordingly provide instructions to control the first vehicle.

Claims

1. A method comprising:

receiving, at a computing device of a first vehicle, a first image of a second vehicle captured by an image-capture device coupled to the first vehicle and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;

determining a polar grid that partitions an image region in the first image and the second image into a plurality of polar bins;

identifying, based on a comparison of image content of polar bins in the first image to image content of corresponding polar bins in the second image, one or more portions of image data exhibiting a change in brightness between the first image and the second image;

determining that several polar bins of the plurality of polar bins are void of image pixels exhibiting the change in brightness between the first image and the second image;

merging the several polar bins into a single polar bin;

determining a type of the one or more flashing light signals of the second vehicle based on the one or more portions exhibiting the change in brightness; and

controlling the first vehicle based on the type of the one or more flashing light signals.

2. The method of claim 1, wherein identifying the one or more portions of image data comprises:

within each polar bin in the first image, identifying image portions that exhibit characteristics of a bright light having an intensity greater than a threshold intensity;

within each corresponding polar bin in the second image, identifying corresponding image portions; and

determining the change in brightness for the image portions from the polar bin in the first image to the corresponding polar bin in the second image.

3. The method of claim 1, wherein the polar bins are non-uniform in size.

4. The method of claim 1, wherein each polar bin of the plurality of polar bins is defined by two lines extending from a center portion of the image region to about a boundary of the image region.

5. The method of claim 1, wherein the image region bounds the second vehicle such that the image region substantially encompasses the second vehicle.

6. The method of claim 5, wherein the second vehicle is moving relative to the first vehicle such that the image region that bounds the second vehicle in the first image is different from a respective image region that bounds the second vehicle in the second image.

7. The method of claim 1, further comprising:

assigning each pixel in the image region to a respective polar bin of the polar bins.

8. The method of claim 1, further comprising:

determining a type of the second vehicle based on (i) a number of portions of image data exhibiting the change in brightness, (ii) color of the one or more portions, and (iii) the brightness of the one or more portions, wherein controlling the first vehicle is further based on the type of the second vehicle.

9. The method of claim 1, wherein determining the type of a given flashing light signal of the one or more flashing light signals comprises determining that the given flashing light is a vehicular turn signal or a brake light.

10. The method of claim 1, wherein the first image and the second image are captured consecutively.

11. The method of claim 1, wherein the computing device is configured to control the first vehicle in an autonomous operation mode.

12. A non-transitory computer readable medium having stored thereon executable instructions that, upon execution by a computing device of a first vehicle, cause the computing device to perform operations comprising:

receiving a first image of a second vehicle captured by an image-capture device coupled to the first vehicle and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;

determining a polar grid that partitions an image region in the first image and the second image into a plurality of polar bins;

identifying, based on a comparison of image content of polar bins in the first image to image content of corresponding polar bins in the second image, one or more portions of image data exhibiting a change in brightness between the first image and the second image;

determining that several polar bins of the plurality of polar bins are void of image pixels exhibiting the change in brightness between the first image and the second image;

merging the several polar bins into a single polar bin;

determining a type of the one or more flashing light signals of the second vehicle based on the one or more portions exhibiting the change in brightness; and

controlling the first vehicle based on the type of the one or more flashing light signals.

13. The non-transitory computer readable medium of claim 12, wherein the operation of identifying the one or more portions of image data comprises:

within each polar bin in the first image, identifying image portions that exhibit characteristics of a bright light having an intensity greater than a threshold intensity;

within each corresponding polar bin in the second image, identifying corresponding image portions; and

determining the change in brightness for the image portions from the polar bin in the first image to the corresponding polar bin in the second image.

14. The non-transitory computer readable medium of claim 12, wherein the polar bins are non-uniform in size.

15. The non-transitory computer readable medium of claim 12, wherein the image region bounds the second vehicle such that the image region substantially encompasses the second vehicle.

16. The non-transitory computer readable medium of claim 12, wherein the operations further comprise:

determining a type of the second vehicle based on (i) a number of portions of image data exhibiting the change in brightness, (ii) color of the one or more portions, and (iii) the brightness of the one or more portions, wherein controlling the first vehicle is further based on the type of the second vehicle.

17. A system comprising:

an image-capture device coupled to a first vehicle;

at least one processor in communication with the image-capture device; and

a memory having stored thereon executable instructions that, upon execution by the at least one processor, cause the system to perform operations comprising;

receiving a first image of a second vehicle captured by the image-capture device coupled to the first vehicle and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;

determining a polar grid that partitions an image region in the first image and the second image into a plurality of polar bins;

identifying, based on a comparison of image content of polar bins in the first image to image content of corresponding polar bins in the second image, one or more portions of image data exhibiting a change in brightness between the first image and the second image;

determining that several polar bins of the plurality of polar bins are void of image pixels exhibiting the change in brightness between the first image and the second image;

merging the several polar bins into a single polar bin;

determining a type of the one or more flashing light signals of the second vehicle based on the one or more portions exhibiting the change in brightness; and

controlling the first vehicle based on the type of the one or more flashing light signals.

18. The system of claim 17, wherein the operation of identifying the one or more portions of image data comprises:

within each polar bin in the first image, identifying image portions that exhibit characteristics of a bright light having an intensity greater than a threshold intensity;

within each corresponding polar bin in the second image, identifying corresponding image portions; and

determining the change in brightness for the image portions from the polar bin in the first image to the corresponding polar bin in the second image.

19. The system of claim 17, wherein the operations further comprise:

determining a type of the second vehicle based on (i) a number of portions of image data exhibiting the change in brightness, (ii) color of the one or more portions, and (iii) the brightness of the one or more portions, wherein controlling the first vehicle is further based on the type of the second vehicle.

20. The system of claim 17, wherein the image region bounds the second vehicle such that the image region substantially encompasses the second vehicle, and wherein the second vehicle is moving relative to the first vehicle such that the image region that bounds the second vehicle in the first image is different from a respective image region that bounds the second vehicle in the second image.