Camera systems using filters and exposure times to detect flickering illuminated objects

• US 10,776,938 B2
• Filed: 09/11/2019
• Issued: 09/15/2020
• Est. Priority Date: 05/19/2017
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Wendel, Andreas, Ingram, Benjamin
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US16/567,221
• Publication Number:
  US 20200005477A1
• Time in Patent Office: 370 Days
• Field of Search: 382104
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle'"'"'s environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle'"'"'s environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.

Claims

1. A method comprising:

capturing images using a first camera and a first exposure time, the first camera being mounted in order to capture images of an environment, the first camera being without an ND filter;

capturing images using a second camera and a second exposure time, the second camera being mounted in order to capture images of the environment, the second exposure time being greater than or equal to the first exposure time and having an ND filter;

using the images captured using the second camera to identify illuminated objects;

using the images captured using the first camera to identify the locations of objects; and

using the identified illuminated objects and identified locations of objects to control a vehicle in an autonomous driving mode.

2. The method of claim 1, wherein the first camera and the second camera each include a near infrared filter.

3. The method of claim 1, wherein the second exposure time is on the order of milliseconds.

4. The method of claim 3, wherein the second exposure time is at least 5 milliseconds and the first exposure time is no greater than 5 milliseconds.

5. The method of claim 1, wherein the ND filter is selected according to the second exposure time.

6. The method of claim 1, wherein the ND filter is implemented at a pixel level for the second camera.

7. The method of claim 1, further comprising, using the images of the second camera to identify illuminated images by identifying light from a PWM light source.

8. The method of claim 1, further comprising using the images of the second camera to identify illuminated images by identifying text generated by a plurality of PWM light sources comprising LEDs.

9. The method of claim 8, further comprising selecting the second exposure time based on a frequency of the PWM light sources.

10. The method of claim 1, further comprising using the images of the second camera to identify illuminated images by identifying light from a light source which flickers at a rate defined by a power grid that supplies power to the light source.

11. The method of claim 10, further comprising, selecting the second exposure time based on a rate defined by the power grid.

12. The method of claim 1, wherein the second exposure time is a fixed exposure time.

13. The method of claim 1, wherein the first exposure time is a variable exposure time, and the method further comprises adjusting the first exposure time according to ambient lighting conditions.

14. The method of claim 13, wherein the second exposure time is always greater than the first exposure time.

15. The method of claim 13, wherein the second exposure time is a variable exposure time.

16. The method of claim 1 wherein the second camera includes a set of photodiodes and the ND filter is arranged to filter light before the light reaches the set of photodiodes, and the method further comprises, exposing the set of photodiodes using a fixed exposure time of at least 5 milliseconds in order to capture an image, wherein the exposure time allows the camera to capture light from a PWM light source during the exposure time, the PWM light being located in an environment of the vehicle.

17. The method of claim 16, further comprising using a near-infrared filter arranged to filter light before the light reaches the set of photodiodes.

18. The method of claim 1, wherein the second camera includes a set of photodiodes, the ND filter is arranged to filter light before the light reaches the set of photodiodes, and the method further includes exposing the set of photodiodes using a fixed exposure time of at least 5 milliseconds in order to capture an image, wherein the exposure time allows the camera to capture light from a light source which flickers at a rate defined by a power grid that supplies power to the light source, the light source being located in an environment of the vehicle.

19. The method of claim 18, further comprising using a near-infrared to filter light before the light reaches the set of photodiodes.