Camera systems using filters and exposure times to detect flickering illuminated objects

• US 10,453,208 B2
• Filed: 06/05/2017
• Issued: 10/22/2019
• Est. Priority Date: 05/19/2017
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Wendel, Andreas, Ingram, Benjamin
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US15/613,546
• Publication Number:
  US 20180336692A1
• Time in Patent Office: 869 Days
• Field of Search: 382104
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

The technology relates to camera systems for vehicles having an autonomous driving mode. An example system includes a first camera mounted on a vehicle in order to capture images of the vehicle'"'"'s environment. The first camera has a first exposure time and being without an ND filter. The system also includes a second camera mounted on the vehicle in order to capture images of the vehicle'"'"'s environment and having an ND filter. The system also includes one or more processors configured to capture images using the first camera and the first exposure time, capture images using the second camera and the second exposure time, use the images captured using the second camera to identify illuminated objects, use the images captured using the first camera to identify the locations of objects, and use the identified illuminated objects and identified locations of objects to control the vehicle in an autonomous driving mode.

Claims

1. A system comprising:

a first camera mounted in order to capture images of an environment, the first camera having a first exposure time and being without an ND filter;

a second camera mounted in order to capture images of the environment, the second camera having a second exposure time that is greater than or equal to the first exposure time and having an ND filter;

one or more processors configured to;

capture images using the first camera and the first exposure time;

capture images using the second camera and the second exposure time;

use the images captured using the second camera to identify illuminated objects;

use the images captured using the first camera to identify the locations of objects; and

use the identified illuminated objects and identified locations of objects to control a vehicle in an autonomous driving mode.

2. The system of claim 1, wherein the first camera and the second camera each include a near infrared filter.

3. The system of claim 1, wherein the second exposure time is on the order of milliseconds.

4. The system of claim 3, wherein the second exposure time is at least 5 milliseconds and the first exposure time is no greater than 5 milliseconds.

5. The system of claim 1, wherein the ND filter is selected according to the second exposure time.

6. The system of claim 1, wherein the ND filter is implemented at a pixel level for the second camera.

7. The system of claim 1, further comprising the vehicle.

8. The system of claim 1, wherein the one or more processors are further configured to use the images of the second camera to identify illuminated images by identifying light from a PWM light source.

9. The system of claim 1, wherein the one or more processors are further configured to use the images of the second camera to identify illuminated images by identifying text generated by a plurality of PWM light sources comprising LEDs.

10. The system of claim 9, wherein the one or more processors are further configured to select the second exposure time based on a frequency of the PWM light sources.

11. The system of claim 1, wherein the one or more processors are further configured to use the images of the second camera to identify illuminated images by identifying light from a light source which flickers at a rate defined by a power grid that supplies power to the light source.

12. The system of claim 11, wherein the one or more processors are further configured to select the second exposure time based on a rate defined by the power grid.

13. The system of claim 1, wherein the second exposure time is a fixed exposure time.

14. The system of claim 1, wherein the first exposure time is a variable exposure time that is adjusted according to ambient lighting conditions.

15. The system of claim 14, wherein the second exposure time is always greater than the first exposure time.

16. The system of claim 14, wherein the second exposure time is a variable exposure time.

17. The system of claim 1 wherein the second camera further comprises:

a set of photodiodes; and

an ND filter arranged to filter light before the light reaches the set of photodiodes; and

a controller configured to expose the set of photodiodes using a fixed exposure time of at least 5 milliseconds in order to capture an image, wherein the exposure time allows the camera to capture light from a PWM light source during the exposure time, the PWM light being located in an environment of the vehicle.

18. The system of claim 17, further comprising a near-infrared filter arranged to filter light before the light reaches the set of photodiodes.

19. The system of claim 1, wherein the second camera further comprises:

a set of photodiodes;

an ND filter arranged to filter light before the light reaches the set of photodiodes; and

a controller configured to expose the set of photodiodes using a fixed exposure time of at least 5 milliseconds in order to capture an image, wherein the exposure time allows the camera to capture light from a light source which flickers at a rate defined by a power grid that supplies power to the light source, the light source being located in an environment of the vehicle.

20. The system of claim 19, further comprising a near-infrared filter arranged to filter light before the light reaches the set of photodiodes.