Beam split extended dynamic range image capture system

• US 9,979,906 B2
• Filed: 08/03/2016
• Issued: 05/22/2018
• Est. Priority Date: 08/03/2016
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Lewkow, Roman, Shand, Mark
• Primary Examiner(s): Flohre, Jason A
• Application Number:
  US15/227,113
• Publication Number:
  US 20180041721A1
• Time in Patent Office: 657 Days
• CPC Class Codes: G02B 27/1013 for colour or multispectral image sensors, e.g. splitting an image into monochromatic image components on respective sensors spectral imaging systems G01J
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 27/141 using dichroic mirrors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/57 Control of the dynamic range
• CPC Class Codes: H04N 25/581 acquired simultaneously
• CPC Class Codes: H04N 25/583 with different integration times

Abstract

An optical system and method may relate to capturing extended dynamic range images. In an example embodiment, the optical system may include a lens element configured to receive incident light and a beam splitter optically coupled to the lens element. The beam splitter is configured to separate the incident light into at least a first portion having a first photon flux and a second portion having a second photon flux. The first photon flux is at least an order of magnitude greater than the second photon flux. A controller may be configured to cause a first image sensor to capture a first image of the first portion of the incident light according to first exposure parameters and cause a second image sensor to capture a second image of the second portion of the incident light according to second exposure parameters.

Claims

1. An optical system comprising:

at least one lens element configured to receive incident light;

at least one beam splitter optically coupled to the at least one lens element, wherein the at least one beam splitter is configured to separate the incident light into at least a first portion having a first photon flux and a second portion having a second photon flux, wherein the first photon flux is at least an order of magnitude greater than the second photon flux;

a first image sensor configured to receive the first portion of the incident light;

a second image sensor configured to receive the second portion of the incident light; and

a controller comprising at least one processor and a memory, wherein the controller is configured to execute instructions stored in the memory so as to carry out operations, the operations comprising;

causing the first image sensor to capture a first image of the first portion of the incident light according to first exposure parameters, wherein the first exposure parameters are predetermined based on a known periodic illumination time associated with a temporally-varying photon flux source; and

causing the second image sensor to capture a second image of the second portion of the incident light according to second exposure parameters, wherein the second exposure parameters are related to the temporally-varying photon flux source, and wherein the second exposure parameters are adjustable based on a scene associated with the incident light.

2. The optical system of claim 1, wherein the operations of the controller further comprise determining an extended dynamic range image based on the first image and the second image.

3. The optical system of claim 1, wherein the at least one beam splitter comprises a partially-silvered mirror or glass having an anti-reflection coating.

4. The optical system of claim 1, wherein the at least one beam splitter comprises a dichroic optical element or a frequency selective surface.

5. The optical system of claim 1, wherein the temporally-varying photon flux source comprises two or more light segments, wherein the second exposure parameters are based on operation of the two or more light segments, and wherein operation of the two or more light segments comprises illuminating at least two of the two or more light segments at different times within a known time frame of the temporally-varying photon flux source, and wherein the incident light comprises light emitted from the temporally-varying photon flux source.

6. The optical system of claim 5, wherein the temporally-varying photon flux source comprises a raster display or a flashing display, and wherein the raster display or the flashing display comprises a traffic signal or an automotive indicator.

7. The optical system of claim 1, further comprising at least one optical element, wherein the at least one optical element is arranged along an optical path between the at least one beam splitter and at least one of the first image sensor or the second image sensor, wherein the at least one optical element comprises at least one of:

an optical attenuator, a spatial filter, or a lens.

8. The optical system of claim 1, wherein the first exposure parameters comprises a first exposure time and the second exposure parameters comprise a second exposure time, wherein the first exposure time is at least an order of magnitude shorter than the second exposure time.

9. The optical system of claim 1, wherein the at least one beam splitter comprises a first beam splitter and a second beam splitter, wherein the at least one beam splitter is further configured to separate the incident light into a third portion having a third photon flux, wherein the second photon flux is at least an order of magnitude greater than the third photon flux.

10. The optical system of claim 9, further comprising a third image sensor, wherein the operations further comprise:

causing the third image sensor to capture a third image of the third portion of the incident light according to third exposure parameters; and

determining a bracketed extended dynamic range image based on the first image, the second image, and the third image;

wherein at least one of the first image sensor, the second image sensor, or the third image sensor has at least one of;

a different spectral response, a different spatial resolution, or a different readout rate with respect to the other image sensors.

17. The optical system of claim 1, wherein the first exposure parameters and the second exposure parameters differ based on operation of two or more light segments comprised within the temporally-varying photon flux source.

18. The optical system of claim 1, wherein the second exposure parameters are based on a prior image of a scene associated with the incident light.

11. A method comprising:

receiving incident light via at least one lens element;

separating, by at least one beam splitter, the incident light into at least a first portion having a first photon flux and a second portion having a second photon flux, wherein the first photon flux is at least an order of magnitude greater than the second photon flux;

causing a first image sensor to capture a first image of the first portion of the incident light according to first exposure parameters, wherein the first exposure parameters are predetermined based on a known periodic illumination time associated with a temporally-varying photon flux source; and

causing a second image sensor to capture a second image of the second portion of the incident light according to second exposure parameters, wherein the second exposure parameters are related to the temporally-varying photon flux source, and wherein the second exposure parameters are adjustable based on a scene associated with the incident light.

12. The method of claim 11, further comprising determining an extended dynamic range image based on the first image and the second image.

13. The method of claim 11, wherein the temporally-varying photon flux source comprises a raster display or a flashing display, wherein the raster display or the flashing display comprises a traffic signal, wherein the first exposure parameters and the second exposure parameters differ based on operation of two or more light segments of the traffic light, and wherein operation of the two or more light segments comprises illuminating at least two of the two or more light segments at different times within a known frame time of the traffic signal.

14. The method of claim 11, wherein the first exposure parameters comprises a first exposure time and the second exposure parameters comprise a second exposure time, wherein the first exposure time is at least an order of magnitude shorter than the second exposure time.

15. The method of claim 11, wherein the at least one beam splitter comprises a first beam splitter and a second beam splitter, wherein separating the incident light further comprises separating the incident light into a third portion having a third photon flux, wherein the second photon flux is at least an order of magnitude greater than the third photon flux.

16. The method of claim 15, further comprising:

causing the third image sensor to capture a third image of the third portion of the incident light according to third exposure parameters; and

determining a bracketed extended dynamic range image based on the first image, the second image, and the third image.

19. A method comprising:

receiving incident light via at least one lens element;

separating, by at least one beam splitter, the incident light into at least a first portion having a first photon flux and a second portion having a second photon flux, wherein the first photon flux is at least an order of magnitude greater than the second photon flux;

causing a first image sensor to capture a first image of the first portion of the incident light according to first exposure parameters, wherein the first exposure parameters are based on a known first periodic illumination time associated with one light segment of two or more light segments of a temporally-varying photon flux source; and

causing a second image sensor to capture a second image of the second portion of the incident light according to second exposure parameters, wherein the second exposure parameters are related to the temporally-varying photon flux source, and wherein the second exposure parameters are based on a known second periodic illumination time associated with at least two of the two or more light segments of the temporally-varying photon flux source.