Rotating LIDAR with Co-Aligned Imager

• US 20190052844A1
• Filed: 08/08/2017
• Published: 02/14/2019
• Est. Priority Date: 08/08/2017
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Droz, Pierre-Yves, Verghese, Simon, Hermalyn, Brendan
• Granted Patent: US 10,447,973 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/101 using cameras with adjustable capturing direction
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4812 transmitted and received beams following a coaxial path
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.

Claims

1. A device comprising:

a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device;

a LIDAR receiver that detects reflections of the emitted light pulses from the scene;

an image sensor that captures an image of the scene according to the pointing direction of the device and based on at least external light originating from one or more external light sources;

a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement; and

an actuator that rotates the platform about an axis to adjust the pointing direction of the device.

2. The device of claim 1, further comprising a controller that associates data from the LIDAR receiver with data from the image sensor.

3. The device of claim 1, further comprising a controller that determines a three-dimensional (3D) representation of the scene based on data from the LIDAR receiver and data from the image sensor.

4. The device of claim 1, further comprising a controller that determines a representation of the scene based on color information indicated by the image sensor and distance information indicated by the LIDAR receiver.

5. The device of claim 1, further comprising a controller that causes the image sensor to capture a plurality of images while the actuator is rotating the platform.

6. The device of claim 5, wherein the controller determines a time delay between capture of two consecutive images by the image sensor based on a rate of rotation of the platform about the axis, and wherein the controller causes the image sensor to capture the plurality of images according to the determined time delay.

7. The device of claim 5, wherein the controller causes the image sensor to capture a given image of the plurality of images at a given time that is based on an emission time of one or more of the emitted light pulses.

8. The device of claim 1, wherein the image sensor comprises an array of sensing elements, wherein the image sensor generates an image pixel of the captured image based on a sum of at least:

a first image pixel indicated by a first sensing element in the array, and a second image pixel indicated by a second sensing element in the array.

9. The device of claim 8, wherein a time delay between capture of the first image pixel and capture of the second image pixel is based on at least a rate of the rotation of the platform about the axis.

10. The device of claim 1, further comprising:

a lens that receives light propagating toward the device and incident on the lens, wherein the lens transmits and focuses the received light through the lens, wherein the image sensor images the scene through the lens, and wherein the LIDAR receiver receives the reflections of the emitted light pulses through the lens.

11. The device of claim 10, further comprising:

an optical element positioned along a path of the focused light from the lens, wherein the optical element directs a first portion of the focused light toward the LIDAR receiver, and wherein the optical element directs a second portion of the focused light toward the image sensor.

12. The device of claim 10, further comprising:

a light filter positioned along a path of the focused light from the lens, wherein the light filter attenuates light having wavelengths outside a wavelength range of the emitted light pulses, wherein the LIDAR receiver receives the reflections of the emitted light pulses through the light filter, wherein the image sensor receives at least a portion of the focused light that does not propagate through the light filter.

13. A vehicle comprising:

a light detection and ranging (LIDAR) sensor that includes a transmitter and a receiver, wherein the transmitter emits light pulses toward a scene according to a pointing direction of the LIDAR sensor, and wherein the receiver detects reflections of the emitted light pulses propagating from the scene;

a camera that captures an image according to a pointing direction of the camera and based on at least external light originating from one or more external light sources;

a platform that supports the LIDAR sensor and the camera in a particular relative arrangement; and

an actuator that rotates the platform about an axis to simultaneously change the pointing direction of the LIDAR sensor and the pointing direction of the camera according to the particular relative arrangement.

14. The vehicle of claim 13, further comprising a controller that causes the camera to capture a plurality of images according to detection times of one or more reflected light pulses detected by the LIDAR sensor.

15. The vehicle of claim 13, wherein the camera comprises one or more columns of sensing elements substantially parallel to the axis of rotation of the platform.

16. The vehicle of claim 13, wherein the camera generates a column of image pixels of a composite image of the scene based on at least a sum of:

a first image pixel column captured using a first column of sensing elements in the camera, and a second image pixel column captured using a second column of sensing elements in the camera.

17. The vehicle of claim 13, further comprising:

a LIDAR lens that focuses the reflections of the emitted light pulses propagating from the environment toward a receiver of the LIDAR sensor; and

a camera lens that focuses the external light toward an array of sensing elements in the camera.

18. A method comprising:

scanning a scene using a light detection and ranging (LIDAR) sensor, wherein the LIDAR sensor emits light pulses toward the scene and detects reflections of the emitted light pulses from the scene;

generating an image of the scene using an image sensor that detects external light originating from one or more external light sources; and

rotating a platform that supports the LIDAR sensor and the image sensor in a particular relative arrangement, wherein rotating the platform comprises simultaneously rotating the LIDAR sensor and the image sensor about an axis.

19. The method of claim 18, further comprising:

obtaining a first image pixel captured using a first sensing element in the image sensor;

obtaining a second image pixel captured, after passage of a time delay from a time when the first image pixel is captured, using a second sensing element in the image sensor; and

determining a particular image pixel of the generated image of the scene based on at least a sum of the first image pixel and the second image pixel.

20. The method of claim 19, further comprising:

selecting the second sensing element of the image sensor based on at least distance information associated with the first image pixel and indicated by data from the LIDAR sensor.