Multi-Range Three-Dimensional Imaging Systems

• US 20170307759A1
• Filed: 10/07/2016
• Published: 10/26/2017
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: Cepton Technologies, Inc.
• Original Assignee: Cepton Technologies, Inc.
• Inventors: Pei, Jun, McCord, Mark, Ye, June
• Granted Patent: US 10,481,266 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4811 common to transmitter and receiver
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/4865 Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak peak detection in noise, signal conditioning G01S7/487
• CPC Class Codes: G01S 7/4868 Controlling received signal intensity or exposure of sensor
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects

Abstract

A three-dimensional imaging system includes a lidar sensor having a first optical axis oriented at a first angle toward a scene and configured to determine a three-dimensional image of one or more first objects in the scene, and an optical three-dimensional sensor having a second optical axis oriented at a second angle toward the scene and configured to construct a three-dimensional image of one or more second objects in the scene. The first three-dimensional sensor is characterized by a first angular field of view. The second three-dimensional sensor is characterized by a second angular field of view different from the first angular field of view.

Claims

1. A system for performing three-dimensional imaging of a scene, the system comprising:

a first lidar sensor having a first optical axis oriented at a first angle toward the scene, the first lidar sensor including;

a first laser source configured to emit a first plurality of laser pulses;

a first emission optical element configured to collimate and direct the first plurality of laser pulses at a first plurality of corresponding incidence angles with respect to the first optical axis toward one or more first objects in the scene, wherein a portion of each of the first plurality of laser pulses is reflected off of the one or more first objects;

a first receiving optical element configured to receive and focus the portion of each of the first plurality of laser pulses reflected off of the one or more first objects; and

a first photodetector configured to receive and detect the portion of each of the first plurality of laser pulses focused by the first receiving optical element;

a second lidar sensor having a second optical axis oriented at a second angle toward the scene, the second lidar sensor comprising;

a second laser source configured to emit a second plurality of laser pulses;

a second emission optical element configured to collimate and direct the second plurality of laser pulses at a second plurality of corresponding incidence angles with respect to the second optical axis toward one or more second objects in the scene, wherein a portion of each of the second plurality of laser pulses is reflected off of the one or more second objects;

a second receiving optical element configured to receive and focus the portion of each of the second plurality of laser pulses reflected off of the one or more second objects; and

a second photodetector configured to receive and detect the portion of each of the second plurality of laser pulses focused by the second receiving optical element; and

a processor including one or more processing units coupled to the first lidar sensor and the second lidar sensor, the processor configured to;

determine a time of flight for each of the first plurality of laser pulses and each of the second plurality of laser pulses from emission to detection; and

construct a three dimensional image of the scene based on the determined time of flight for each of the first plurality of laser pulses and each of the second plurality of laser pulses, the first angle of the first optical axis, the first plurality of incidence angles, the second angle of the second optical axis, and the second plurality of incidence angles.

2. The system of claim 1 wherein the first lidar sensor is characterized by a first angular field of view, and the second lidar sensor is characterized by a second angular field of view, the second angular field of view being greater than the first angular field view.

3. The system of claim 2 wherein the first emission optical element has a first focal length, and the second emission optical element has a second focal length less than the first focal length.

4. The system of claim 2 wherein the first lidar sensor has a first sensing range, and the second lidar sensor has a second sensing range shorter than the first sensing range.

5. The system of claim 2 wherein the second angle of the second optical axis is different from the first angle of the first optical axis, and the first angular field of view does not overlap with the second angular field of view.

6. The system of claim 2 wherein the second angle of the second optical axis is different from the first angle of the first optical axis, and the first angular field of view partially overlaps with the second angular field of view.

7. The system of claim 2 wherein the second angular field of view encompasses the first angular field of view.

8. The system of claim 1 wherein:

the first laser source is configured to be translated through a first plurality of emission locations in a focal plane of the first emission optical element, wherein each of the first plurality of laser pulses is emitted at a respective one of the first plurality of emission locations, resulting in a corresponding one of the first plurality of incidence angles after being collimated by the first emission optical element;

the first photodetector is configured to be translated through a first plurality of corresponding detection locations in a focal plane of the first receiving optical element, wherein translation of the first photodetector is performed synchronously with translation of the first laser source such that a location of the first photodetector is always conjugate with a location of the first laser source at any given time;

the second laser source is configured to be translated through a second plurality of emission locations in a focal plane of the second emission optical element, wherein each of the second plurality of laser pulses is emitted at a respective one of the second plurality of emission locations, resulting in a corresponding one of the second plurality of incidence angles after being collimated by the second emission optical element; and

the second photodetector is configured to be translated through a second plurality of corresponding detection locations in a focal plane of the second receiving optical element, wherein translation of the second photodetector is performed synchronously with translation of the second laser source such that a location of the second photodetector is always conjugate with a location of the second laser source at any given time.

9. A three-dimensional imaging system comprising:

a first three-dimensional sensor having a first optical axis oriented at a first angle toward a scene, the first three-dimensional sensor including;

a laser source configured to emit a plurality of laser pulses;

an emission optical element configured to collimate and direct the plurality of laser pulses at a plurality of corresponding incidence angles with respect to the first optical axis toward one or more first objects in the scene, wherein a portion of each of the plurality of laser pulses is reflected off of the one or more first objects;

a receiving optical element configured to receive and focus the portion of each of the plurality of laser pulses reflected off of the one or more first objects;

a photodetector configured to receive and detect the portion of each of the plurality of laser pulses focused by the receiving optical element; and

a processor including one or more processing units coupled to the laser source and the photodetector and configured to;

determine a time of flight for each of the plurality of laser pulses; and

construct a three dimensional image of the one or more first objects based on the determined time of flight for each of the plurality of laser pulses, the first angle of the first optical axis, and the plurality of incidence angles;

wherein the first three-dimensional sensor is characterized by a first angular field of view; and

a second three-dimensional sensor configured to construct a three-dimensional image of one or more second objects in the scene, the second three-dimensional sensor characterized by a second angular field of view;

wherein the processor is coupled to the second three-dimensional sensor and configured to construct a three-dimensional image of the scene based on the three-dimensional image of the one or more first objects and the three-dimensional image of the one or more second objects.

10. The three-dimensional imaging system of claim 9 wherein:

the laser source is configured to be translated through a plurality of emission locations in a focal plane of the emission optical element, wherein each of the plurality of laser pulses is emitted at a respective one of the plurality of emission locations, resulting in a corresponding one of the plurality of incidence angles after being collimated by the emission optical element; and

the photodetector is configured to be translated through a plurality of corresponding detection locations in a focal plane of the receiving optical element, wherein translation of the photodetector is performed synchronously with translation of the laser source such that a location of the photodetector is always conjugate with a location of the laser source at any given time.

11. The three-dimensional imaging system of claim 9 wherein the second three-dimensional sensor comprises an optical three-dimensional sensor having a second optical axis oriented at a second angle toward the scene, the second angle being different from the first angle, and wherein the second angular field of view at least partially overlaps with the first angular field of view.

12. The three-dimensional imaging system of claim 11 wherein the first three-dimensional sensor and the second three-dimensional sensor are configured to construct a first three-dimensional image and a second three-dimensional image, respectively, of a third object positioned within an angular field of view where the first angular field of view and the second angular field of view overlap, and wherein the processor is configured to calibrate distance information determined by the second three-dimensional sensor based on the first three-dimensional image and the second three-dimensional image of the third object.

13. The three-dimensional imaging system of claim 11 wherein the first three-dimensional sensor and the second three-dimensional sensor are configured to construct a first three-dimensional image and a second three-dimensional image, respectively, of a third object positioned within an angular field of view where the first angular field of view and the second angular field of view overlap, and wherein the processor is configured to calibrate angular information determined by the second three-dimensional sensor based on the first three-dimensional image and the second three-dimensional image of the third object.

14. The three-dimensional imaging system of claim 13 wherein the angular information includes angular information in a first plane.

15. The three-dimensional imaging system of claim 14 wherein the angular information further includes angular information in a second plane orthogonal to the first plane.

16. The three-dimensional imaging system of claim 9 wherein the second three-dimensional sensor comprises a lidar sensor.

17. The three-dimensional imaging system of claim 9 wherein the second three-dimensional sensor comprises an optical three-dimensional sensor having a second optical axis oriented at the first angle toward the scene, and wherein the second angular field of view encompasses the first angular field of view.

18. A method of three-dimensional sensing for an autonomous vehicle, the method comprising:

sensing one or more first objects in a forward direction using a lidar sensor including a laser source, a photodetector, an emission optical element, a receiving optical element, and a processor including one or more processing units coupled to the laser source and the photodetector, by;

emitting, using the laser source, a plurality of laser pulses;

collimating and directing, using the emission optical element, the plurality of laser pulses at a plurality of corresponding incidence angles with respect to the forward direction toward the one or more first objects, wherein a portion of each of the plurality of laser pulses is reflected off of the one or more first objects;

receiving and focusing, using the receiving optical element, the portion of each of the plurality of laser pulses reflected off of the one or more first objects;

detecting, using the photodetector, the portion of each of the plurality of laser pulses focused by the receiving optical element;

determining, using the processor, a time of flight for each of the plurality of laser pulses from emission to detection; and

constructing, using the processor, a three dimensional image of the one or more first objects based on the determined time of flight for each of the plurality of laser pulses and the plurality of incidence angles;

sensing one or more second objects in a direction to the left or right using an optical three-dimensional sensor to obtain a three-dimensional image of the one or more second objects; and

combining, using the processor, the three dimensional image of the one or more first objects and the three-dimensional image of the one or more second objects.

19. The method of claim 18 wherein the lidar sensor has a first angular field of view, and the optical three-dimensional sensor has a second angular field of view wider than the first angular field of view.

20. The method of claim 19 wherein the lidar sensor has a first sensing range, and the optical three-dimensional sensor has a second sensing range less than the first sensing range.

21. The method of claim 19 wherein the second angular field of view at least partially overlaps with the first angular field of view.

22. The method of claim 21 further comprising:

sensing a third object using the lidar sensor to obtain a first three-dimensional image of the third object, the third object positioned within an angular field of view where the first angular field of view and the second angular field of view overlap;

sensing the third object using the optical three-dimensional sensor to obtain a second three-dimensional image of the third object; and

calibrating, using the processor, distance information determined by the optical three-dimensional sensor based on the first three-dimensional image and the second three-dimensional image of the third object.

23. The method of claim 21 further comprising:

sensing a third object using the lidar sensor to obtain a first three-dimensional image of the third object, the third object positioned within an angular field of view where the first angular field of view and the second angular field of view overlap;

sensing the third object using the optical three-dimensional sensor to obtain a second three-dimensional image of the third object; and

calibrating, using the processor, angular information determined by the optical three-dimensional sensor based on the first three-dimensional image and the second three-dimensional image of the third object.

24. The method of claim 21 further comprising:

sensing a third object using the lidar sensor to obtain a first three-dimensional image of the third object, the third object positioned within an angular field of view where the first angular field of view and the second angular field of view overlap;

sensing the third object using the optical three-dimensional sensor to obtain a second three-dimensional image of the third object; and

calibrating, using the processor, angular information determined by the lidar sensor based on the first three-dimensional image and the second three-dimensional image of the third object.