Foveated Imaging in a Lidar System

• US 20180284234A1
• Filed: 09/25/2017
• Published: 10/04/2018
• Est. Priority Date: 03/29/2017
• Status: Abandoned Application

• Current Assignee: Luminar Technologies, Inc.
• Original Assignee: Luminar Technologies, Inc.
• Inventors: Curatu, George C.
• Application Number:
  US15/714,646
• Publication Number:
  US 20180284234A1
• Time in Patent Office: Days
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2540/21 Voice
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2552/50 Barriers
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/60 Traversable objects, e.g. speed bumps or curbs
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/06 Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/483 Details of pulse systems
• CPC Class Codes: G05D 1/0231 : using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S

Abstract

To identify the most important areas in front of a vehicle for avoiding collisions, a lidar system obtains a foveated imaging model. The foveated imaging model is generated by detecting the direction at which drivers'"'"' are facing at various points in time for several scenarios based on road conditions or upcoming maneuvers. The lidar system identifies an upcoming maneuver for the vehicle or a road condition and applies the identified maneuver or road condition to the foveated imaging model to identify a region of a field of regard at which to increase the resolution. The lidar system then increases the resolution at the identified region by increasing the pulse rate for transmitting light pulses within the identified region, filtering pixels outside of the identified region, or in any other suitable manner.

Claims

1. A method for foveated imaging in lidar systems, the method comprising:

obtaining a model for foveated imaging in a lidar system operating in a vehicle, the model indicating, for each of a plurality of road conditions or maneuvers, one or more respective regions of interest within a field of regard of the lidar system in which the lidar system is to obtain data at higher quantity and/or quality relative to other regions within the field of regard;

emitting light pulses by a light source in the lidar system at a certain pulse rate;

scanning, by a scanner in the lidar system, the field of regard of the lidar system, in accordance with a certain scan pattern at a certain scan speed, including directing the light pulses toward different points within the field of regard;

determining at least one of a road condition or an upcoming maneuver of the vehicle;

applying the road condition or the upcoming maneuver to the model to identify a region of interest within the field of regard; and

modifying at least one operational parameter of the lidar system to obtain data at higher quantity and/or quality within the identified region of interest, relative to other regions within the field of regard.

2. The method of claim 1, wherein modifying the at least one operational parameter of the lidar system includes increasing the pulse rate when the scanner scans the region of interest.

3. The method of claim 1, wherein modifying the at least one operational parameter of the lidar system includes decreasing the scan speed when the scanner scans the region of interest.

4. The method of claim 1, wherein modifying the at least one operational parameter of the lidar system includes modifying the scan pattern to increase pixel density within the region of interest.

5. The method of claim 1, further comprising, for each of the plurality of maneuvers:

receiving first indications of respective maneuvers executed by a plurality of human drivers;

receiving, from sensors measuring tracking head and/or eyeball movement of the human drivers, second indications of where the human drivers look when executing the respective maneuver; and

training the model using the received first and second indications.

6. The method of claim of claim 5, further comprising, for each of the plurality of maneuvers:

receiving, from one or more of the plurality of human drivers, voice comments pertaining to the respective maneuvers; and

training the model further using the received voice comments.

7. The method of claim 5, including training the model at a network server, and obtaining the model at a vehicle controller configured to automatically maneuver the vehicle in an autonomous driving mode.

8. The method of claim 1, wherein the model indicates, for one of the plurality of road conditions or maneuvers, an ordered set of multiple non-adjacent regions of interest within the field of regard, the method further comprising:

prioritizing the multiple non-adjacent regions of interest in accordance with the order of the ordered set when modifying the at least one operational parameter.

9. The method of claim 1, further comprising:

detecting, by a receiver in the lidar system, light from some of the light pulses scattered by one or more remote targets to generate respective pixels in a point cloud.

10. The method of claim 9, further comprising:

filtering the point cloud according to the model to decrease the optical resolution of regions of pixels within the point cloud that are outside of the region of interest.

11. The method of claim 1, wherein the determined upcoming maneuver is a right turn and the identified region of interest is between −

60 degrees horizontal and −

30 degrees horizontal with respect to a forward-facing direction of the vehicle.

12. A lidar system comprising:

a light source configured to emit light pulses at a certain pulse rate;

a scanner configured to scan a field of regard of the lidar system in accordance with a certain scan pattern at a certain scan speed;

one or more controllers configured to;

obtain a model for foveated imaging in the lidar system operating within a vehicle, the model indicating, for each of a plurality of road conditions or maneuvers, one or more respective regions of interest within a field of regard of the lidar system in which the lidar system is to increase optical resolution,

determine at least one of a road condition or an upcoming maneuver of the vehicle,

apply the road condition or the upcoming maneuver to the model to identify a region of interest within the field of regard, and

transmit a control signal to the light source to increase the pulse rate for one or more of the light pulses that are directed within the identified region of interest; and

a detector configured to detect light from some of the light pulses scattered by one or more remote targets to generate respective pixels in a point cloud.

13. The lidar system of claim 12, wherein to transmit the control signal to the light source to increase the pulse rate for the one or more light pulses that are directed within the identified region of the field of regard, the controller is configured to:

receive an indication of a scan angle of the scanner;

determine that the scan angle is within the identified region of the field of regard; and

transmit a control signal to the light source to emit the one or more light pulses corresponding to a range of scan angles that includes the scan angle at the increased pulse rate.

14. The lidar system of claim 13, wherein the scan angle is a first scan angle and the controller is further configured to:

receive an indication of a second scan angle of the scanner after the scanner directs the light pulses at the first scan angle;

determine that the second scan angle is outside of the identified region of the field of regard; and

transmit a control signal to the light source to emit subsequent light pulses corresponding to a second range of scan angles that includes the second scan angle at a default pulse rate.

15. The lidar system of claim 12, wherein the controller is further configured to:

filter the point cloud according to the model to decrease the optical resolution of a region of pixels within the point cloud that are outside of the identified region of the field of regard.

16. The lidar system of claim 12, wherein the determined upcoming maneuver is a right turn and the identified region of the field of regard at which to increase optical resolution is between −

60 degrees horizontal and −

30 degrees horizontal with respect to a forward-facing direction of the vehicle

17. The lidar system of claim 12, wherein the model is obtained from a remote server that receives driver data from a plurality of client computing devices.

18. An autonomous vehicle comprising:

one or more components configured to maneuver the autonomous vehicle;

a lidar system configured to detect objects within a threshold distance of the autonomous vehicle, the lidar system including;

a light source configured to emit light pulses at a certain pulse rate,

a scanner configured to scan a field of regard of the lidar system according to a certain scan pattern and at a certain scan speed, and

a detector configured to detect light from some of the light pulses scattered by one or more remote targets to generate respective pixels in a point cloud; and

a vehicle controller configured to;

provide control signals to the one or more components to maneuver the autonomous vehicle in accordance with data from the lidar system,

obtain a model for foveated imaging in the lidar system, the model indicating, for each of a plurality of road conditions or maneuvers, one or more respective regions of interest within a field of regard of the lidar system,

determine at least one of a road condition or an upcoming maneuver, and

cause the lidar system to obtain data at higher quantity and/or quality within the one or more identified region of interest, relative to other regions within the field of regard.

19. The autonomous vehicle of claim 18, wherein the lidar system is configured to modify at least one of (i) a pulse rate, (ii) a scan speed, or (iii) a scan pattern to obtain the data at higher quantity and/or quality within the one or more identified region of interest.

20. The autonomous vehicle of claim 18, wherein the lidar controller is configured to:

receive an indication of a current scan angle of the scanner;

determine that the scan angle is within the identified region of interest; and

transmit a control signal to the light source to emit the one or more light pulses corresponding to a range of scan angles that includes the scan angle at an increased pulse rate.

21. The autonomous vehicle of claim 20, wherein the scan angle is a first scan angle and the lidar controller is further configured to:

receive an indication of a second scan angle of the scanner after the scanner directs the light pulses at the first scan angle;

determine that the second scan angle is outside of the identified region of the field of regard; and

transmit a control signal to the light source to emit subsequent light pulses corresponding to a second range of scan angles that includes the second scan angle at a default pulse rate.

22. The autonomous vehicle of claim 18, wherein the model is generated by placing one or more sensors on one or more drivers of one or more user operated vehicles to track areas the one or more drivers are facing during at least one of a particular road condition or a particular maneuver.