OBJECT DETECTION BASED ON LIDAR INTENSITY

• US 20200041652A1
• Filed: 10/22/2018
• Published: 02/06/2020
• Est. Priority Date: 07/31/2018
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Inventors: Jain, Shantanu, Yang, Gehua
• Granted Patent: US 10,598,791 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

Aspects of the present disclosure involve systems, methods, and devices for determining reflectance properties of objects based on Lidar intensity values. A system includes one or more processors of a machine and a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising accessing an incoming data point output by a Lidar unit during operation of a vehicle. The operations may further include inferring, using a reflectance inference model, a reflectance value of an object based on the incoming data point. The reflectance inference model comprises a mapping of previously collected data points to a coordinate system using associated range values and raw intensity values. The operations may further include determining one or more characteristics of the object based on the inferred reflectance value.

Claims

1. An autonomous vehicle (AV) system comprising:

one or more processors of a machine; and

a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising;

accessing an incoming data point output by a light detection and ranging (Lidar) unit during operation of the AV system;

inferring, using a reflectance inference model, a reflectance value of an object based on the incoming data point, the reflectance inference model comprising a mapping of a set of previously collected data points to a coordinate system using range and raw intensity values of the previously collected data points as coordinates, the reflectance value being interpolated from at least two previously collected data points of the set of previously collected data points mapped to the coordinate system, each of the two previously collected data points corresponding to one of multiple known reflectance values; and

determining one or more characteristics of the object based on the inferred reflectance value.

2. The AV system of claim 1, wherein the inferring of the reflectance value of the object comprises:

mapping the incoming data point to the coordinate system;

identifying two or more nearest neighbors of the incoming data point mapped to the coordinate system, the two or more nearest neighbors including the at least two previously collected data points; and

interpolating the reflectance value from the two or more nearest neighbors.

3. The AV system of claim 1, wherein:

the reflectance inference model comprises a look-up table comprising a plurality of reflectance values interpolated from the set of previously collected data points mapped to the coordinate system, each reflectance value corresponding to a particular combination of range and raw intensity values; and

the inferring of the reflectance value of the object comprises;

accessing the look-up table; and

determining the reflectance value from the look-up table based on a raw intensity value and a range value of the incoming data point.

4. The AV system of claim 1, wherein:

the Lidar unit comprises a plurality of channels, each channel being capable of operating at a plurality of power levels;

the reflectance inference model comprises multiple mappings of the previously collected data points to the coordinate system, each of the mappings corresponding to a particular power level of a particular channel of the Lidar unit; and

the inferring of the reflectance value of the object comprises selecting the mapping of the set of previously collected data points from the multiple mappings based on a channel of the Lidar unit that output the incoming data point and a power level of the channel.

5. The AV system of claim 1, wherein the operations further comprise generating state data that describes the object, the state data comprising at least the inferred reflectance value.

6. The AV system of claim 1, wherein the operations further comprise:

generating a reflectance map that includes an indication of the inferred reflectance value of the object; and

performing vehicle localization based on the reflectance map.

7. The AV system of claim 1, wherein the operations further comprise:

collecting a data set comprising data points output by the Lidar unit, each of the data points corresponding to one of multiple targets, each target having one of the multiple known reflectance values, the data set corresponding to the set of previously collected data points; and

generating the reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the set of previously collected data points to the coordinate system.

8. A light detection and ranging (Lidar) calibration system comprising:

one or more processors of a machine; and

a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising;

collecting a data set comprising a plurality of data points output by Lidar unit, each of the data points corresponding to one of multiple targets, each target having a known reflectance value; and

generating a reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the plurality of data points to a coordinate system using range and raw intensity values of each data point as coordinates, the reflectance inference model operable to infer reflectance values from incoming data points output by the Lidar unit.

9. The Lidar calibration system of claim 8, wherein:

the generating of the reflectance inference model further comprises generating a look-up table comprising a plurality of reflectance values interpolated from the plurality of data points mapped to the coordinate system,

each reflectance value corresponding to a particular combination of range and raw intensity values.

10. The Lidar calibration system of claim 9, wherein the operations further comprise:

compressing the look-up table to generate a compressed look-up table; and

storing the compressed look-up table in a memory device of a vehicle computing system.

11. The Lidar calibration system of claim 9, wherein:

the collecting of the data set includes collecting data points output by each channel of the Lidar unit; and

the generating of the reflectance inference model includes generating, for each channel, a mapping of data points to the coordinate system.

12. The Lidar calibration system of claim 11, wherein:

each channel of the Lidar unit is capable of operating at a plurality of power levels;

the collecting of the data points output by each channel of the Lidar unit includes collecting data points output by each channel at each power level; and

the generating of the reflectance inference model includes generating multiple mappings of the data points to the coordinate system, each mapping corresponding to a particular power level of a particular channel.

13. A method comprising:

collecting a data set comprising a plurality of data points output by a light detection and ranging (Lidar) unit of an autonomous vehicle, each of the data points corresponding to one of multiple targets, each target having a known reflectance value;

generating a reflectance inference model for the Lidar unit based on the data set, the generating of the reflectance inference model comprising mapping the plurality of data points to a coordinate system using range and raw intensity values of each data point as coordinates;

accessing, by a vehicle computing system of the autonomous vehicle, an incoming data point output by the Lidar unit during operation of the autonomous vehicle;

inferring, using the reflectance inference model, a reflectance value of an object based on the incoming data point, the reflectance value being interpolated from at least two data points mapped to the coordinate system; and

determining, by the vehicle computing system, a motion plan for the autonomous vehicle based on the inferred reflectance value, the motion plan of the autonomous vehicle being used to control motion of the autonomous vehicle.

14. The method of claim 13, wherein the inferring of the reflectance value of the object comprises:

mapping the incoming data point to the coordinate system;

identifying two or more nearest neighbors of the incoming data point mapped to the coordinate system, the two or more nearest neighbors including the at least two data points; and

interpolating the reflectance value from the two or more nearest neighbors.

15. The method of claim 13, wherein:

the generating of the reflectance inference model further comprises;

generating a look-up table comprising a plurality of reflectance values interpolated from the plurality of data points mapped to the coordinate system, each reflectance value corresponding to a particular combination of range and raw intensity values; and

the inferring of the reflectance value of the object comprises;

accessing, by the vehicle computing system, the look-up table from a memory device of the vehicle computing system; and

determining, by the vehicle computing system, the reflectance value from the look-up table based on a range value and a raw intensity value of the incoming data point.

16. The method of claim 13, wherein:

the collecting of the data set includes collecting data points output by each channel of the Lidar unit; and

the generating of the reflectance inference model includes generating, for each channel, a mapping of data points to the coordinate system.

17. The method of claim 16, wherein:

each channel of the Lidar unit is capable of operating at a plurality of power levels;

the collecting of the data points output by each channel of the Lidar unit includes collecting data points output by each channel at each power level; and

the generating of the reflectance inference model includes generating multiple mappings of the data points to the coordinate system, each mapping corresponding to a particular power level of a particular channel.

18. The method of claim 17, wherein:

the inferring of the reflectance value of the object includes selecting a mapping from the multiple mappings based on a channel of the Lidar unit that output the incoming data point and a power level of the channel.

19. The method of claim 13, wherein the collecting of the data set comprises:

arranging a plurality of targets, each of the targets having a known reflectance; and

collecting at least one data point corresponding to each target.

20. The method of claim 19, wherein the operations further comprise:

mounting the Lidar unit on a gimbal; and

using the gimbal to position the Lidar unit at multiple angles; and

wherein the collecting of the at least one data point corresponding to each target includes collecting at least one data point corresponding to each target at each angle of the multiple angles.