Calibration for autonomous vehicle operation

• US 9,916,703 B2
• Filed: 11/04/2015
• Issued: 03/13/2018
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Levinson, Jesse Sol, Douillard, Bertrand Robert, Sibley, Gabriel Thurston
• Primary Examiner(s): Mott, Genna M
• Application Number:
  US14/756,996
• Publication Number:
  US 20170124781A1
• Time in Patent Office: 860 Days
• Field of Search: 702 85-107, 701 297, 701 302, 701 305- 307, 701 316, 701 331
• CPC Class Codes: B60Q 1/26 the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
• CPC Class Codes: B60Q 1/28 for indicating front of vehicle
• CPC Class Codes: B60Q 1/30 for indicating rear of vehicle, e.g. by means of reflecting surfaces
• CPC Class Codes: B60Q 1/507 specific to autonomous vehicles
• CPC Class Codes: B60Q 1/508 specific to vehicles driving in fleets or convoys
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 15/872 Combination of several systems for attitude determination using inertial means G01C9/00, control of attitude G05D1/49
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/875 for determining attitude
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/40 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G01S 7/52004 Means for monitoring or calibrating short-range imaging G01S7/5205
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0268 : using internal positioning means
• CPC Class Codes: G05D 1/027 : comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0272 : comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 : using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0291 : Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127
• CPC Class Codes: G06Q 10/06 : Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/08 : Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G07C 5/0808 : Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G07C 5/0841 : Registering performance data recording measured values G01D; information storage G11B
• CPC Class Codes: G07C 5/0866 : the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/056 : with provision for distinguishing direction of travel
• CPC Class Codes: G08G 1/096816 : where the complete route is transmitted to the vehicle at once
• CPC Class Codes: G08G 1/096838 : where the user preferences are taken into account or the user selects one route out of a plurality
• CPC Class Codes: G08G 1/096844 : where the complete route is dynamically recomputed based on new data
• CPC Class Codes: G08G 1/20 : Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles
• CPC Class Codes: G08G 1/202 : Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: G08G 1/207 : with respect to certain areas, e.g. forbidden or allowed areas with possible alerting when inside or outside boundaries

Abstract

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include receiving data associated with a sensor measurement of a perceived object, determining a label associated with the perceived object based on an initial calibration, retrieving log file data associated with the label, determining a calibration parameter associated with the sensor measurement based on the retrieved log file data, and storing the calibration parameter in association with a sensor associated with the sensor measurement. Sensors may be calibrated on the fly while the autonomous vehicle is in operation using one or more other sensors and/or fused data from multiple types of sensors.

Claims

1. A method comprising:

receiving data at an autonomous vehicle system comprising a plurality of sensors, the plurality of sensors including an inertial measurement unit (IMU) sensor, an odometry sensor, and a multi-beam LIDAR sensor, and the data representing measurements made using the plurality of sensors;

determining a distance traveled by the autonomous vehicle system based on one or more sensor measurements of the odometry sensor;

determining a velocity of the autonomous vehicle system based on one or more sensor measurements of the IMU sensor;

identifying an abnormal sensor measurement, made using the multi-beam LIDAR sensor, based at least in part on the data;

generating an initial calibration parameter, associated with the multi-beam LIDAR sensor, based at least in part on the abnormal sensor measurement;

generating an expected sensor measurement, associated with the multi-beam LIDAR sensor, based on the data;

modifying the initial calibration parameter based at least in part on the expected sensor measurement, wherein modifying the initial calibration parameter includes generating a modified calibration parameter associated with the multi-beam LIDAR sensor; and

calibrating the multi-beam LIDAR sensor based at least in part on the modified calibration parameter, the distance traveled by the autonomous vehicle system, and the velocity of the autonomous vehicle system.

2. The method of claim 1, wherein the plurality of sensors includes an array of LIDAR sensors arranged at one end of the autonomous vehicle system,

the array including the multi-beam LIDAR sensor.

3. The method of claim 2, further comprising:

determining an extrinsic calibration of the multi-beam LIDAR sensor based on a physical location of the multi-beam LIDAR sensor relative to the autonomous vehicle system.

4. The method of claim 2, further comprising:

determining an intrinsic calibration of the multi-beam LIDAR sensor based on data associated with at least one additional LIDAR sensor of the array disposed adjacent to the multi-beam LIDAR sensor.

5. The method of claim 1, further comprising:

retrieving one or more log files, including information indicative of past sensor measurements, based on the abnormal sensor measurement; and

generating the expected sensor measurement based, in part, on the information indicative of past sensor measurements.

6. The method of claim 1, wherein the expected sensor measurement is further based at least in part on a teleoperation interface selection made via a teleoperator system.

7. The method of claim 1, wherein the expected sensor measurement is further based at least in part on a map tile associated with a pose of the autonomous vehicle system at a time of the abnormal sensor measurement.

8. The method of claim 1, wherein the initial calibration parameter is generated based at least in part on a first probabilistic model, the method further comprising:

generating a second probabilistic model; and

generating an additional expected sensor measurement, associated with the multi-beam LIDAR sensor, using the second probabilistic model, wherein the additional expected sensor measurement is generated based on a measurement made using at least one additional sensor of the plurality of sensors.

9. A method comprising:

receiving, at an autonomous vehicle system comprising a plurality of sensors, a plurality of sensor measurements associated with an environmental feature, the plurality of sensors comprising an inertial measurement unit (IMU) sensor, an odometry sensor, and a multi-beam LIDAR sensor;

determining a distance traveled by the autonomous vehicle system based on one or more sensor measurements of the odometry sensor;

determining a velocity of the autonomous vehicle system based on one or more sensor measurements of the IMU sensor;

identifying an abnormal sensor measurement of the plurality of sensor measurements made using the multi-beam LIDAR sensor, wherein the abnormal sensor measurement is incongruent with one or more other measurements of the plurality of sensor measurements made using the multi-beam LIDAR sensor;

determining a plurality of calibration parameters associated with the multi-beam LIDAR sensor, wherein at least one parameter of the plurality of calibration parameters is determined based on the abnormal sensor measurement and a statistical computation associated with the environmental feature;

determining an expectation model associated with the multi-beam LIDAR sensor, wherein the expectation model is configured to output one or more expected sensor measurements;

generating an expected sensor measurement with the expectation model and using at least one measurement of the plurality of sensor measurements made using the multi-beam LIDAR sensor;

modifying the plurality of calibration parameters based at least in part on the expected sensor measurement, wherein modifying the plurality of calibration parameter includes generating a modified calibration parameter associated with the multi-beam LIDAR sensor; and

calibrating the multi-beam LIDAR sensor based at least in part on the modified calibration parameter, the distance traveled by the autonomous vehicle system, and the velocity of the autonomous vehicle system.

10. The method of claim 9, further comprising:

comparing the plurality of sensor measurements to a predetermined sensor measurement range; and

identifying the abnormal sensor measurement based at least in part on the comparing.

11. The method of claim 9, wherein the abnormal sensor measurement is incongruent with the one or more other measurements of the plurality of sensor measurements based on the abnormal sensor measurement not matching the one or more other measurements of the plurality of sensor measurements.

12. The method of claim 9, wherein the statistical computation associated with the environmental feature comprises an average of the plurality of sensor measurements associated with the environmental feature.

13. The method of claim 9, wherein the statistical computation associated with the environmental feature comprises a median of the plurality of sensor measurements associated with the environmental feature.

14. The method of claim 9, wherein modifying the plurality of calibration parameters comprises:

determining a heuristic rule to identify a subset of calibration parameters based on the expectation model and the plurality of calibration parameters;

identifying the subset of calibration parameters using the heuristic rule; and

replacing the plurality of calibration parameters with the subset of calibration parameters.

15. A method comprising:

receiving, at an autonomous vehicle system comprising a plurality of sensors, data associated with a measurement made by a multi-beam LIDAR sensor of the plurality of sensors, wherein the measurement indicates detection of an object by the multi-beam LIDAR sensor, and the plurality of sensors further comprises at least one RADAR sensor;

determining a current physical location of the multi-beam LIDAR sensor, relative to the autonomous vehicle system, using the at least one RADAR sensor;

comparing the current physical location to a stored indication of an initial position of the multi-beam LIDAR sensor;

determining a labeled data point associated with the object, wherein the labeled data point is determined based at least in part on an initial calibration of the multi-beam LIDAR sensor;

retrieving log file data associated with the labeled data point, the log file data comprising one or more contradictory sensor measurements from the multi-beam LIDAR sensor;

determining, based at least in part on the one or more contradictory sensor measurements, an expected sensor measurement of the multi-beam LIDAR sensor;

determining a calibration parameter associated with the multi-beam LIDAR sensor based at least in part on the retrieved log file data and the expected sensor measurement;

associating the calibration parameter with information, identifying the multi-beam LIDAR sensor, in a memory associated with the autonomous vehicle system; and

calibrating the multi-beam LIDAR sensor based at least in part on the calibration parameter and on the comparing.

16. The method of claim 15, wherein:

the calibration parameter is determined based at least in part on at least one generative probabilistic model,

the measurement is made by the multi-beam LIDAR sensor at a particular time, and

the one or more contradictory sensor measurements are made concurrently with the measurement of the object.

17. The method of claim 15, wherein:

the calibration parameter is determined based at least in part on a generative probabilistic model using a plurality of fused sensor measurements from one or more heterogeneous sensors,

the measurement is made by the multi-beam LIDAR sensor at a particular time, and

each measurement of the plurality of fused sensor measurements is made concurrently with the measurement of the object.

18. The method of claim 15, wherein:

the calibration parameter is determined based at least in part on a generative probabilistic model using map tile data,

the map tile data comprises information indicative of past measurements of the object, and

the past measurements were made using one or more heterogeneous sensors.