CALIBRATION OF MULTI-SENSOR SYSTEM

• US 20100235129A1
• Filed: 03/10/2009
• Published: 09/16/2010
• Est. Priority Date: 03/10/2009
• Status: Abandoned Application

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Sharma, Manuj, Eswara, Lalitha, Rao, Shrikant
• Application Number:
  US12/400,980
• Publication Number:
  US 20100235129A1
• Time in Patent Office: Days
• US Class Current: 702/97
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 2201/02 Control of position of land vehicles

Abstract

A method for preprocessing sensor data applicable to sensor fusion for one or more sensors mounted on a vehicle is presented. The method comprises obtaining sensor data relating to common obstacles between the vision sensors and a lidar sensor, calculating range and azimuth values for the common obstacles from the lidar sensor data, and calculating range and azimuth values for the common obstacles from the vision sensor data. Then the method correlates the lidar sensor data pertaining to the common obstacles with the vision sensors pixels, formulates translations between the range values of the common obstacles and the one or more sensor tilt parameters, and performs recursive least squares to estimate an estimated sensor tilt for the vision sensors that can reduce range errors in the vision sensors.

Claims

1. A method for calibrating sensors mounted on a vehicle, comprising:

obtaining sensor data relating to one or more common obstacles between the one or more vision sensors and a lidar sensor;

correlating the lidar sensor data pertaining to the one or more common obstacles with the one or more vision sensors pixels;

calculating range and azimuth values for the one or more common obstacles from the lidar sensor data;

calculating range and azimuth values for the one or more common obstacles from the one or more vision sensor data;

formulating translations between the range values of the one or more common obstacles and the one or more sensor tilt parameters; and

perform recursive least squares to estimate an estimated sensor tilt for the one or more vision sensors that can reduce range errors in the vision sensors.

2. The method of claim 1, further comprising:

calculating the range and azimuth information for the one or more common obstacles using the estimated sensor tilt;

performing sensor fusion on the one or more common obstacles; and

announcing to a processor the one or more fused obstacles and their range and azimuth information.

3. The method of claim 1, further comprising:

initializing alignment and mounting of the lidar sensor and the one or more vision sensors on the vehicle.

4. The method of claim 1, further comprising:

performing on-line preprocessing of sensor measurements for sensor fusion.

5. The method of claim 1, wherein the method is performed off-line and further comprises:

repeating the method from time to time to ensure correctness of the estimated calibration parameters.

6. The method of claim 1, further comprising:

calibrating the one or more vision sensors using the estimated sensor tilt.

7. The method of claim 1, wherein obtaining sensor data relating to one or more common obstacles between the one or more vision sensors and a lidar sensor further comprises:

obtaining sensor data relating to one or more vision sensor segmented bounding boxes corresponding to the one or more common obstacles.

8. The method of claim 1, wherein the one or more vision sensors comprises:

at least a monocular color camera.

9. An autonomous vehicle navigation system, comprising:

one or more vision sensors mounted on a vehicle;

a lidar sensor mounted on the vehicle;

a processing unit coupled to the one or more vision sensors and the lidar sensor operable to;

receive data pertaining to the initial alignment and mounting of the lidar sensor and the one or more vision sensors on the vehicle;

receive data relating to one or more common obstacles between the one or more vision sensors and the lidar sensor;

calculate range and azimuth values for the one or more common obstacles from the lidar sensor data;

calculate range and azimuth values for the one or more common obstacles from the one or more vision sensor data;

correlate the lidar sensor data pertaining to the one or more common obstacles with the one or more vision sensors pixels;

formulate translations between the range values of the one or more common obstacles and the one or more sensor tilt parameters; and

perform recursive least squares to estimate an estimated sensor tilt for the one or more vision sensors that can reduce range errors in the vision sensors.

10. The system of claim 9, wherein what the processing unit is operable for further comprises:

calculate the range and azimuth information for the one or more common obstacles using the estimated sensor tilt;

perform sensor fusion on the one or more common obstacles; and

announce to a display the one or more fused obstacles and their range and azimuth information.

11. The system of claim 9, wherein what the processing unit is operable for further comprises:

performing on-line preprocessing of sensor measurements for sensor fusion.

12. The system of claim 9, wherein the one or more vision sensors comprises:

at least a monocular color camera.

13. The system of claim 9, wherein the lidar sensor can scan at least 180 degrees.

14. A computer program product, comprising:

a computer readable medium having instructions stored thereon for a method of calibrating one or more sensors on a vehicle, the method comprising;

obtaining sensor data relating to one or more common obstacles between the one or more vision sensors and a lidar sensor;

calculating range and azimuth values for the one or more common obstacles from the lidar sensor data;

calculating range and azimuth values for the one or more common obstacles from the one or more vision sensor data;

correlating the lidar sensor data pertaining to the one or more common obstacles with the one or more vision sensors pixels;

formulating translations between the range values of the one or more common obstacles and the one or more sensor tilt parameters; and

perform recursive least squares to estimate an estimated sensor tilt for the one or more vision sensors that can reduce range errors in the vision sensors.

15. The computer program product of claim 14, further comprising:

calculating the range and azimuth information for the one or more common obstacles using the estimated sensor tilt;

performing sensor fusion on the one or more common obstacles; and

announcing to a processor the one or more fused obstacles and their range and azimuth information.

16. The computer program product of claim 14, further comprising:

initializing alignment and mounting of the lidar sensor and the one or more vision sensors on the vehicle.

17. The computer program product of claim 14, further comprising:

performing on-line preprocessing of sensor measurements for sensor fusion.

18. The computer program product of claim 14, wherein the method is performed off-line and further comprises:

repeating the method from time to time to ensure correctness of the estimated calibration parameters.

19. The computer program product of claim 14, further comprising:

calibrating the one or more vision sensors using the estimated sensor tilt.

20. The computer program product of claim 14, wherein obtaining sensor data relating to one or more common obstacles between the one or more vision sensors and a lidar sensor further comprises:

obtaining sensor data relating to one or more vision sensor segmented bounding boxes corresponding to the one or more common obstacles.