DETECTION AND RECALIBRATION FOR A CAMERA SYSTEM USING LIDAR DATA

• US 20180096493A1
• Filed: 12/04/2017
• Published: 04/05/2018
• Est. Priority Date: 12/04/2017
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: BIER, SOLOMON
• Granted Patent: US 10,430,970 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Systems and method are provided for calibrating a camera system of an autonomous vehicle. In one embodiment, a method includes: identifying, by the processor, a planar object from sensor data generated by a sensor of the autonomous vehicle while the autonomous vehicle is operating; identifying, by the processor, a pattern of the planar object; selecting, by the processor, planar points from the pattern of the planar object; and calibrating, by the processor, the camera system, while the autonomous vehicle is operating, based on the planar points.

Claims

1. A method of calibrating a camera system of an autonomous vehicle, comprising:

identifying, by the processor, a planar object from sensor data generated by a sensor of the autonomous vehicle while the autonomous vehicle is operating;

identifying, by the processor, a pattern of the planar object;

selecting, by the processor, planar points from the pattern of the planar object; and

calibrating, by the processor, the camera system, while the autonomous vehicle is operating, based on the planar points.

2. The method of claim 1, wherein the sensor data includes lidar data.

3. The method of claim 2, wherein the identifying the pattern is based on gradient values computed from points of the lidar data.

4. The method of claim 1, wherein the sensor data includes image data.

5. The method of claim 4, further comprising:

computing gradient values for pixels in the image data; and

selecting the planar points based on gradient values in the image data.

6. The method of claim 1, wherein the identifying the pattern is based on gradient values computed from points of lidar data and image data.

7. The method of claim 1, further comprising filling holes of the pattern based on at least one data enhancing technique.

8. The method of claim 7, wherein the data enhancing technique includes anti-aliasing.

9. The method of claim 7, wherein the data enhancing technique is based on a predefined three dimensional map.

10. A system for calibrating a camera system of an autonomous vehicle, comprising:

a first non-transitory module that, by a processor, identifies a planar object from sensor data generated by a sensor of the autonomous vehicle while the autonomous vehicle is operating;

a second non-transitory module that, by a processor, identifies a pattern of the planar object, and selects planar points from the pattern of the planar object; and

a third non-transitory module that, by a processor, calibrates the camera system, while the autonomous vehicle is operating, based on the planar points.

11. The system of claim 10, wherein the sensor data includes lidar data.

12. The system of claim 11, wherein the second non-transitory module identifies the pattern based on gradient values computed from points of the lidar data.

13. The system of claim 10, wherein the sensor data includes image data and lidar data.

14. The system of claim 13, wherein the second non-transitory module computes gradient values for pixels in the image data, and selects the planar points based on the gradient values in the image data.

15. The system of claim 10, wherein the second non-transitory module identifies the pattern is based on gradient values computed from points of lidar data.

16. The system of claim 10, the second non-transitory module fills holes of the pattern based on at least one data enhancing technique.

17. The system of claim 16, wherein the data enhancing technique includes anti-aliasing.

18. The system of claim 16, wherein the data enhancing technique is based on a predefined three dimensional map of the world.

19. A vehicle, comprising:

a lidar system;

a camera system; and

a controller configured to, by a processor, identify a planar object from sensor data generated by the lidar system of the autonomous vehicle while the autonomous vehicle is operating, identify a pattern of the planar object, select planar points from the pattern of the planar object, and calibrate the camera system, while the autonomous vehicle is operating, based on the planar points.

20. The vehicle of claim 19, wherein the controller is further configured to identify a pattern of the planar object based on gradient values computed from the sensor data.