CALIBRATION VALIDATION FOR AUTONOMOUS VEHICLE OPERATIONS

• US 20180307238A1
• Filed: 04/20/2017
• Published: 10/25/2018
• Est. Priority Date: 04/20/2017
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Wisniowski, Matthias
• Granted Patent: US 10,268,203 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/93273 on the top of the vehicles
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

Systems and method are provided for controlling a vehicle. In one embodiment, a vehicle includes: an imaging device, a reference data source, one or more sensors, one or more actuators, and a controller. The controller detects a stationary condition based at least in part on the sensor output, obtains an image during the stationary condition, identifies a reference position for a reference object based on reference data for a field of view of the imaging device during the stationary condition, identifies the reference object at a second position within the image, validates a transformation associated with the imaging device based on a difference between the reference position and the second position, and thereafter autonomously operates the one or more actuators in a manner that is influenced by the validated transformation.

Claims

1. A method of controlling a vehicle, the method comprising:

detecting, by a control module onboard the vehicle, a stationary condition of the vehicle based at least in part on output of a sensor system onboard the vehicle;

in response to the stationary condition, obtaining, by the control module, an image from an imaging device onboard the vehicle during the stationary condition; and

obtaining, by the control module, reference data for a field of view of the imaging device at a vehicle pose during the stationary condition;

identifying, by the control module, a reference position for a reference object based on the reference data;

identifying, by the control module, the reference object at a second position within the image; and

validating, by the control module, a transformation associated with the imaging device based on a difference between the reference position and the second position.

2. The method of claim 1, further comprising, after validating the transformation:

assigning, by the control module, one or more attributes to the image based at least in part on the transformation; and

autonomously operating, by the control module, one or more actuators onboard the vehicle in a manner that is influenced by the one or more attributes assigned to the image.

3. The method of claim 1, wherein obtaining the reference data comprises:

obtaining, from a map database, mapping data corresponding to the vehicle pose; and

identifying a subset of the mapping data corresponding to the field of view of the imaging device based on a pose of the imaging device relative to the vehicle.

4. The method of claim 3, further comprising identifying the reference object within the subset of the mapping data.

5. The method of claim 4, the reference object having a fixed position within the subset of the mapping data, wherein identifying the reference position comprises determining an expected position of the reference object in a coordinate frame associated with the imaging device based on the transformation.

6. The method of claim 5, wherein identifying the reference object at the second position comprises:

determining a region of the image encompassing the expected position of the reference object; and

searching the region of the image for the reference object.

7. The method of claim 6, further comprising initiating, by the control module, a remedial action with respect to the imaging device in response to a failure to identify the reference object within the region of the image.

8. The method of claim 6, wherein searching the region of the image of the reference object comprises applying a convolutional neural network to the region of the image to locate the reference object within the region.

9. The method of claim 5, wherein validating the transformation comprises:

calculating a validity metric value based on the difference between the expected position and the second position; and

validating the transformation when the validity metric value is less than a threshold.

10. The method of claim 5, further comprising initiating, by the control module, a remedial action with respect to the imaging device when the difference between the expected position and the second position is greater than a threshold.

11. A non-transitory computer-readable medium having instructions stored thereon that are executable by the control module to perform the method of claim 1.

12. A vehicle, comprising:

an imaging device onboard the vehicle;

a reference data source;

one or more sensors onboard the vehicle;

one or more actuators onboard the vehicle; and

a controller that, by a processor, detects a stationary condition based at least in part on output of the one or more sensors, obtains an image from the imaging device during the stationary condition, obtains reference data for a field of view of the imaging device at a vehicle pose during the stationary condition from the reference data source, identifies a reference position for a reference object based on the reference data, identifies the reference object at a second position within the image, validates a transformation associated with the imaging device based on a difference between the reference position and the second position, and thereafter autonomously operates the one or more actuators onboard the vehicle in a manner that is influenced by the validated transformation.

13. The vehicle of claim 12, wherein:

the reference data source comprises a map database storing mapping data; and

the reference data comprises a subset of a mapping data corresponding to the field of view of the imaging device based on the vehicle pose and a pose of the imaging device relative to the vehicle.

14. The vehicle of claim 13, wherein:

the reference object has a fixed position within the subset of the mapping data; and

the reference position comprises an expected position of the reference object in a coordinate frame associated with the imaging device based on the transformation and the fixed position within the subset of the mapping data.

15. The vehicle of claim 12, wherein the controller determines a region of the image encompassing an expected position of the reference object based on the reference position and searches the region of the image for the reference object to identify the reference object at the second position within the image.

16. The vehicle of claim 15, wherein the controller initiates a remedial action with respect to the imaging device in response to an absence of the reference object within the region of the image.

17. The vehicle of claim 12, further comprising a ranging device onboard the vehicle providing ranging data, wherein the controller autonomously operates the one or more actuators onboard the vehicle in a manner that is influenced by the validated transformation by:

assigning one or more attributes to the image based at least in part on the ranging data and one or more calibrated parameter values associated with the transformation;

identifying an object based at least in part on the one or more attributes; and

determining a command for the one or more actuators in a manner that is influenced by the object.

18. The vehicle of claim 17, wherein the imaging device comprises a camera and the ranging device comprises a lidar device.

19. A method of controlling a vehicle, the method comprising:

obtaining first data from a sensing device onboard the vehicle during a stationary condition of the vehicle; and

identifying an expected position for a reference object within the first data based on reference data corresponding to a field of view of the sensing device and one or more calibrated transformation parameter values associated with the sensing device;

identifying the reference object at an observed position within the first data; and

initiating a remedial action with respect to the sensing device when a difference between the expected position and the observed position is greater than a threshold; and

validating the one or more calibrated transformation parameter values when the difference between the expected position and the observed position is less than the threshold.

20. The method of claim 19, wherein identifying the expected position comprises:

obtaining mapping data corresponding to a vehicle pose during the stationary condition;

identifying a subset of the mapping data corresponding to the field of view of the sensing device based on a pose of the sensing device relative to the vehicle;

identifying the reference object at a fixed position within the subset of the mapping data;

converting the fixed position to the expected position in a coordinate frame associated with the sensing device based on the one or more calibrated transformation parameter values.