Methods and systems for calibrating sensors using road map data

• US 10,724,865 B1
• Filed: 06/23/2017
• Issued: 07/28/2020
• Est. Priority Date: 07/23/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Ferguson, Dave, Iordache, Dorel Ionut
• Primary Examiner(s): Lee, Paul D
• Application Number:
  US15/631,642
• Time in Patent Office: 1,131 Days
• Field of Search: 702 94
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 19/40 Correcting position, velocity or attitude
• CPC Class Codes: G01S 3/781 Details
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/4868 Controlling received signal intensity or exposure of sensor
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Example methods and systems for calibrating sensors using road map data are provided. An autonomous vehicle may use various vehicle sensors to assist in navigation. Within examples, the autonomous vehicle may calibrate vehicle sensors through performing a comparison or analysis between information about the environment received by sensors with similar information provided by map data (e.g., a road map). The autonomous vehicle may compare object locations as provided by the sensors and as shown by map data. Based on the comparison, the autonomous vehicle may adjust various sensors to accurately reflect the information as provided by the road map. In some instances, the autonomous vehicle may adjust the position, height, orientation, direction-of-focus, scaling, or other parameters of a sensor based on the information provided by a road map.

Claims

1. A method comprising:

receiving, at a computing system and from a first set of sensors coupled to a vehicle, first sensor data depicting an environment of the vehicle;

based on the first sensor data, generating first map data that represents the environment of the vehicle;

receiving, at a computing system and from a second set of sensors coupled to the vehicle, second sensor data depicting the environment of the vehicle;

based on the second sensor data, generating second map data that represents the environment of the vehicle;

performing a comparison between the first map data and the second map data; and

based on the comparison, modifying operation of one or more sensors of the second set of sensors while the vehicle navigates a path using the first map data.

2. The method of claim 1, wherein the first set of sensors includes one or more LIDAR sensors.

3. The method of claim 1, wherein the first set of sensors includes at least a first type of sensor and at least a second type of sensor, and wherein the first type of sensor is different from the second type of sensor.

4. The method of claim 1, wherein the second set of sensors includes one or more cameras.

5. The method of claim 1, wherein the first map data represents respective positions of lane markers in the environment of the vehicle.

6. The method of claim 1, wherein modifying operation of one or more sensors of the second set of sensors while the vehicle navigates the path using the first map data comprises:

adjusting the one or more sensors such that subsequent second sensor data depicting the environment of the vehicle from the second set of sensors aligns with subsequent first sensor data depicting the environment of the vehicle from the first set of sensors.

7. The method of claim 1, wherein modifying operation of one or more sensors of the second set of sensors while the vehicle navigates the path using the first map data comprises:

causing one or more sensors of the second set of sensors to power off.

8. The method of claim 1, further comprising:

receiving third map data representative of the environment; and

wherein performing the comparison between the first map data and the second map data further includes comparing the first map data and the second map data with the third map data.

9. The method of claim 8, wherein based on the comparison, modifying operation of one or more sensors of the second set of sensors while the vehicle navigates a path using the first map data further comprises:

modifying the operation of one or more sensors of the second set of sensors while the vehicle navigates the path using the first map data and the third map data.

10. A non-transitory computer readable storage medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

receiving, from a first set of sensors coupled to a vehicle, first sensor data depicting an environment of the vehicle;

based on the first sensor data, generating first map data that represents the environment of the vehicle;

receiving, from a second set of sensors coupled to the vehicle, second sensor data depicting the environment of the vehicle;

based on the second sensor data, generating second map data that represents the environment of the vehicle;

performing a comparison between the first map data and the second map data; and

based on the comparison, modifying operation of one or more sensors of the second set of sensors while the vehicle navigates a path using the first map data.

11. The non-transitory computer readable storage medium of claim 10, wherein the first set of sensors includes one or more LIDAR sensors.

12. The non-transitory computer readable storage medium of claim 10, wherein the first set of sensors includes at least a first type of sensor and at least a second type of sensor, and wherein the first type of sensor is different from the second type of sensor.

13. The non-transitory computer readable storage medium of claim 10, wherein the second set of sensors includes one or more cameras.

14. The non-transitory computer readable storage medium of claim 10, wherein the first map data represents respective positions of lane markers in the environment of the vehicle.

15. The non-transitory computer readable storage medium of claim 10, wherein modifying operation of one or more sensors of the second set of sensors while the vehicle navigates the path using the first map data comprises:

adjusting the one or more sensors such that subsequent second sensor data depicting the environment of the vehicle from the second set of sensors aligns with subsequent first sensor data depicting the environment of the vehicle from the first set of sensors.

16. The non-transitory computer readable storage medium of claim 10, wherein modifying operation of one or more sensors of the second set of sensors while the vehicle navigates the path using the first map data comprises:

causing one or more sensors of the second set of sensors to power off.

17. A system, comprising:

a vehicle;

a computing system positioned at a remote location from the vehicle;

a non-transitory computer readable medium; and

program instructions stored on the non-transitory computer readable medium executable by the computing system to;

receive, from a first set of sensors coupled to the vehicle, first sensor data depicting an environment of the vehicle;

based on the first sensor data, generate first map data that represents the environment of the vehicle;

receive, from a second set of sensors coupled to the vehicle, second sensor data depicting the environment of the vehicle;

based on the second sensor data, generate second map data that represents the environment of the vehicle;

perform a comparison between the first map data and the second map data; and

based on the comparison, modify operation of one or more sensors of the second set of sensors while the vehicle navigates a path using the first map data.

18. The system of claim 17, wherein the first sensor data and the second sensor data depict the environment of the vehicle during a given time period.

19. The system of claim 17, wherein the first set of sensors includes one or more LIDAR sensors, and wherein the second set of sensors includes one or more cameras.

20. The system of claim 17, wherein the computing system is further configured to:

receive third map data representative of the environment; and

wherein the comparison between the first map data and the second map data further includes comparing the first map data and the second map data with the third map data.