Map building with sensor measurements

• US 10,558,185 B2
• Filed: 01/31/2017
• Issued: 02/11/2020
• Est. Priority Date: 09/08/2016
• Status: Active Grant

• Current Assignee: Siemens Electronic Design Automation GmbH (Siemens AG)
• Original Assignee: Mentor Graphics Corporation (Siemens AG)
• Inventors: Mercep, Ljubo, Pollach, Matthias
• Primary Examiner(s): Heinle, Courtney D
• Application Number:
  US15/420,976
• Publication Number:
  US 20180067966A1
• Time in Patent Office: 1,106 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2556/35 Data fusion
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/3407 specially adapted for specific applications
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G01C 21/3667 Display of a road map G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/931 : of land vehicles
• CPC Class Codes: G01S 2013/93271 : in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 : in the back of the vehicles
• CPC Class Codes: G01S 2013/93274 : on the side of the vehicles
• CPC Class Codes: G01S 7/4808 : Evaluating distance, position or velocity data
• CPC Class Codes: G05B 17/02 : electric
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0255 : using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/0257 : using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0259 : using magnetic or electromagnetic means
• CPC Class Codes: G05D 1/0268 : using internal positioning means
• CPC Class Codes: G05D 1/0276 : using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G05D 1/0278 : using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06F 16/29 : Geographical information databases
• CPC Class Codes: G06F 16/5854 : using shape and object relationship
• CPC Class Codes: G06F 30/20 : Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08G 1/161 : Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 : involving continuous checking
• CPC Class Codes: G08G 1/164 : Centralised systems, e.g. external to vehicles

Abstract

This application discloses a computing system to implement map building in an assisted or automated driving system. The computing system can track movement of a vehicle based on sensor measurement data populated in an environmental model and vehicle movement measurements. The computing system can correlate the tracked movement of the vehicle to map data based on a previously detected location of the vehicle relative to the map data. The computing system can modify the map data to include the sensor measurement data utilized to track the movement of the vehicle based on the correlation of the tracked movement of the vehicle to map data. The computing system can modify the map data by building a map of the sensor measurement data and the tracked movement of the vehicle, and populating the map data with the built map.

Claims

1. A method comprising:

when a vehicle is without an external reference utilized to detect a location of the vehicle relative to map data, tracking, by a computing system in the vehicle, movement of the vehicle from the detected location of the vehicle based, at least in part, on vehicle movement measurements and sensor measurement data populated in an environmental model, and generating, by the computing system in a map building mode, new map data from the sensor measurement data and the tracked movement of the vehicle;

exiting, by the computing system, the map building mode by directing the vehicle to move towards an area having the map data; and

in response to reacquiring the external reference in the area having the map data, aligning, by the computing system, the sensor measurement data utilized to track the movement of the vehicle to a global coordinate field associated with the map data, and populating, by the computing system, the new map data into the map data based, at least in part, on the alignment of the tracked movement of the vehicle to the global coordinate field associated with the map data, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the modified map data.

2. The method of claim 1, further comprising identifying, by the computing system, a portion of the map data is incomplete or missing, and tracking the movement of the vehicle in response to the identification.

3. The method of claim 2, further comprising, in response to the identification that the portion of the map data is incomplete or missing, prompting, by the computing system, the vehicle to reduce speed and sensors to capture the sensor measurement data with increased sensor refresh rate or increased data precision.

4. The method of claim 1, wherein tracking the movement of the vehicle is based on at least one of vehicle movement measurements or in-lane localization for the vehicle from on the sensor measurement data populated in the environmental model.

5. The method of claim 1, wherein modifying the map data to include the sensor measurement data is performed based on the reacquired external reference.

6. The method of claim 1, wherein the external reference corresponding to the location of the vehicle relative to the map data is at least one of global positioning system (GPS) information providing a global reference to the location of the vehicle relative to the map data or a correlation of the sensor measurement data populated in the environmental model to the map data.

7. The method of claim 1, wherein modifying the map data further comprises building a map based, at least in part, on the sensor measurement data utilized to track the movement of the vehicle and the tracked movement of the vehicle, and populating the map data with the built map.

8. An apparatus comprising at least one computer-readable memory device storing instructions configured to cause one or more processing devices to perform operations comprising:

when a vehicle is without an external reference utilized to detect a location of the vehicle relative to map data, tracking movement of the vehicle from the detected location of the vehicle based, at least in part, on vehicle movement measurements and sensor measurement data populated in an environmental model, and, in a map building mode, generating new map data from the sensor measurement data and the tracked movement of the vehicle;

exiting the map building mode by directing the vehicle to move towards an area having the map data; and

in response to reacquiring the external reference in the area having the map data, aligning the sensor measurement data utilized to track the movement of the vehicle to a global coordinate field associated with the map data, and populating the new map data into the map data based, at least in part, on the alignment of the tracked movement of the vehicle to the global coordinate field associated with the map data, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the modified map data.

9. The apparatus of claim 8, wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising identifying a portion of the map data is incomplete or missing, and tracking the movement of the vehicle in response to the identification.

10. The apparatus of claim 9, wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising, in response to the identification that the portion of the map data is incomplete or missing, prompting the vehicle to reduce speed and sensors to capture the sensor measurement data with increased sensor refresh rate or increased data precision.

11. The apparatus of claim 8, wherein tracking the movement of the vehicle is based on at least one of vehicle movement measurements or in-lane localization for the vehicle from on the sensor measurement data populated in the environmental model.

12. The apparatus of claim 8, wherein modifying the map data to include the sensor measurement data is performed based on the reacquired external reference.

13. The apparatus of claim 8, wherein the external reference corresponding to the location of the vehicle relative to the map data is at least one of global positioning system (GPS) information providing a global reference to the location of the vehicle relative to the map data or a correlation of the sensor measurement data populated in the environmental model to the map data.

14. The apparatus of claim 8, wherein modifying the map data further comprises building a map based, at least in part, on the sensor measurement data utilized to track the movement of the vehicle and the tracked movement of the vehicle, and populating the map data with the built map.

15. A system comprising:

a memory device configured to store machine-readable instructions; and

a computing system including one or more processing devices, in response to executing the machine-readable instructions, configured to;

when a vehicle is without an external reference utilized to detect a location of the vehicle relative to map data, track movement of the vehicle from the detected location of the vehicle based, at least in part, on vehicle movement measurements and sensor measurement data populated in an environmental model, and, in a map building mode, generate new map data from the sensor measurement data and the tracked movement of the vehicle;

exit the map building mode by directing the vehicle to move towards an area having the map data; and

in response to reacquiring the external reference in the area having the map data, align the sensor measurement data utilized to track the movement of the vehicle to a global coordinate field associated with the map data, and populate the new map data into the map data based, at least in part, on the alignment of the tracked movement of the vehicle to the global coordinate field associated with the map data, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the modified map data.

16. The system of claim 15, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify a portion of the map data is incomplete or missing, and tracking the movement of the vehicle in response to the identification.

17. The system of claim 16, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to, in response to the identification that the portion of the map data is incomplete or missing, prompt the vehicle to reduce speed and sensors to capture the sensor measurement data with increased sensor refresh rate or increased data precision.

18. The system of claim 15, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to track the movement of the vehicle based on at least one of vehicle movement measurements or in-lane localization for the vehicle from on the sensor measurement data populated in the environmental model.

19. The system of claim 15, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to-modify the map data to include the sensor measurement data based on the reacquired external reference.

20. The system of claim 15, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to modify the map data by building a map based, at least in part, on the sensor measurement data utilized to track the movement of the vehicle and the tracked movement of the vehicle, and populating the map data with the built map.