System and method for mapping through inferences of observed objects

• US 10,753,750 B2
• Filed: 07/12/2018
• Issued: 08/25/2020
• Est. Priority Date: 07/12/2018
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Research Institute, Inc. (Toyota Motor Corporation)
• Inventors: Wolcott, Ryan W., Walls, Jeffrey M.
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US16/033,281
• Publication Number:
  US 20200018606A1
• Time in Patent Office: 775 Days
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3819 Road shape data, e.g. outline of a route
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G05D 1/0219 ensuring the processing of the whole working surface
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

System, methods, and other embodiments described herein relate to improving mapping of a surrounding environment by a mapping vehicle. In one embodiment, a method includes identifying dynamic objects within the surrounding environment that are proximate to the mapping vehicle from sensor data of at least one sensor of the mapping vehicle. The dynamic objects are trackable objects that are moving within the surrounding environment. The method includes generating paths of the dynamic objects through the surrounding environment relative to the mapping vehicle according to separate observations of the dynamic objects embodied within the sensor data. The method includes producing a map of the surrounding environment from the paths.

Claims

1. A mapping system for improving mapping of a surrounding environment using a mapping vehicle, comprising:

one or more processors;

a memory communicably coupled to the one or more processors and storing;

a tracking module including instructions that when executed by the one or more processors cause the one or more processors to identify dynamic objects within the surrounding environment that are proximate to the mapping vehicle from sensor data of at least one sensor of the mapping vehicle, wherein the dynamic objects are trackable objects that are moving within the surrounding environment; and

an inference module including instructions that when executed by the one or more processors cause the one or more processors to generate paths of the dynamic objects through the surrounding environment relative to the mapping vehicle according to separate observations of the dynamic objects embodied within the sensor data, and

wherein the inference module includes instructions to produce a map of the surrounding environment from the paths by forming the paths into at least roads that relate logged locations of the mapping vehicle with the paths of the dynamic objects to infer navigable portions of the surrounding environment in place of independently navigating the navigable portions using the mapping vehicle.

2. The mapping system of claim 1, wherein the inference module includes instructions to produce the map including instructions to define at least sidewalks and lane configurations of the roads associated with the mapping vehicle and the dynamic objects according to the navigable portions of the surrounding environment as defined from the paths, and wherein the paths of the dynamic objects are observed prior positions of the dynamic objects.

3. The mapping system of claim 1, wherein the inference module includes instructions to produce the map including instructions to analyze the paths to determine features of the surrounding environment as a function of associated classes and the navigable portions, and

wherein the features include at least lane configurations of a road.

4. The mapping system of claim 1, wherein the inference module includes instructions to generate the paths including instructions to interpolate the paths from observations of the dynamic objects embodied by the sensor data that provide for inferring the navigable portions of the surrounding environment relative to the mapping vehicle.

5. The mapping system of claim 1, wherein the tracking module includes instructions to acquire the sensor data from the at least one sensor iteratively as the mapping vehicle travels along a route, wherein the tracking module includes instructions to identify the dynamic objects including instructions to analyze the sensor data according to an object identification function to determine a presence of the dynamic objects and track the dynamic objects to produce the separate observations, and

wherein the inference module includes instructions to produce the map including instructions to use the paths to infer navigable portions of a surrounding environment in place of independently navigating lanes of an associated road to produce the mapping.

6. The mapping system of claim 1, wherein the inference module includes instructions to correlate the paths with detected static elements in the surrounding environment to improve producing the map from the paths, and

wherein the inference module includes instructions to produce the map including instructions to identify changes within the surrounding environment by comparing the map with previously mapped features and updating the previously mapped features according to the map.

7. The mapping system of claim 1, wherein the tracking module includes instructions to determine a location of the mapping vehicle by using at least the sensor data to localize the mapping vehicle in the surrounding environment, and

wherein observations of the dynamic objects are labeled with the location of the mapping vehicle to provide a position relative to the mapping vehicle.

8. The mapping system of claim 1, wherein the inference module includes instructions to produce the map according to features inferred from the paths including lane configurations, intersection configurations, crosswalk configurations, and a presence of sidewalks, and

wherein the dynamic objects include at least vehicles and pedestrians in the surrounding environment.

9. A non-transitory computer-readable medium for improving mapping of a surrounding environment using a mapping vehicle and including instructions that when executed by one or more processors cause the one or more processors to:

identify dynamic objects within the surrounding environment that are proximate to the mapping vehicle from sensor data of at least one sensor of the mapping vehicle, wherein the dynamic objects are trackable objects that are moving within the surrounding environment; and

generate paths of the dynamic objects through the surrounding environment relative to the mapping vehicle according to separate observations of the dynamic objects embodied within the sensor data; and

produce a map of the surrounding environment from the paths by forming the paths into at least roads that relate logged locations of the mapping vehicle with the paths of the dynamic objects to infer navigable portions of the surrounding environment in place of independently navigating the navigable portions using the mapping vehicle.

10. The non-transitory computer-readable medium of claim 9, wherein the paths of the dynamic objects are observed prior positions of the dynamic objects

wherein the instructions to produce the map include instructions to analyze the paths to determine features of the surrounding environment as a function of associated classes and the navigable portions, and wherein the features include at least lane configurations of the roads.

11. The non-transitory computer-readable medium of claim 9, wherein the instructions to generate the paths include instructions to interpolate the paths from observations of the dynamic objects embodied by the sensor data that provide for inferring the navigable portions of the surrounding environment relative to the mapping vehicle.

12. The non-transitory computer-readable medium of claim 10, wherein the instructions include instructions to acquire the sensor data from the at least one sensor iteratively as the mapping vehicle travels along a route, and

wherein the instructions to identify the dynamic objects include instructions to analyze the sensor data according to an object identification function to determine a presence of the dynamic objects and track the dynamic objects to produce the separate observations.

13. The non-transitory computer-readable medium of claim 9, wherein the instructions include instructions to determine a location of the mapping vehicle by using at least the sensor data to localize the mapping vehicle in the surrounding environment, and

wherein observations of the dynamic objects from the sensor data are labeled with the location of the mapping vehicle to provide a position relative to the mapping vehicle.

14. A method of improving mapping of a surrounding environment by a mapping vehicle, comprising:

identifying dynamic objects within the surrounding environment that are proximate to the mapping vehicle from sensor data of at least one sensor of the mapping vehicle, wherein the dynamic objects are trackable objects that are moving within the surrounding environment;

generating paths of the dynamic objects through the surrounding environment relative to the mapping vehicle according to separate observations of the dynamic objects embodied within the sensor data; and

producing a map of the surrounding environment from the paths by forming the paths into at least roads that relate logged locations of the mapping vehicle with the paths of the dynamic objects to infer navigable portions of the surrounding environment in place of independently navigating the navigable portions using the mapping vehicle.

15. The method of claim 14, wherein producing the map includes defining at least sidewalks and lane configurations of one or more roadways associated with the mapping vehicle and the dynamic objects from the navigable portions of the surrounding environment as defined from the paths, and wherein the paths of the dynamic objects are observed prior positions of the dynamic objects.

16. The method of claim 14, wherein producing the map includes analyzing the paths to determine features of the surrounding environment as a function of associated classes of the dynamic objects and the navigable portions, and wherein the features include at least configurations of lanes within a road.

17. The method of claim 14, wherein generating the paths includes interpolating the paths from observations of the dynamic objects embodied by the sensor data to infer the navigable portions of the surrounding environment relative to the mapping vehicle, and

wherein generating the map includes correlating the paths with detected static elements in the surrounding environment to improve identification of the paths.

18. The method of claim 14, further comprising:

acquiring the sensor data from the at least one sensor iteratively as the mapping vehicle travels along a route, wherein identifying the dynamic objects includes analyzing the sensor data according to an object identification function to determine a presence of the dynamic objects and track the dynamic objects to produce the separate observations.

19. The method of claim 14, wherein producing the map includes identifying changes within the surrounding environment by comparing the map with previously mapped features and updating the previously mapped features according to the map.

20. The method of claim 14, further comprising:

determining a location of the mapping vehicle by using at least the sensor data to localize the mapping vehicle in the surrounding environment, wherein observations of the dynamic objects are labeled with the location of the mapping vehicle to provide a position relative to the mapping vehicle.