Sensor System for Multiple Perspective Sensor Data Sets

• US 20190212746A1
• Filed: 06/05/2018
• Published: 07/11/2019
• Est. Priority Date: 01/11/2018
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Cheng, Bin, Bansal, Gaurav, Yee, Ryan Matthew, Chan, Ellick Ming Huen, Senatore, Carmine
• Granted Patent: US 11,422,561 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2554/402 Type
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/05 Big data
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2556/35 Data fusion
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: G01K 3/14 in respect of space
• CPC Class Codes: G01V 9/005 by thermal methods, e.g. after generation of heat by chemical reactions
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

The disclosure includes embodiments for a sensor system for multiple perspective sensor data sets. In some embodiments, a method executed by an ego vehicle includes receiving, from a set of remote vehicles, a set of sensor data describing a set of preliminary heatmaps for a roadway environment. The method includes reconciling discrepancies in the set of heatmaps and a preliminary heatmap of the ego vehicle to form a combined heatmap that describes the objects in the roadway environment as collectively observed by the onboard sensors of the set of remote vehicles and the ego vehicle. The method includes providing the combined heatmap to one or more of the remote vehicles included in the set of remote vehicles. The method includes modifying an operation of the ego vehicle based on the combined heatmap. For example, an operation of an autonomous driving system of the ego vehicle is modified.

Claims

1. A method implemented by an ego vehicle, the method comprising:

receiving, from a set of remote vehicles, a set of sensor data describing a set of preliminary heatmaps for a roadway environment, wherein the heatmaps included in the set of preliminary heatmaps describes objects in the roadway environment as observed by onboard sensors of individual remote vehicles included in the set of remote vehicles;

reconciling discrepancies in the set of heatmaps and a preliminary heatmap of the ego vehicle to form a combined heatmap that describes the objects in the roadway environment as collectively observed by the onboard sensors of the set of remote vehicles and the ego vehicle;

providing the combined heatmap to one or more of the remote vehicles included in the set of remote vehicles; and

modifying an operation of the ego vehicle based on the combined heatmap.

2. The method of claim 1, wherein the preliminary heatmap is a two-dimensional electronic map.

3. The method of claim 2, wherein the preliminary heatmap includes a bounding box around each object identified in the roadway environment.

4. The method of claim 1, wherein the preliminary heatmap describes one or more of the following:

a preliminary estimate of a position of the objects within the environment;

a preliminary classification for each object; and

a preliminary confidence value that the preliminary classification is correct.

5. The method of claim 1, wherein the combined heatmap is a three-dimensional electronic map.

6. The method of claim 1, wherein the combined heatmap removes one or more of the following that are present in the preliminary heatmap for an individual remote vehicle:

an occlusion;

a missed detection of an object; and

an uncertain detection of an object.

7. The method of claim 1, wherein the ego vehicle is an autonomous vehicle.

8. The method of claim 1, wherein the ego vehicle is a highly autonomous vehicle.

9. A system of an ego vehicle comprising:

a processor that is communicatively coupled to a non-transitory memory, wherein the non-transitory memory stores computer code which, when executed by the processor, causes the processor to;

receive, from a set of remote vehicles, a set of sensor data describing a set of preliminary heatmaps for a roadway environment, wherein the heatmaps included in the set of preliminary heatmaps describes objects in the roadway environment as observed by onboard sensors of individual remote vehicles included in the set of remote vehicles;

reconcile discrepancies in the set of heatmaps and a preliminary heatmap of the ego vehicle to form a combined heatmap that describes the objects in the roadway environment as collectively observed by the onboard sensors of the set of remote vehicles and the ego vehicle;

provide the combined heatmap to one or more of the remote vehicles included in the set of remote vehicles; and

modify an operation of the ego vehicle based on the combined heatmap.

10. The system of claim 9, wherein the preliminary heatmap is a two-dimensional electronic map.

11. The system of claim 10, wherein the preliminary heatmap includes a bounding box around each object identified in the roadway environment.

12. The system of claim 9, wherein the preliminary heatmap describes one or more of the following:

a preliminary estimate of a position of the objects within the environment;

a preliminary classification for each object; and

a preliminary confidence value that the preliminary classification is correct.

13. The system of claim 9, wherein the combined heatmap is a three-dimensional electronic map.

14. The system of claim 9, wherein the combined heatmap removes one or more of the following that are present in the preliminary heatmap for an individual remote vehicle:

an occlusion;

a missed detection of an object; and

an uncertain detection of an object.

15. The system of claim 9, wherein the ego vehicle is an autonomous vehicle.

16. The system of claim 9, wherein the ego vehicle is a highly autonomous vehicle.

17. A computer program product of an ego vehicle comprising a non-transitory memory storing computer-executable code that, when executed by a processor, causes the processor to:

receive, from a set of remote vehicles, a set of sensor data describing a set of preliminary heatmaps for a roadway environment, wherein the heatmaps included in the set of preliminary heatmaps describes objects in the roadway environment as observed by onboard sensors of individual remote vehicles included in the set of remote vehicles;

reconcile discrepancies in the set of heatmaps and a preliminary heatmap of the ego vehicle to form a combined heatmap that describes the objects in the roadway environment as collectively observed by the onboard sensors of the set of remote vehicles and the ego vehicle;

provide the combined heatmap to one or more of the remote vehicles included in the set of remote vehicles; and

modify an operation of the ego vehicle based on the combined heatmap.

18. The computer program product of claim 17, wherein the preliminary heatmap is a two-dimensional electronic map.

19. The computer program product of claim 18, wherein the preliminary heatmap includes a bounding box around each object identified in the roadway environment.

20. The computer program product of claim 17, wherein the preliminary heatmap describes one or more of the following:

a preliminary estimate of a position of the objects within the environment;

a preliminary classification for each object; and

a preliminary confidence value that the preliminary classification is correct.