Enhancing autonomous vehicle perception with off-vehicle collected data

• US 10,101,745 B1
• Filed: 04/26/2017
• Issued: 10/16/2018
• Est. Priority Date: 04/26/2017
• Status: Active Grant

• Current Assignee: Charles Stark Draper Laboratory Incorporated
• Original Assignee: Charles Stark Draper Laboratory Incorporated
• Inventors: Sun, Fei, Jones, Troy, Lennox, Scott
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US15/497,821
• Publication Number:
  US 20180314247A1
• Time in Patent Office: 538 Days
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence
• CPC Class Codes: G05B 2219/2637 Vehicle, car, auto, wheelchair
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06Q 10/047 Optimisation of routes or paths, e.g. travelling salesman problem
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

In an embodiment, a method includes receiving, at an autonomous vehicle, reported data regarding an object in proximity to the autonomous vehicle. The data is collected by a collecting device external to the autonomous vehicle, and is relayed to the autonomous vehicle via a server. The reported data includes a current location, type, or predicted location of the object. The method further includes determining whether the reported data of the object matches an object in an object list determined by on-board sensors of the autonomous vehicle. If the determination finds a found object in the object list, the method correlates the reported data of the object to the found object in the object list. Otherwise, the method adds the reported data of the object to an object list of objects detected by sensor from on-board sensors of the autonomous vehicle. In embodiments, the collecting device is a mobile device.

Claims

1. A method comprising:

receiving, at an autonomous vehicle, reported data regarding an object in proximity to the autonomous vehicle, the data collected by a collecting device external to the autonomous vehicle and relayed to the autonomous vehicle via a server, the reported data including at least one of a current location of the object, a type of the object, and a predicted location of the object;

determining, at the autonomous vehicle, whether the reported data of the object correlates with a found object in an object list;

if the determination finds the found object in the object list, adding the reported data of the object to data associated with the found object in the object list; and

otherwise, adding the reported data of the object to an object list of objects detected by sensor from on-board sensors of the autonomous vehicle.

2. The method of claim 1, wherein the type of the object is a pedestrian, bicycle, vehicle, or vehicle type.

3. The method of claim 1, wherein the reported data includes location data collected from the collecting device of the object, further wherein the location data provides the current location of the object, and further comprising:

predicting a future location of the object based on a history of the location data.

4. The method of claim 1, wherein the type of the object is determined based on a vibration pattern of the collecting device, wherein the collecting device is a mobile device.

5. The method of claim 1, wherein the type of object is determined by a user of the collecting device self-identifying the type of object.

6. The method of claim 1, wherein the reported data further includes at least one of velocity and acceleration data of the object, a route map of the object, and a calendar of a user of the object.

7. The method of claim 6, wherein the predicted location of the object is determined by loading the route map of the object, or by loading a destination location from the calendar of the user and generating a route map from the current location to the destination location.

8. The method of claim 1, further comprising building a sensor model based on the reported data from the collection device and data from the on-board sensors of the autonomous vehicle.

9. The method of claim 1, further comprising modifying a sensor model based on discrepancies between the reported data from the collection device and data from the on-board sensors of the autonomous vehicle.

10. The method of claim 1, further comprising generating the reported data of the object by analyzing at least one image taken by the collecting device.

11. The method of claim 1, further comprising identifying the reported data as an emergency vehicle signal.

12. The method of claim 1, further comprising:

reporting, from the autonomous vehicle to the server, at least one of a location and direction of the vehicle, such that the data returned from the server is related to a current and future locations of the autonomous vehicle.

13. The method of claim 12, wherein the data returned from the server is selected from a list of object data reported from multiple collecting devices stored at the server.

14. The method of claim 1, wherein at least one of the objects of the object list is determined by on-board sensors of the autonomous vehicle.

15. The method of claim 1, wherein the predicted location is calculated based on a propagation delay of the reported data.

16. A system comprising:

a processor; and

a memory with computer code instructions stored therein, the memory operatively coupled to said processor such that the computer code instructions configure the processor to implement;

a machine interaction controller configured to receive, at an autonomous vehicle, reported data regarding an object in proximity to the autonomous vehicle, the data collected by a collecting device external to the autonomous vehicle and relayed to the autonomous vehicle via a server, the reported data including at least one of a current location of the object, a type of the object, and a predicted location of the object; and

a perception controller configured to;

determine, at the autonomous vehicle, whether the reported data of the object correlates with a found object in an object list, wherein at least one of the objects of the object list is determined by on-board sensors of the autonomous vehicle;

if the determination finds the found object in the object list, add the reported data of the object to data associated with the found object in the object list, and

otherwise, add the reported data of the object to an object list of objects detected by sensor from on-board sensors of the autonomous vehicle.

17. The system of claim 16, wherein the type of the object is a pedestrian, bicycle, vehicle, or vehicle type.

18. The system of claim 16, wherein the reported data includes location data collected from the collecting device of the object, further wherein the location data provides the current location of the object, and further wherein the perception controller is configured to predict a future location of the object based on a history of the location data.

19. The system of claim 16, wherein the type of the object is determined based on a vibration pattern of the mobile device.

20. The system of claim 16, wherein the type of object is determined by a user of the collecting device self-identifying the type of object, wherein the collecting device is a mobile device.

21. The system of claim 16, wherein the reported data further includes at least one of acceleration data of the object, a route map of the object, and a calendar of a user of the object.

22. The system of claim 21, wherein the predicted location of the object is determined by loading the route map of the object, or by loading a destination location from the calendar of the user and generating a route map from the current location to the destination location.

23. The system of claim 16, wherein the processor is further configured to implement:

a model modification module configured to build a sensor model based on the reported data from the collection device and data from the on-board sensors of the autonomous vehicle.

24. The system of claim 16, wherein the processor is further configured to implement:

a model modification module configured to modifying a sensor model based on discrepancies between the reported data from the collection device and data from the on-board sensors of the autonomous vehicle.

25. The system of claim 16, wherein the reported data of the object is generated by analyzing at least one image taken by the collecting device.

26. The system of claim 16, wherein the perception controller is further configured to identifying the reported data as an emergency vehicle signal.

27. The system of claim 16, wherein the machine interaction controller is further configured to report, from the autonomous vehicle to the server, at least one of a location and direction of the vehicle, such that the data returned from the server is related to a current and future locations of the autonomous vehicle.

28. The system of claim 16, further comprising, wherein the data returned from the server is selected from a list of object data reported from multiple collecting devices stored at the server.

29. The system of claim 16, wherein at least one of the objects of the object list is determined by on-board sensors of the autonomous vehicle.

30. The system of claim 16, wherein the predicted location is calculated based on a propagation delay of the reported data.

31. A non-transitory computer-readable medium configured to store instructions for providing data to an autonomous vehicle, the instructions, when loaded and executed by a processor, causes the processor to:

receive, at an autonomous vehicle, reported data regarding an object in proximity to the autonomous vehicle, the data collected by a collecting device external to the autonomous vehicle and relayed to the autonomous vehicle via a server, the reported data including at least one of a current location of the object, a type of the object, and a predicted location of the object;

determine, at the autonomous vehicle, whether the reported data of the object correlates with a found object in an object list;

if the determination finds the found object in the object list, add the reported data of the object to data associated with the found object in the object list; and

otherwise, add the reported data of the object to an object list of objects detected by sensor from on-board sensors of the autonomous vehicle.