SENSOR FUSION BY OPERATIONS-CONTROL VEHICLE FOR COMMANDING AND CONTROLLING AUTONOMOUS VEHICLES

• US 20200133288A1
• Filed: 10/26/2018
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Grant

• Current Assignee: LYFT Incorporated
• Original Assignee: LYFT Incorporated
• Inventors: Abari, Farzad Cyrus Foroughi, Clement, Romain, Paine, Roxie Lynn
• Granted Patent: US 10,928,826 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 7/003 Transmission of data between radar, sonar or lidar systems and remote stations
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0291 Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127

Abstract

In one embodiment, a method includes receiving sensor data from one or more sensors of each of one or more vehicles, processing a combination of the sensor data to generate an assessment of an area surrounding the one or more vehicles based on one or more points-of-view of the area, the one or more points-of-view of the area generated based on synchronizing the combination of the sensor data, and detecting an occurrence of an event based on the assessment of the area. The method further includes identifying one or more instructions corresponding to the event, the instructions associated with a particular vehicle and sending one or more executable instructions based on the instructions to the particular vehicle.

Claims

1. A method comprising:

receiving, by a computing device, sensor data from one or more sensors of each of one or more vehicles;

processing, by the computing device, a combination of the sensor data to generate an assessment of an area surrounding the one or more vehicles based on one or more points-of-view of the area, the one or more points-of-view of the area generated based on synchronizing the combination of the sensor data;

detecting, by the computing device, an occurrence of an event based on the assessment of the area;

identifying, by the computing device, one or more instructions corresponding to the event, the instructions associated with a particular vehicle; and

sending, by the computing device, one or more executable instructions based on the instructions to the particular vehicle.

2. The method of claim 1, further comprising, prior to receiving the sensor data:

requesting, by the computing device, sensor data from the one or more vehicles.

3. The method of claim 2, wherein the sensor data is automatically sent from each of the one or more vehicles.

4. The method of claim 1, wherein the processing of the sensor data from one or more of the vehicles further comprises:

associating the sensor data with one or more time stamps;

synchronizing the sensor data for all vehicles of the one or more vehicles based on the time stamps;

generating the one or more points-of-view of a geographic area based on the synchronized sensor data; and

assessing the one or more points-of-view of the geographic area to determine the occurrence of the event.

5. The method of claim 1, wherein the particular vehicle that the instructions are associated with is selected by:

determining one or more of the vehicles that will make a route alteration based on the occurrence of the event;

determining a priority associated with each of the one or more vehicles; and

selecting a vehicle with the highest priority as the particular vehicle.

6. The method of claim 5, wherein the priority is determined based on at least one of:

a degree of route alteration associated with a vehicle due to the occurrence of the event,

a distance from a vehicle to a location of the event,

a current route of a vehicle, or

a distance from a vehicle to a location associated with the computing device.

7. The method of claim 1, wherein the occurrence of the event is detected by:

determining a relevant route associated with the one or more vehicles;

requesting a report of any events on the relevant route from the one or more vehicles or one or more other vehicles; and

receiving the report of events on the relevant route.

8. The method of claim 1, wherein the executable instructions sent to the particular vehicle instruct the particular vehicle to make one or more route alterations.

9. The method of claim 1, wherein the executable instructions sent to the particular vehicle instruct the particular vehicle to process and assess additional sensor data from one or more other vehicles.

10. The method of claim 1, further comprising:

assessing, by the computing device, a state of the sensors on each of the one or more vehicles based on a comparison of the sensor data from each of the one or more vehicles, and

determining, by the computing device, whether to calibrate the sensors of the one or more vehicles based on the comparison.

11. The method of claim 1, wherein the autonomous vehicle is an unmanned aerial vehicle.

12. The method of claim 1, wherein the event comprises a predetermined event, and wherein the instructions comprise predetermined instructions.

13. A system comprising:

one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising;

receiving, by a computing device, sensor data from one or more sensors of each of one or more vehicles;

processing, by the computing device, a combination of the sensor data to generate an assessment of an area surrounding the one or more vehicles based on one or more points-of-view of the area, the one or more points-of-view of the area generated based on synchronizing the combination of the sensor data;

detecting, by the computing device, an occurrence of an event based on the assessment of the area;

identifying, by the computing device, one or more instructions corresponding to the event, the instructions associated with a particular vehicle; and

sending, by the computing device, one or more executable instructions based on the instructions to the particular vehicle.

14. The system of claim 13, wherein the processing of the sensor data from one or more of the vehicles further comprises:

associating the sensor data with one or more time stamps;

synchronizing the sensor data for all vehicles of the one or more vehicles based on the time stamps;

generating the one or more points-of-view of a geographic area based on the synchronized sensor data; and

assessing the one or more points-of-view of the geographic area to determine the occurrence of the event.

15. The system of claim 13, wherein the particular vehicle that the instructions are associated with is selected by:

determining one or more of the vehicles that will make a route alteration based on the occurrence of the event;

determining a priority associated with each of the one or more vehicles; and

selecting a vehicle with the highest priority as the particular vehicle,

wherein the priority is determined based on at least one of;

a degree of route alteration associated with a vehicle due to the occurrence of the event,

a distance from a vehicle to a location of the event,

a current route of a vehicle, or

a distance from a vehicle to a location associated with the computing device.

16. The system of claim 13, wherein the occurrence of the event is detected by:

determining a relevant route associated with the one or more vehicles;

requesting a report of any events on the relevant route from the one or more vehicles or one or more other vehicles; and

receiving the report of events on the relevant route.

17. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

receiving, by a computing device, sensor data from one or more sensors of each of one or more vehicles;

processing, by the computing device, a combination of the sensor data to generate an assessment of an area surrounding the one or more vehicles based on one or more points-of-view of the area, the one or more points-of-view of the area generated based on synchronizing the combination of the sensor data;

detecting, by the computing device, an occurrence of an event based on the assessment of the area;

identifying, by the computing device, one or more instructions corresponding to the event, the instructions associated with a particular vehicle; and

sending, by the computing device, one or more executable instructions based on the instructions to the particular vehicle.

18. The media of claim 17, wherein the processing of the sensor data from one or more of the vehicles further comprises:

associating the sensor data with one or more time stamps;

synchronizing the sensor data for all vehicles of the one or more vehicles based on the time stamps;

generating the one or more points-of-view of a geographic area based on the synchronized sensor data; and

assessing the one or more points-of-view of the geographic area to determine the occurrence of the event.

19. The media of claim 17, wherein the particular one vehicle that the instructions are associated with is selected by:

determining one or more of the vehicles that will make a route alteration based on the occurrence of the event;

determining a priority associated with each of the one or more vehicles; and

selecting a vehicle with the highest priority as the particular vehicle,

wherein the priority is determined based on at least one of;

a degree of route alteration associated with a vehicle due to the occurrence of the event,

a distance from a vehicle to a location of the event,

a current route of a vehicle, or

a distance from a vehicle to a location associated with the computing device.

20. The media of claim 17, wherein the occurrence of the event is detected by:

determining a relevant route associated with the one or more vehicles;

requesting a report of any events on the relevant route from the one or more vehicles or one or more other vehicles; and

receiving the report of events on the relevant route.