TELEASSISTANCE DATA PRIORITIZATION FOR SELF-DRIVING VEHICLES

• US 20180257643A1
• Filed: 03/07/2017
• Published: 09/13/2018
• Est. Priority Date: 03/07/2017
• Status: Active Grant

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Uber Technologies, Inc.
• Inventors: Kroop, Benjamin, Ross, William, Heine, Andrew
• Granted Patent: US 10,293,818 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/45 Pedestrian sidewalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/029 Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
• CPC Class Codes: B60W 60/0015 specially adapted for safety
• CPC Class Codes: B60W 60/00253 Taxi operations
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: B60W 60/00276 : for two or more other traffic participants
• CPC Class Codes: G05D 1/0022 : characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0038 : by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0061 : for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G05D 1/0231 : using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G07C 5/008 : communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: H04W 84/04 : Large scale networks; Deep hierarchical networks
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]

Abstract

A self-driving vehicle can operate by analyzing a live sensor view to autonomously operate acceleration, braking, and steering systems of the SDV along a current route. Based on a detected anomaly associated with the live sensor view, the SDV can transmit a teleassistance inquiry to a backend transport system in accordance with a data prioritization scheme. The SDV can receive a resolution response from the backend transport system based on the teleassistance inquiry, and proceed in accordance with the resolution response.

Claims

1. A self-driving vehicle (SDV) comprising:

a communication interface;

a sensor system generating sensor data providing a live sensor view of a situational environment of the SDV;

acceleration, braking, and steering systems; and

a control system to execute instructions that cause the control system to;

analyze the live sensor view to autonomously operate the acceleration, braking, and steering systems along a current route;

based on a detected anomaly associated with the live sensor view, transmit a teleassistance inquiry to a backend transport system in accordance with a data prioritization scheme;

receive a resolution response from the backend transport system based on the teleassistance inquiry; and

cause the SDV proceed in accordance with the resolution response.

2. The SDV of claim 1, wherein the transmitted teleassistance inquiry includes sensor data from the live sensor view, and wherein the sensor data comprises LIDAR data from at least one LIDAR sensor of the sensor system, or image data from at least one camera sensor of the sensor system.

3. The SDV of claim 2, wherein the detected anomaly corresponds to a detection hazard threshold being crossed.

4. The SDV of claim 3, wherein the detection hazard threshold corresponds to an obstruction in the live sensor view.

5. The SDV of claim 3, wherein the data prioritization scheme prioritizes transmission of the LIDAR data when the detection hazard threshold is crossed.

6. The SDV of claim 2, wherein the executed instructions further cause the control system to:

detect, in the live sensor view, objects of interest along the current route; and

attempt to classify each of the objects of interest in accordance with a certainty threshold;

wherein the detected anomaly corresponds to an indeterminate object, from the objects of interest, not meeting the certainty threshold.

7. The SDV of claim 6, wherein the data prioritization scheme prioritizes image data when the certainty threshold has not been met.

8. The SDV of claim 1, wherein data transmissions between the SDV and the backend transport system comprise a plurality of data transmission types, and wherein the data prioritization scheme associates transmission of teleassistance inquiries as a highest priority data transmission type.

9. The SDV of claim 1, wherein the communication interface comprises a plurality of wireless communication modules enabling communication with the backend transport system via a plurality of wireless communication providers, and wherein the executed instructions further cause the control system to:

in response to identifying the detected anomaly in the live sensor view, determine a highest bandwidth communication module from the plurality of wireless communication modules;

wherein the data prioritization scheme causes the control system to transmit the teleassistance inquiry to the backend transport system using the highest bandwidth communication module.

10. The SDV of claim 9, wherein the executed instructions further cause the control system to:

duplicate transmission of the teleassistance inquiry by transmitting the teleassistance inquiry over a second communication module from the plurality of wireless communication modules.

11. The SDV of claim 10, wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

12. The SDV of claim 10, wherein the highest bandwidth communication module comprises a LTE communication module and the second communication module comprises a Wi-Fi module.

13. The SDV of claim 1, wherein the teleassistance inquiry includes telemetry data for the SDV.

14. The SDV of claim 1, wherein the teleassistance inquiry includes localization information for the SDV.

15. The SDV of claim 1, wherein the teleassistance inquiry includes diagnostics data for the SDV.

16. A non-transitory computer readable medium storing instructions that, when executed by a control system of a self-driving vehicle (SDV), cause the control system to:

analyze a live sensor view of the SDV to autonomously operate acceleration, braking, and steering systems of the SDV along a current route;

based on a detected anomaly associated with the live sensor view, transmit a teleassistance inquiry to a backend transport system in accordance with a data prioritization scheme;

receive a resolution response from the backend transport system based on the teleassistance inquiry; and

cause the SDV proceed in accordance with the resolution response.

17. The non-transitory computer readable medium of claim 16, wherein the executed instructions further cause the control system to:

wherein the transmitted teleassistance inquiry includes sensor data from the live sensor view, and wherein the sensor data comprises at least one of LIDAR data from at least one LIDAR sensor of the SDV, or image data from at least one camera sensor of the SDV.

18. The non-transitory computer readable medium of claim 16, wherein data transmissions between the SDV and the backend transport system comprise a plurality of data transmission types, and wherein the data prioritization scheme associates transmission of teleassistance inquiries as a highest priority data transmission type.

19. The non-transitory computer readable medium of claim 16, wherein the communication interface comprises a plurality of wireless communication modules enabling communication with the backend transport system via a plurality of wireless communication providers, and wherein the executed instructions further cause the control system to:

in response to identifying the detected anomaly in the live sensor view, determine a highest bandwidth communication module from the plurality of wireless communication modules;

wherein the data prioritization scheme causes the control system to transmit the teleassistance inquiry to the backend transport system using the highest bandwidth communication module.

20. A computer-implemented method of operating a self-driving vehicle (SDV), the method being performed by a control system of the SDV and comprising:

analyzing a live sensor view of the SDV to autonomously operate acceleration, braking, and steering systems of the SDV along a current route;

based on a detected anomaly associated with the live sensor view, transmit a teleassistance inquiry to a backend transport system in accordance with a data prioritization scheme;

receive a resolution response from the backend transport system based on the teleassistance inquiry; and

cause the SDV proceed in accordance with the resolution response.