Autonomous vehicle advanced sensing and response

• US 10,332,320 B2
• Filed: 04/17/2017
• Issued: 06/25/2019
• Est. Priority Date: 04/17/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Lakshamanan, Barath, Hurd, Linda L., Ashbaugh, Ben J., Ould-Ahmed-Vall, Elmoustapha, Ma, Liwei, Jin, Jingyi, Gottschlich, Justin E., Sakthivel, Chandrasekaran, Strickland, Michael S., Lewis, Brian T., Kuper, Lindsey, Koker, Altug, Appu, Abhishek R., Surti, Prasoonkumar, Ray, Joydeep, Vembu, Balaji, Turek, Javier S., Farooqui, Naila
• Primary Examiner(s): Patel, Shardul D
• Application Number:
  US15/488,914
• Publication Number:
  US 20180300964A1
• Time in Patent Office: 799 Days
• Field of Search: 701 24
• CPC Class Codes: B60W 2050/0006 Digital architecture hierarchy
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G01C 21/3492 employing speed data or traffic data, e.g. real-time or historical traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/012 : from other sources than vehicle or roadside beacons, e.g. mobile networks
• CPC Class Codes: G08G 1/052 : with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: H04L 43/0852 : Delays
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/40 : for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

One embodiment provides for a computing device within an autonomous vehicle, the compute device comprising a wireless network device to enable a wireless data connection with an autonomous vehicle network, a set of multiple processors including a general-purpose processor and a general-purpose graphics processor, the set of multiple processors to execute a compute manager to manage execution of compute workloads associated with the autonomous vehicle, the compute workload associated with autonomous operations of the autonomous vehicle, and offload logic configured to execute on the set of multiple processors, the offload logic to determine to offload one or more of the compute workloads to one or more autonomous vehicles within range of the wireless network device.

Claims

1. A computing device within an autonomous vehicle, the computing device comprising:

a wireless network device to enable a wireless data connection with an autonomous vehicle network; and

a set of multiple processors including a general-purpose processor, a general-purpose graphics processor, and a vision processor, the set of multiple processors to execute compute workloads that enable autonomous driving and navigation for the autonomous vehicle, the vision processor to accelerate processing of real-time sensory data, wherein one or more processors in the set of multiple processors is to determine to offload one or more of the compute workloads to a remote computing resource accessible via the autonomous vehicle network in response to a determination that a number of compute workloads exceeds a compute capacity of the set of multiple processors, the remote computing resource including one or more autonomous vehicles within range of the wireless network device or a remote datacenter connected to the autonomous vehicle network, wherein the wireless network device is to receive a result of one or more offloaded compute workloads and perform an autonomous driving or navigation operation based on the result.

2. The computing device as in claim 1, wherein a set of compute workloads having common dependencies is to be scheduled as a unit to the remote computing resource.

3. The computing device as in claim 2, wherein the set of multiple processors is to determine to offload one or more of the compute workloads to one of the one or more autonomous vehicles within range of the wireless network device and the remote datacenter via the autonomous vehicle network based on factors including computational demand and communication latency.

4. The computing device as in claim 3, wherein the set of multiple processors, in response to a notice that a computation workload is to be offloaded, is to determine whether a first latency to a cloud datacenter is below a first threshold and offload the compute workload to the cloud datacenter in response to a determination that the first latency is below the first threshold.

5. The computing device as in claim 3, wherein the set of multiple processors, in response to a notice that a compute workload is to be offloaded, is to determine whether a second latency to one of the one or more autonomous vehicles within range of the wireless network device is below a second threshold and offload the compute workload to the autonomous vehicle associated with the second latency in response to a determination that the second latency is below the second threshold.

6. The computing device as in claim 5, wherein the set of multiple processors is to determine whether compute resources available at the autonomous vehicle associated with the second latency are sufficient to process the compute workload to be offloaded before the compute workload is offloaded to the autonomous vehicle associated with the second latency.

7. The computing device as in claim 1, wherein the set of multiple processors is to determine that a secure and trusted relationship exists with the one or more autonomous vehicles within range of the wireless network device before the set of multiple processors is to determine to offload one or more of the compute workloads to the one or more autonomous vehicles.

8. The computing device as in claim 1, wherein the set of multiple processors are to receive a compute workload offloaded from a neighboring autonomous vehicle.

9. The computing device as in claim 8, wherein the set of multiple processors receive sensor data from a set of connected sensors and the compute workload offloaded from the neighboring autonomous vehicle includes a search operation to be performed based on received sensor data.

10. The computing device as in claim 9, wherein the search operation includes to locate a target of a search within a sensor detection area of an autonomous vehicle sensor.

11. A method of performing machine learning operations for an autonomous vehicle, the method comprising:

determining that a computational workload is to be processed by a computing device of the autonomous vehicle, the computational workload to enable autonomous driving and navigation for the autonomous vehicle, wherein the computing device of the autonomous vehicle includes a vision processor to accelerate processing of real-time sensory data;

determining a first latency to a remote datacenter via an autonomous vehicle network;

dispatching at least a first portion of the computational workload for processing via the remote datacenter in response to determining that the first latency is below a threshold associated with the computational workload;

determining a second latency to an autonomous vehicle within range of wireless network device in response to determining that the first latency is above the threshold associated with the computational workload;

dispatching at least a second portion of the computational workload in response determining that the second latency is below the threshold associated with the computational workload;

receiving an offload result of at least the first portion or the second portion of the computational workload; and

performing an autonomous driving or navigation operation based on the result.

12. The method as in claim 11, additionally comprising executing the computational workload using local compute resources of the autonomous vehicle in response to determining that the second latency is above the threshold associated with the computational workload.

13. The method as in claim 12, additionally comprising limiting autonomous functionality of the autonomous vehicle when executing the computational workload using local resources.

14. The method as in claim 13, wherein limiting autonomous functionality of the autonomous vehicle includes reducing a maximum autonomous travel seed for the autonomous vehicle.

15. The method as in claim 13, wherein limiting autonomous functionality of the autonomous vehicle includes reducing speed at which the autonomous vehicle selects alternative routes based on dynamic traffic conditions.

16. An autonomous vehicle comprising:

a wireless network device to enable a wireless data connection with an autonomous vehicle network; and

a set of multiple processors including a general-purpose processor, a general-purpose graphics processor, and a vision processor, the set of multiple processors to execute a compute manager to manage execution of compute workloads associated with the autonomous vehicle, the vision processor to accelerate processing of real-time sensory data, the compute manager to determine to offload one or more compute workloads to a remote computing resource accessible via the autonomous vehicle network in response to a determination that a number of compute workloads exceeds a compute capacity of the set of multiple processors, the remote computing resource including one or more autonomous vehicles within range of the wireless network device or a remote datacenter connected to the autonomous vehicle network, wherein the wireless network device is to receive a result of one or more offloaded compute workloads and perform an autonomous driving or navigation operation based on the result.

17. The autonomous vehicle as in claim 16, wherein the compute manager is additionally to schedule a set of compute workloads having common dependencies to the remote computing resource.

18. The autonomous vehicle as in claim 17, wherein the compute manager is to determine to offload one or more of the compute workloads to one of the one or more autonomous vehicles within range of the wireless network device and the remote datacenter via the autonomous vehicle network based on factors including computational demand and communication latency.

19. The autonomous vehicle as in claim 18, wherein the compute manager, in response to a notice that a computation workload is to be offloaded, is to determine whether a first latency to a cloud datacenter is acceptable for the compute workload to be offloaded and offload the compute workload to the cloud datacenter in response to a determination that the first latency is within an acceptable range.

20. The autonomous vehicle as in claim 18, wherein the compute manager, in response to a notice that a compute workload is to be offloaded is to determine whether a second latency to one of the one or more autonomous vehicles within range of the wireless network device is acceptable and offload the compute workload to the autonomous vehicle associated with the second latency.