Neural network system for autonomous vehicle control

• US 10,452,068 B2
• Filed: 10/17/2016
• Issued: 10/22/2019
• Est. Priority Date: 10/17/2016
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Djuric, Nemanja, Houston, John
• Primary Examiner(s): Antonucci, Anne M
• Assistant Examiner(s): Stroud, James E
• Application Number:
  US15/295,088
• Publication Number:
  US 20180107215A1
• Time in Patent Office: 1,100 Days
• Field of Search: 701 27
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/3407 specially adapted for specific applications
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A neural network may be utilized for autonomously driving a self-driving vehicle (SDV). The neural network can establish a destination location in local coordinates relative to the SDV. The neural network may then identify one or more navigation points in a forward operational direction of the SDV, and process sensor data from a sensor system of the SDV, the sensor data providing a sensor view of the forward operational direction of the SDV. Utilizing the sensor data, the neural network can operate acceleration, braking, and steering systems of the SDV to continuously follow the one or more navigation points along an established route to the destination location.

Claims

1. A neural network system for autonomous control of a self-driving vehicle (SDV), the neural network system comprising:

one or more processors; and

one or more memory resources storing a machine learning model that, when executed by the one or more processors, cause the neural network system to;

establish a destination location in local coordinates relative to the SDV;

identify one or more navigation points in a forward operational direction of the SDV;

process sensor data from a sensor system of the SDV, the sensor data providing a sensor view of the forward operational direction of the SDV;

determine, using a location-based resource, a current position of the SDV, wherein the one or more navigation points are computed based on the current position of the SDV and the established route to the destination location; and

utilizing the sensor data, operate acceleration, braking, and steering systems of the SDV to continuously follow the one or more navigation points along an established route to the destination location;

wherein noise is incorporated into location signals corresponding to the one or more navigation points, and wherein the noise causes the neural network system to rely on processing the sensor data in conjunction with continuously following the one or more navigation points.

2. The neural network system of claim 1, wherein the executed machine learning model causes the neural network system to identify each of the one or more navigation points at (i) a constant distance ahead of the SDV along the established route, or (ii) a temporal location ahead of the SDV, based on a current speed of the SDV, along the established route.

3. The neural network system of claim 1, wherein the one or more navigation points each comprises a coordinate point in global coordinates, the coordinate point having values that vary as the SDV progresses towards the destination location, and wherein the executed machine learning model causes the neural network system to continuously follow the one or more navigation points along the established route to the destination location by continuously comparing the values of the coordinate point with vehicle coordinates of the SDV.

4. The neural network system of claim 1, wherein the one or more navigation points comprise a plurality of navigation points established at differing distances ahead of the SDV along the established route.

5. The neural network system of claim 4, wherein the executed machine learning model causes the neural network system to (i) utilize the plurality of navigation points to dynamically determine an immediate route plan, and (ii) analyze the sensor data to execute control actions on the acceleration, braking, and steering systems of the SDV in order to dynamically implement the immediate route plan.

6. A self-driving vehicle (SDV) comprising:

a sensor system to detect a situational environment of the SDV;

acceleration, braking, and steering systems; and

a control system comprising a neural network implementing a machine learning model that causes the control system to;

establish a destination location in local coordinates relative to the SDV;

identify one or more navigation points in a forward operational direction of the SDV;

process sensor data from the sensor system of the SDV, the sensor data providing a sensor view of the forward operational direction of the SDV;

determine, using a location-based resource, a current position of the SDV, wherein the one or more navigation points are computed based on the current position of the SDV and the established route to the destination location; and

utilize the sensor data, operate acceleration, braking, and steering systems of the SDV to continuously follow the one or more navigation points along an established route to the destination location;

wherein noise is incorporated into location signals corresponding to the one or more navigation points, and wherein the noise causes the neural network system to rely on processing the sensor data in conjunction with continuously following the one or more navigation points.

7. The SDV of claim 6, wherein the machine learning model implemented by the neural network causes the control system to identify each of the one or more navigation points at (i) a constant distance ahead of the SDV along the established route, or (ii) a temporal location ahead of the SDV, based on a current speed of the SDV, along the established route.

8. The SDV of claim 6, wherein the one or more navigation points each comprises a coordinate point in global coordinates, the coordinate point having values that vary as the SDV progresses towards the destination location, and wherein the machine learning model implemented by the neural network causes the control system to continuously follow the one or more navigation points along the established route to the destination location by continuously comparing the values of the coordinate point with vehicle coordinates of the SDV.

9. The SDV of claim 6, wherein the one or more navigation points comprise a plurality of navigation points established at differing distances ahead of the SDV along the established route.

10. The SDV of claim 9, wherein the machine learning model implemented by the neural network causes the control system to (i) utilize the plurality of navigation points to dynamically determine an immediate route plan, and (ii) analyze the sensor data to execute control actions on the acceleration, braking, and steering systems of the SDV in order to dynamically implement the immediate route plan.

11. A computer implemented method of autonomously operating a vehicle, the method being performed by one or more processors of a neural network system of a self-driving vehicle (SDV) and comprising:

establishing a destination location in local coordinates relative to the SDV;

identifying one or more navigation points in a forward operational direction of the SDV;

processing sensor data from a sensor system of the SDV, the sensor data providing a sensor view of the forward operational direction of the SDV;

determining, using a location-based resource, a current position of the SDV, wherein the one or more navigation points are computed based on the current position of the SDV and the established route to the destination location; and

utilizing the sensor data, operating acceleration, braking, and steering systems of the SDV to continuously follow the one or more navigation points along an established route to the destination location;

wherein noise is incorporated into location signals corresponding to the one or more navigation points, and wherein the noise causes the neural network system to rely on processing the sensor data in conjunction with continuously following the one or more navigation points.

12. The method of claim 11, wherein the neural network system identifies each of the one or more navigation points at (i) a constant distance ahead of the SDV along the established route, or (ii) a temporal location ahead of the SDV, based on a current speed of the SDV, along the established route.

13. The method of claim 11, wherein the one or more navigation points each comprises a coordinate point in global coordinates, the coordinate point having values that vary as the SDV progresses towards the destination location, and wherein the neural network system continuously follows the one or more navigation points along the established route to the destination location by continuously comparing the values of the coordinate point with vehicle coordinates of the SDV.

14. The method of claim 11, wherein the one or more navigation points comprise a plurality of navigation points established at differing distances ahead of the SDV along the established route.