Autonomous operation capability configuration for a vehicle

• US 10,235,881 B2
• Filed: 07/28/2017
• Issued: 03/19/2019
• Est. Priority Date: 07/28/2017
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toyota Chuo Kenkyusho (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Nishi, Tomoki
• Primary Examiner(s): Paige, Tyler D
• Application Number:
  US15/663,245
• Publication Number:
  US 20190035275A1
• Time in Patent Office: 599 Days
• Field of Search: 701301
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2205/01 specially adapted for specific applications
• CPC Class Codes: G01S 5/0072 Transmission between mobile stations, e.g. anti-collision systems
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/0291 Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/161 Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method and device for configuring an autonomous operation capability of a vehicle are disclosed. A first dataset is identified relating to a plurality of vehicles, wherein the vehicle is one of the plurality of vehicles. A state value function is produced for the autonomous vehicle capability in relation to the first dataset. A second dataset is identified relating to a portion of the plurality of vehicles, wherein the portion of the plurality of vehicles includes the vehicle. A policy control gain is optimized for the autonomous vehicle capability in relation to the second dataset. The autonomous vehicle capability is operable to generate an autonomous vehicle action for progressing to a next state based on the state value function in cooperation with the policy control gain.

Claims

1. A method for configuring an autonomous operation capability of a vehicle, the method comprising:

identifying, by at least one processor, a first dataset relating to a plurality of vehicles, wherein the vehicle is one of the plurality of vehicles and the first dataset is a passive environmental dynamic dataset including one of simulation data and data previously collected from one or more sensors of the vehicle and stored in a memory, the first dataset including present state data and next state data, the one or more sensors including at least one of an image sensor, a Light Detection and Ranging (LIDAR) sensor, and a radar sensor;

producing, by the at least one processor, a state value function for the autonomous vehicle capability in relation to the first dataset, the state value function including a present state, a next state, and a state cost, the state cost relating to one or more performance characteristics of the vehicle;

identifying, by the at least one processor, a second dataset relating to a portion of the plurality of vehicles, wherein the portion of the plurality of vehicles includes the vehicle and the second dataset is a control dynamics model stored in the memory, the second dataset including state cost data and control dynamics data; and

optimizing, by the at least one processor, a policy control gain for the autonomous vehicle capability in relation to the second dataset;

wherein the autonomous vehicle capability is operable to generate an autonomous vehicle action for progressing to a next state based on the state value function in cooperation with the policy control gain and the autonomous vehicle capability includes a reinforcement learning (RL) module including an actor-critic module to reduce active environmental scanning by the one or more sensors as the vehicle operates autonomously.

2. The method of claim 1, wherein the portion of the plurality of vehicles includes a vehicle class.

3. The method of claim 1, wherein the portion of the plurality of vehicles includes the vehicle.

4. The method of claim 1, wherein the actor-critic module includes:

an actor module operable to generate a prediction of an action of the autonomous vehicle capability based on an adaptive policy generated from the first dataset and the second dataset; and

a critic module operable to produce feedback data for the actor module relating to a reward parameter responsive to the first dataset and the second dataset, wherein the critic module, via the feedback data, further operates to modify a policy parameter of the actor module to form the adaptive policy.

5. The method of claim 4, wherein the feedback data includes temporal displacement error data.

6. A method for configuring an autonomous vehicle capability of a vehicle, the method comprising:

identifying, by at least one processor, the vehicle with a plurality of vehicles;

retrieving, by the at least one processor, a first dataset relating to a plurality of vehicles, wherein the first dataset is a passive environmental dynamic dataset including one of simulation data and data previously collected from one or more sensors of the vehicle and stored in a memory, the first dataset including present state data and next state data, the one or more sensors including at least one of an image sensor, a Light Detection and Ranging (LIDAR) sensor, and a radar sensor;

producing, by the at least one processor, a state value function for the autonomous vehicle capability based on the first dataset, the state value function including a present state, a next state, and a state cost, the state cost relating to one or more performance characteristics of the vehicle;

identifying, by the at least one processor, a second dataset relating to a portion of the plurality of vehicles wherein the portion of the plurality of vehicles includes the vehicle and the second dataset is a control dynamics model stored in the memory, the second dataset including state cost data and control dynamics data; and

optimizing, by the at least one processor, a policy control gain for the autonomous vehicle capability in relation to the second dataset;

wherein the autonomous vehicle capability is operable to generate an autonomous vehicle action for progressing to a next state based on the state value function in cooperation with the policy control gain and the autonomous vehicle capability includes a reinforcement learning (RL) module including an actor-critic module to reduce active environmental scanning by the one or more sensors as the vehicle operates autonomously.

7. The method of claim 6, wherein the portion of the plurality of vehicles includes a vehicle class.

8. The method of claim 6, wherein the portion of the plurality of vehicles includes a single make and model of a vehicle class.

9. The method of claim 6, wherein the control dynamics model includes a plurality of operational modes for the portion of the plurality of vehicles.

10. The method of claim 6, wherein the actor-critic module includes:

an actor module operable to generate a prediction of an action of the autonomous vehicle capability based on an adaptive policy generated from the first dataset and the second dataset; and

a critic module operable to produce feedback data for the actor module relating to a reward parameter responsive to the first dataset and the second dataset, wherein the critic module, via the feedback data, further operates to modify a policy parameter of the actor module to form the adaptive policy.

11. The method of claim 10, wherein the feedback data includes temporal displacement error data.

12. A vehicle control unit being configured for an autonomous vehicle capability for a vehicle, the vehicle control unit comprising:

a wireless communication interface to service communication with a vehicle network;

a processor communicably coupled to the wireless communication interface and to a plurality of vehicle sensor devices; and

a memory communicably coupled to the processor and storing;

a reinforcement learning module including instructions that, when executed by the processor, cause the processor to configure the reinforcement learning module for the autonomous vehicle capability by;

accessing a passive environmental dynamic dataset relating to a plurality of vehicles, wherein the vehicle is one of the plurality of vehicles and the passive environmental dynamic dataset includes one of simulation data and data previously collected from one or more sensors of the vehicle, the passive environmental dynamic dataset including present state data and next state data, the one or more sensors including at least one of an image sensor, a Light Detection and Ranging (LIDAR) sensor, and a radar sensor;

producing a state value function for the autonomous vehicle capability in relation to the passive environmental dynamic dataset, the state value function including a present state, a next state, and a state cost, the state cost relating to one or more performance characteristics of the vehicle;

identifying a control dynamics model relating to a portion of the plurality of vehicles, wherein the portion of the plurality of vehicles includes the vehicle, the control dynamics model including state cost data and control dynamics data; and

optimizing a policy control gain for the autonomous vehicle capability in relation to the second dataset; and

wherein the autonomous vehicle capability is operable to generate an autonomous vehicle action for progressing to a next state based on the state value function in cooperation with the policy control gain and the autonomous vehicle capability includes a reinforcement learning (RL) module including an actor-critic module to reduce active environmental scanning by the one or more sensors as the vehicle operates autonomously.

13. The vehicle control unit of claim 12, wherein the

actor-critic module includes instructions that, when executed by the processor, cause the processor to form the adaptive policy by:

optimizing a vehicular policy to form the adaptive policy; and

generating a prediction for the autonomous vehicle action based on the adaptive policy.

14. The vehicle control unit of claim 13, wherein the actor-critic module includes:

an actor module including instructions that, when executed by the processor, cause the processor to generate the prediction for the autonomous vehicle action based on the adaptive policy; and

a critic module including instructions that, when executed by the processor, cause the processor to produce feedback data for the actor module relating to a reward parameter responsive to the vehicular action, wherein the critic module, via the feedback data, further operates to modify a policy parameter of the actor module to form the adaptive policy.

15. The vehicle control unit of claim 14, wherein the feedback data includes temporal displacement error data.

16. The vehicle control unit of claim 12, wherein the portion of the plurality of vehicles includes a vehicle class.