Vehicle control through machine learning

• US 10,393,880 B2
• Filed: 09/10/2015
• Issued: 08/27/2019
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Volvo Car Corporation (AB Volvo)
• Original Assignee: Volvo Car Corporation (AB Volvo)
• Inventors: Arvidsson, Rickard, Akerblom, Niklas
• Primary Examiner(s): Smith, Isaac G
• Application Number:
  US14/850,037
• Publication Number:
  US 20160075341A1
• Time in Patent Office: 1,447 Days
• Field of Search: 701 25, 701 22, 701424
• CPC Class Codes: B60K 6/46 Series type
• CPC Class Codes: B60W 2050/0014 Adaptive controllers
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: Y02T 10/62 Hybrid vehicles
• CPC Class Codes: Y10S 903/903 having energy storing means, e.g. battery, capacitor

Abstract

A method is disclosed for managing rules or policies in a vehicle having a controllable unit. The method may include determining, at the start of a driving, an active route of the vehicle, selecting a set of control values from multiple stored sets of control values, where one control value of the set of control values corresponds to one predetermined interval of the determined active route, and controlling the at least one controllable unit based on the selected set of control values. The method may also include recording, during the current driving of the vehicle, a set of control result values, where one control result value corresponds to a predetermined interval of the determined active route which the vehicle has driven and, at the end of the driving based on the set of control result values, determining and storing an updated set of control values for the actually driven route.

Claims

1. A method for managing rules or policies in a vehicle, the vehicle comprising at least one controllable unit, the method performed by a management unit and a memory unit associated with the management unit, the method comprising:

determining, at the start of a driving, an active route of the vehicle;

dividing the determined active route into a plurality of non-overlapping predetermined intervals,

selecting a set of control values from a plurality of stored sets of control values, the plurality of stored sets of control values being based on a set of previously driven routes, where the selected set of control values corresponds to the determined active route, wherein each control value of the selected set of control values corresponds to one of the plurality of predetermined intervals of the determined active route;

controlling the at least one controllable unit based on the selected set of control values;

recording, during current driving of the vehicle, a set of control result values, wherein each control result value corresponds to one of the plurality of predetermined intervals of the determined active route which the vehicle has driven, and wherein the control result values are indicative of a result of the controlling during the corresponding predetermined interval;

determining, at the end of the driving based on the set of control result values, an updated set of control values for an actually driven route; and

storing the updated set of control values, wherein the set of control values corresponds to a control model;

wherein the at least one controllable unit of the vehicle is an energy unit, the rules or policies relate to managing energy usage of the energy unit, the control values and sets of control values respectively are energy unit control values and sets of energy unit control values relating to the control of the energy unit of the vehicle, the set of control result values are energy unit control result values indicative of the energy usage during the corresponding predetermined interval, and the updated set of control values is an updated set of energy unit control values; and

wherein the vehicle is a Hybrid Electric Vehicle, the vehicle further comprises an electric propulsion unit and a battery unit adapted to power the electric propulsion unit, and the at least one energy unit is an internal combustion engine arranged to charge the battery unit, and wherein controlling the internal combustion engine is based on the selected set of energy unit control values;

wherein the control model comprises a plurality of vehicle states and a plurality of transitions between the vehicle states, each transition corresponding to one of the plurality of predetermined intervals, wherein each predetermined interval is associated with an expected energy unit control result value that is determined based on the recorded set of energy unit control result values, wherein each vehicle state contains data regarding the total expected future energy unit control result value based on entering that state, and wherein each transition corresponding to a predetermined interval is selected based on all of the determined active route remaining.

2. The method according to claim 1 further comprising:

transmitting the set of control result values to a remote data unit; and

receiving the updated control values from the remote data unit;

wherein determining, based on the set of control result values, an updated set of control values for the actually driven route, is performed at the remote data unit.

3. The method according to claim 1 where the set of control result values further includes at least one of:

Driver Power Request levels, Propulsion Energy Needs, GPS positions, Speed Levels or Timestamps.

4. The method according to claim 3 where controlling the at least one controllable unit corresponds to a transition in the control model.

5. The method according to claim 1 where recording a set of control result values corresponds to a transition in the control model.

10. The method according to claim 1 where the control model comprises a Markov Decision Process, a Continuous-Time Markov Process and/or a Transition Rate Matrix.

6. A system for managing rules and policies in a vehicle, where the vehicle comprises at least one controllable unit, the system comprising:

a management unit; and

a memory unit associated with the management unit;

wherein the system is configured to;

determine an active route of the vehicle at the start of a driving of the vehicle;

divide the determined active route into a plurality of non-overlapping predetermined intervals,

select a set of control values from a plurality of sets of control values being based on a set of previously driven routes and stored in the memory unit associated with the management unit, where the selected set of control values corresponds to the determined active route, wherein each control value of the selected set of control values corresponds to one of the plurality of predetermined intervals of the determined active route;

control the at least one controllable unit based on the selected set of control values;

record, at a predetermined interval, a set of control result values relevant to current driving of the vehicle in the memory unit associated with the management unit, wherein each control result value corresponds to one of the plurality of predetermined intervals of the determined active route which the vehicle has driven, and wherein the control result values are indicative of a result of controlling the at least one controllable unit during the corresponding predetermined interval;

determine an updated set of control values for an actually driven route at the end of the driving based on the recorded set of control values; and

store the updated control values in the memory unit associated with the management unit, wherein the set of control values corresponds to a control model;

wherein the at least one controllable unit of the vehicle is an energy unit, the rules or policies relate to managing energy usage of the energy unit, the control values and sets of control values respectively are energy unit control values and sets of energy unit control values relating to the control of the energy unit of the vehicle, the set of control result values are energy unit control result values indicative of the energy usage during the corresponding predetermined interval, and the updated set of control values is an updated set of energy unit control values; and

wherein the vehicle is a Hybrid Electric Vehicle, the vehicle further comprises an electric propulsion unit and a battery unit adapted to power the electric propulsion unit, and the at least one energy unit is an internal combustion engine arranged to charge the battery unit, and wherein the energy management unit is arranged to control the internal combustion engine based on the selected set of energy unit control values;

wherein the control model comprises a plurality of vehicle states and a plurality of transitions between the vehicle states, each transition corresponding to one of the plurality of predetermined intervals, wherein each predetermined interval is associated with an expected energy unit control result value that is determined based on the recorded set of energy unit control result values, wherein each vehicle state contains data regarding the total expected future energy unit control result value based on entering that state, and wherein each transition corresponding to a predetermined interval is selected based on all of the determined active route remaining.

7. The system according to claim 6, the system arranged to communicate with a remote data unit, the system further comprising:

a transmitter arranged to transmit, to the remote data unit, the set of control result values; and

a receiver arranged to receive, from the remote data unit, the updated set of control values;

where the determined updated set of control values for the actually driven route at the end of the driving based on the recorded set of control result values is performed at the remote data unit.

8. The system according to claim 6 where the vehicle further comprises a GPS unit arranged to receive GPS positions, and where the system is arranged to determine that the active route has changed based on the GPS positions received by the GPS unit, whereupon the system determines a new active route of the vehicle.

11. The system according to claim 6 where the control model comprises a Markov Decision Process, a Continuous-Time Markov Process and/or a Transition Rate Matrix.

9. A vehicle comprising:

at least one controllable unit; and

a system for managing rules and policies in the vehicle, the system comprising a management unit, and a memory unit associated with the management unit, wherein the system is configured to;

determine an active route of the vehicle at the start of a driving of the vehicle,

divide the determined active route into a plurality of non-overlapping predetermined intervals,

select a set of control values from a plurality of sets of control values being based on a set of previously driven routes and stored in the memory unit associated with the management unit, where the selected set of control values corresponds to the determined active route, wherein each control value of the selected set of control values corresponds to one of the plurality of predetermined intervals of the determined active route,

control the at least one controllable unit based on the selected set of control values,

record, at a predetermined interval, a set of control result values relevant to current driving of the vehicle in the memory unit associated with the management unit, wherein each control result value corresponds to one of the plurality of predetermined intervals of the determined active route which the vehicle has driven, and wherein the control result values are indicative of a result of control of the at least one controllable unit during the corresponding predetermined interval,

determine an updated set of control values for an actually driven route at the end of the driving based on the recorded set of control values, and

store the updated control values in the memory unit associated with the management unit, wherein the set of control values corresponds to a control model,

wherein the at least one controllable unit of the vehicle is an energy unit, the rules or policies relate to managing energy usage of the energy unit, the control values and sets of control values respectively are energy unit control values and sets of energy unit control values relating to the control of the energy unit of the vehicle, the set of control result values are energy unit control result values indicative of the energy usage during the corresponding predetermined interval, and the updated set of control values is an updated set of energy unit control values, and

wherein the vehicle is a Hybrid Electric Vehicle, the vehicle further comprises an electric propulsion unit and a battery unit adapted to power the electric propulsion unit, and the at least one energy unit is an internal combustion engine arranged to charge the battery unit, and wherein the energy management unit is arranged to control the internal combustion engine based on the selected set of energy unit control values,

wherein the control model comprises a plurality of vehicle states and a plurality of transitions between the vehicle states, each transition corresponding to one of the plurality of predetermined intervals, wherein each predetermined interval is associated with an expected energy unit control result value that is determined based on the recorded set of energy unit control result values, wherein each vehicle state contains data regarding the total expected future energy unit control result value based on entering that state, and wherein each transition corresponding to a predetermined interval is selected based on all of the determined active route remaining.

12. The vehicle according to claim 9 where the control model comprises a Markov Decision Process, a Continuous-Time Markov Process or a Transition Rate Matrix.