Hybrid vehicle fuel efficiency using inverse reinforcement learning

• US 9,090,255 B2
• Filed: 03/15/2013
• Issued: 07/28/2015
• Est. Priority Date: 07/12/2012
• Status: Expired due to Fees

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Gupta, Rakesh, Ramachandran, Deepak, Vogel, Adam C., Raux, Antoine
• Primary Examiner(s): HOLLOWAY, JASON R
• Application Number:
  US13/838,922
• Publication Number:
  US 20140018985A1
• Time in Patent Office: 865 Days
• Field of Search: 701/22, 701/99, 701/123, 701/424, 701/521, 340/995.18, 340/995.19, 340/995.27, 180/65.265, 903/930
• US Class Current: 1/1
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/26 for electrical energy, e.g. batteries or capacitors
• CPC Class Codes: B60W 20/12 using control strategies taking into account route information estimation or calculation of non-directly measurable driving parameters B60W40/00
• CPC Class Codes: B60W 2552/20 Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments
• CPC Class Codes: B60W 2556/00 Input parameters relating to data
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: G01C 21/3469 Fuel consumption; Energy use; Emission aspects
• CPC Class Codes: Y10S 903/93 Conjoint control of different elements

Abstract

A powertrain of a hybrid electric vehicle (HEV) is controlled. A first value α₁ and a second value α₂ are determined. α₁ represents a proportion of an instantaneous power requirement (P_req) supplied by an engine of the HEV. α₂ controls a recharging rate of a battery of the HEV. A determination is performed, based on α₁ and α₂, regarding how much engine power to use (P_eng) and how much battery power to use (P_batt). P_eng and P_batt are sent to the powertrain.

Claims

1. A computer based method for controlling a powertrain of a hybrid electric vehicle (HEV) having an engine and a battery, comprising steps of:

collecting route information about a plurality of driving routes, wherein each driving route is comprised of a plurality of route segments connected by a plurality of intersections;

predicting a probability distribution over possible future route segments of multiple possible routes to be traversed by the HEV based on the collected route information;

computing a first value α

₁ and a second value α

₂, wherein α

₁ represents a proportion of an instantaneous power requirement (P_req) supplied by an engine of the HEV, and α

₂ controls a recharging rate of a battery of the HEV, such that an expected energy expenditure over the probability distribution is reduced;

determining, based on α

₁ and α

₂, how much engine power to use (P_eng) and how much battery power to use (P_batt); and

operating the powertrain according to P_eng and P_batt.

2. The method of claim 1, further comprising repeating the steps of claim 1 after a time period has elapsed.

3. The method of claim 1, wherein predicting the probability distribution over possible future paths of the HEV is performed based on past driver history.

4. The method of claim 3, wherein past driver history the route information includes a set of action features (f) that represent aspects of each of the plurality of driving route intersections.

5. The method of claim 4, wherein f includes one or more elements of the group containing:

an identity of an outgoing road segment;

a type of a road segment;

an angle of turn between an incoming road segment and an outgoing road segment;

an elevation change of a road segment; and

a cardinal direction of a road segment.

6. The method of claim 1, wherein computing the first value α

₁ and the second value α

₂ such that the expected energy expenditure over the probability distribution is reduced comprises maximizing a value function V given by;

7. The method of claim 6, wherein the reward function R of a state (r, x, f) is equal to a sum of residual fuel energy and battery energy at the state.

8. The method of claim 1, wherein determining, based on α

₁ and α

₂, how much engine power to use (P_eng) and how much battery power to use (P_batt) comprises;

determining whether a state-of-charge of the HEV'"'"'s battery (x) is larger than a maximal relative charge allowed for the battery;

responsive to determining that x is larger than the maximal relative charge allowed for the battery;

determining that P_eng=α

₁·

P_req; and

determining that P_batt=(1−

α

₁) ·

P_req; and

responsive to determining that x is not larger than the maximal relative charge allowed for the battery;

determining that P_eng=P_req+α

₂·

P_charging; and

determining that P_batt=−

α

₂·

P_charging;

wherein P_charging is a maximum charging power capacity of the HEV'"'"'s battery.

17. The method of claim 1, wherein determining a driver model using an inverse reinforcement learning algorithm further comprises representing the route information as a Markov decision process having states s each state having a plurality of actions a.

18. The method of claim 17, wherein each intersection is represented as a state s_i and each route segment chosen at the state s_i is represented as an action a_i.

21. The method of claim 1, further comprising:

determining a driver model by using an inverse reinforcement learning algorithm on the collected route information; and

predicting a probability distribution over possible future route segments of the HEV based on the determined driver model given a current position of the HEV.

9. A non-transitory computer-readable storage medium storing executable computer program instructions for controlling a powertrain of a hybrid electric vehicle (HEV) having an engine and a battery, the instructions performing steps comprising:

collecting route information about a plurality of driving routes, wherein each driving route is comprised of a plurality of route segments connected by a plurality of intersections;

predicting a probability distribution over possible future route segments of multiple possible routes to traverse by the HEV based on the collected route information;

computing a first value α

₁ and a second value α

₂, wherein α

_l represents a proportion of an instantaneous power requirement (P_req) supplied by an engine of the HEV, and a₂ controls a recharging rate of a battery of the HEV, such that an expected energy expenditure over the probability distribution is reduced;

determining, based on α

_l and α

₂, how much engine power to use (P_eng) and how much battery power to use (P_batt); and

operating the powertrain according to P_eng and P_batt.

10. The computer-readable storage medium of claim 9, wherein the instructions perform steps further comprising repeating the steps of claim 9 after a time period has elapsed.

11. The computer-readable storage medium of claim 9, wherein the route information includes a set of action features (f) that represent aspects of each of the plurality of intersections.

12. The computer-readable storage medium of claim 11, wherein f includes one or more elements of the group containing:

an identity of an outgoing road segment;

a type of a road segment;

an angle of turn between an incoming road segment and an outgoing road segment;

an elevation change of a road segment; and

a cardinal direction of a road segment.

13. The computer-readable storage medium of claim 9, wherein computing the first value α

₁ and the second value α

₂ such that the expected energy expenditure over the probability distribution is reduced comprises maximizing a value function V given by;

14. The computer-readable storage medium of claim 13, wherein the reward function R of a state (r, x, f) is equal to a sum of residual fuel energy and battery energy at the state.

15. The computer-readable storage medium of claim 9, wherein determining, based on α

₁ and α

₂, how much engine power to use (P_eng) and how much battery power to use (P_batt) comprises;

determining whether a state-of-charge of the HEV'"'"'s battery (x) is larger than a maximal relative charge allowed for the battery;

responsive to determining that x is larger than the maximal relative charge allowed for the battery;

determining that P_eng =α

₁·

P_req; and

determining that P_batt =(1−

α

₁)·

P_req; and

responsive to determining that x is not larger than the maximal relative charge allowed for the battery;

determining that P_eng=P_req+α

₂ ·

P_charging; and

determining that P<sub>batt =−

α

2</sub> ·

P_charging;

wherein P_charging is a maximum charging power capacity of the HEV'"'"'s battery.

19. The computer-readable storage medium of claim 9, wherein determining a driver model using an inverse reinforcement learning algorithm further comprises representing the route information as a Markov decision process having states s each state having a plurality of actions a.

20. The computer-readable storage medium of claim 19, wherein each intersection is represented as a state s_i and each route segment chosen at the state s_i is represented as an action a_i.

22. The computer-readable storage medium of claim 9, performing steps further comprising:

determining a driver model by using an inverse reinforcement learning algorithm on the collected route information; and

predicting a probability distribution over possible future route segments of the HEV based on the determined driver model given a current position of the HEV.

16. A system for controlling a powertrain of a hybrid electric vehicle (HEV) having an engine and a battery, the system comprising:

at least one non-transitory computer-readable storage medium storing executable computer program instructions comprising instructions for;

collecting route information about a plurality of driving routes, wherein each driving route is comprised of a plurality of route segments connected by a plurality of intersections;

predicting a probability distribution over possible future route segments of the HEV based on the collected route information;

computing a first value α

₁ and a second value α

₂, wherein α

₁ represents a proportion of an instantaneous power requirement (P_req) supplied by an engine of the HEV, and α

₂ controls a recharging rate of a battery of the HEV, such that an expected energy expenditure over the probability distribution is reduced;

determining, based on α

₁ and α

₂, how much engine power to use (P_eng) and how much battery power to use (P_batt); and

operating the powertrain according to P_eng and P_batt; and

a processor for executing the computer program instructions.