Machine learning for personalized driving

• US 10,286,915 B2
• Filed: 01/17/2017
• Issued: 05/14/2019
• Est. Priority Date: 01/17/2017
• Status: Active Grant

• Current Assignee: NIO Technology Company Limited (NIO, Inc. (China))
• Original Assignee: Nio Usa Inc. (NIO, Inc. (China))
• Inventors: Xiao, Fei, Pouliot, Christopher F., Liang, Anthony Tao, Wang, Guan
• Primary Examiner(s): Soofi, Yazan A
• Application Number:
  US15/408,143
• Publication Number:
  US 20180201273A1
• Time in Patent Office: 847 Days
• Field of Search: 701 23
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2050/0088 Adaptive recalibration
• CPC Class Codes: B60W 2510/18 Braking system
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/10 Accelerator pedal position
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: B60W 30/181 Preparing for stopping
• CPC Class Codes: B60W 30/18145 Cornering
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

An automated system for controlling a vehicle gathers a set of sensor information for a series of driving actions that comprises a driving path of an individual driver. A context is determined that is related to the set of sensor information for the series of driving actions that comprise the driving path of the individual driver. For example, the context may be a weather condition or when the driver merges into traffic. A personalization score is determined for the series of driving actions for the individual driver based on the context. A personalization score is determined for one or more other driving paths and a driving path with the highest personalization score is identified. The driving path with the highest personalization score is then chosen for automated driving.

Claims

1. An automated vehicle control system, comprising:

a plurality of sensors that collect sensor information when a driver is driving a vehicle manually, the collected sensor information including information associated with one or more manual driving actions provided by the driver during a drive;

a personalized driving unit that determines first context information from the sensor information and conditions associated with the vehicle at a time when the one or more manual driving actions are provided by the driver during the drive, wherein the conditions include information for vehicle settings other than settings associated with the one or more manual driving actions provided by the driver during the drive; and

an autonomous driving planning unit that is programmed to;

based on the first context information, generate first personalization information, the first personalization information mapping a first observed driver behavior to the one or more manual driving actions and the conditions, the first observed driver behavior corresponding to one or more automated driving actions that produce a vehicle behavior similar to the one or more manual driving actions when at least one condition of the conditions is detected by the plurality of sensors; and

when the vehicle is in a high or full automation level, detects an event associated with the first personalization information, wherein the event includes the at least one condition of the conditions and, in response, causes the vehicle to perform autonomously the one or more automated driving actions.

2. The control system of claim 1, wherein a personalized scoring unit determines, based on the first context information, a first personalization score for the first driver behavior, wherein the personalized driving unit receives further collected sensor information for a series of manual driving actions by the driver, the collected sensor information describing a second observed driver behavior, and determines second context information describing a second context that is related to the series of manual driving actions, wherein the autonomous driving planning unit converts the series of manual driving actions into corresponding second personalization information, the second personalization information mapping a second observed driver behavior to the series of manual driving actions, the second observed driver behavior corresponding to a series of automated driving actions that produce a vehicle behavior similar to the series of manual driving actions, wherein the personalized scoring unit determines, based on the second context information, a second personalization score for the second driver behavior, wherein the second personalization score is less than the first personalization score, wherein the event is associated with both the first and second personalization information, and wherein the autonomous driving planning unit, in response to the event, selects the first personalization information over the second personalization information.

3. The control system of claim 1, wherein the context comprises at least one:

of a specific acceleration pattern for merging into traffic, a specific acceleration pattern for passing another vehicle, a specific acceleration pattern from a traffic signal, a specific acceleration pattern for a portion of a route, a specific acceleration pattern for a hill, a specific breaking pattern for stopping at a stop sign, a specific breaking pattern for stopping at a traffic light, a specific breaking pattern for taking curves, a specific turning pattern for a right turn, a specific turning pattern for a left turn, a cruise control setting for a traffic pattern, a cruise control setting for a specific road, a time when lights are turned on or off, a time when a specific radio station is listened to, a time period for turning on turn signals, a specific seat adjustment, a specific person in the automated vehicle, a mirror adjustment time, a mirror/seat setting, or a time period for turning on window wipers and wherein the first observed driver behavior comprises at least one of the specific turning pattern for a right turn or the specific turning pattern for a left turn.

4. The control system of claim 1, wherein the first personalization information is stored in connection with an identity of the driver and wherein the first personalization information is disregarded by the autonomous driving planning unit when the driver is absent from the vehicle when the vehicle is in the high or full automation level.

5. The control system of claim 1, wherein the control system comprises different personalization information for different drivers, the different personalization information being associated with a common event.

6. The control system of claim 5, wherein the autonomous driving planning unit, in response to both the different drivers being present simultaneously in the vehicle when the vehicle is in the high or full automation level, selects, in response to the event, the one or more automated driving actions based on the different personalization information.

7. The control system of claim 6, wherein the autonomous driving planning unit selects the personalization information having a highest personalization score.

8. The control system of claim 1, wherein the autonomous driving planning unit disregards the first or second personalization information when the first or second personalization information would violate a driving restriction if executed.

18. The control system of claim 1, wherein the microprocessor determines, based on the first context information, a first personalization score for the first driving path, wherein the microprocessor receives further collected sensor information for a series of manual driving actions by the driver, the collected sensor information describing a second driving path, and determines second context information describing a second context that is related to the series of manual driving actions, wherein the microprocessor converts the series of manual driving actions into corresponding second personalization information, the second personalization information mapping a second driving path, corresponding to the series of manual driving actions, to a series of automated driving actions that produce a vehicle behavior similar to the series of manual driving actions, wherein the microprocessor determines, based on the second context information, a second personalization score for the second driving path, wherein the second personalization score is less than the first personalization score, wherein the event is associated with both the first and second personalization information, and wherein the microprocessor, in response to the event, selects the first personalization information over the second personalization information.

20. The control system of claim 18, wherein the microprocessor disregards the first or second personalization information when the first or second personalization information would violate a driving restriction if executed, wherein the first personalization information is stored in connection with an identity of the driver, and wherein the first personalization information is disregarded by the microprocessor when the driver is absent from the vehicle when the vehicle is in the high or full automation level.

21. The control system of claim 1, wherein the microprocessor comprises different personalization information for different drivers, the different personalization information being associated with a common event, wherein the microprocessor, in response to both the different drivers being present simultaneously in the vehicle when the vehicle is in the high or full automation level, selects, in response to the event, the one or more automated driving actions based on the different personalization information, wherein the microprocessor selects the personalization information having a highest personalization score, and wherein the microprocessor determines a specific type of weather or road condition, determines a personalization score based on how the driver uses a steering wheel, and/or uses a brake during the specific type of weather or road condition, determines a personalization score for one or more other driving paths, and identifies a driving path with the highest personalization score.

9. A method for operating a vehicle, comprising:

collecting, by a plurality of sensors, sensor information when a driver is driving the vehicle manually, the collected sensor information including information associated with one or more manual driving actions provided by the driver during a drive;

determining, by a microprocessor, first context information from the sensor information and conditions associated with the vehicle at a time when the one or more manual driving actions are provided by the driver during the drive, wherein the conditions include information for vehicle settings other than settings associated with the one or more manual driving actions provided by the driver during the drive; and

based on the first context information, generating, by the microprocessor, first personalization information, the first personalization information mapping a first observed driver behavior to one or more manual driving actions and the conditions, the first observed driver behavior corresponding to the one or more automated driving actions that produce a vehicle behavior similar to the one or more manual driving actions when at least one condition of the conditions is detected by the plurality of sensors; and

when the vehicle is in a high or full automation level, detecting, by the microprocessor, an event associated with the first personalization information, wherein the event includes the at least one condition of the conditions and, in response, causes the vehicle to perform autonomously the one or more automated driving actions.

10. The method of claim 9, wherein the microprocessor determines, based on the first context information, a first personalization score for the first driver behavior, wherein the microprocessor receives further collected sensor information for a series of manual driving actions by the driver, the collected sensor information describing a second observed driver behavior, and determines second context information describing a second context that is related to the series of manual driving actions, wherein the autonomous driving planning unit converts the series of manual driving actions into corresponding second personalization information, the second personalization information mapping a second observed driver behavior to the series of manual driving actions, the second observed driver behavior corresponding to a series of automated driving actions that produce a vehicle behavior similar to the series of manual driving actions, wherein the microprocessor determines, based on the second context information, a second personalization score for the second driver behavior, wherein the second personalization score is less than the first personalization score, wherein the event is associated with both the first and second personalization information, and wherein the microprocessor, in response to the event, selects the first personalization information over the second personalization information.

11. The method of claim 9, wherein the context comprises at least one:

of a specific acceleration pattern for merging into traffic, a specific acceleration pattern for passing another vehicle, a specific acceleration pattern from a traffic signal, a specific acceleration pattern for a portion of a route, a specific acceleration pattern for a hill, a specific breaking pattern for stopping at a stop sign, a specific breaking pattern for stopping at a traffic light, a specific breaking pattern for taking curves, a specific turning pattern for a right turn, a specific turning pattern for a left turn, a cruise control setting for a traffic pattern, a cruise control setting for a specific road, a time when lights are turned on or off, a time when a specific radio station is listened to, a time period for turning on turn signals, a specific seat adjustment, a specific person in the automated vehicle, a mirror adjustment time, a mirror/seat setting, or a time period for turning on window wipers and wherein the first observed driver behavior comprises at least one of the specific turning pattern for a right turn or the specific turning pattern for a left turn.

12. The method of claim 9, wherein the first personalization information is stored in connection with an identity of the driver and wherein the first personalization information is disregarded by the microprocessor when the driver is absent from the vehicle when the vehicle is in the high or full automation level.

13. The method of claim 9, wherein the control system comprises different personalization information for different drivers, the different personalization information being associated with a common event.

14. The method of claim 13, wherein the microprocessor, in response to both the different drivers being present simultaneously in the vehicle when the vehicle is in the high or full automation level, selects, in response to the event, the one or more automated driving actions based on the different personalization information.

15. The method of claim 14, wherein microprocessor selects the personalization information having a highest personalization score.

16. The method of claim 9, wherein the microprocessor disregards the first or second personalization information when the first or second personalization information would violate a driving restriction if executed.

17. An automated vehicle control system, comprising:

a plurality of sensors that collect sensor information when a driver is driving a vehicle manually, the collected sensor information including information associated with one or more manual driving actions provided by the driver during a drive; and

a microprocessor programmed to;

determine first context information from the sensor information and conditions associated with the vehicle at a time when the one or more manual driving actions are provided by the driver during the drive, wherein the conditions include information for vehicle settings other than settings associated with the one or more manual driving actions provided by the driver during the drive;

based on the first context information, generate first personalization information, the first personalization information mapping a first driving path, corresponding to the one or more manual driving actions and the conditions, to one or more automated driving actions that produce a vehicle behavior similar to the one or more manual driving actions when at least one condition of the conditions is detected by the plurality of sensors; and

when the vehicle is in a high or full automation level, detects an event associated with the first personalization information, wherein the event includes the at least one condition of the conditions and, in response, causes the vehicle to perform autonomously the first driving path.

19. The control system of claim 17, wherein the context comprises at least one:

of a specific acceleration pattern for merging into traffic, a specific acceleration pattern for passing another vehicle, a specific acceleration pattern from a traffic signal, a specific acceleration pattern for a portion of a route, a specific acceleration pattern for a hill, a specific breaking pattern for stopping at a stop sign, a specific breaking pattern for stopping at a traffic light, a specific breaking pattern for taking curves, a specific turning pattern for a right turn, a specific turning pattern for a left turn, a cruise control setting for a traffic pattern, a cruise control setting for a specific road, a time when lights are turned on or off, a time when a specific radio station is listened to, a time period for turning on turn signals, a specific seat adjustment, a specific person in the automated vehicle, a mirror adjustment time, a mirror/seat setting, or a time period for turning on window wipers and wherein the first driving path comprises at least one of the specific turning pattern for a right turn or the specific turning pattern for a left turn.