Vehicle autonomy level selection based on user context

• US 10,410,250 B2
• Filed: 12/30/2016
• Issued: 09/10/2019
• Est. Priority Date: 11/21/2016
• Status: Active Grant

• Current Assignee: NIO Technology Company Limited (Shanghai Anbin Technology Co., Ltd.)
• Original Assignee: Nio Usa Inc.
• Inventors: Singhal, Abhishek, Muralidhar, Gautam, Pouliot, Christopher F., Baik, Edward H., Cox, Jonathan A.
• Primary Examiner(s): To, Tuan C
• Application Number:
  US15/395,924
• Publication Number:
  US 20180141568A1
• Time in Patent Office: 984 Days
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60S 1/56 specially adapted for cleaning other parts or devices than front windows or windscreens rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602
• CPC Class Codes: B60S 1/62 Other vehicle fittings for cleaning
• CPC Class Codes: B60W 10/00 Conjoint control of vehicle sub-units of different type or different function for propulsion of purely electrically-propelled vehicles with power supplied within the vehicle B60L50/00 - B60L58/00
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/08 including control of electric propulsion units, e.g. motors or generators
• CPC Class Codes: B60W 10/10 including control of change-speed gearings
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2040/0809 Driver authorisation; Driver identity check
• CPC Class Codes: B60W 2050/0004 In digital systems, e.g. discrete-time systems involving sampling
• CPC Class Codes: B60W 2050/0014 Adaptive controllers
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2300/34 Compact city vehicles, e.g., microcars or kei cars
• CPC Class Codes: B60W 2420/403 : Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 : Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 : Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2510/08 : Electric propulsion units
• CPC Class Codes: B60W 2510/18 : Braking system
• CPC Class Codes: B60W 2520/04 : Vehicle stop
• CPC Class Codes: B60W 2520/105 : Longitudinal acceleration
• CPC Class Codes: B60W 2540/043 : Identity of occupants
• CPC Class Codes: B60W 2540/18 : Steering angle
• CPC Class Codes: B60W 2540/22 : Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/30 : Driving style
• CPC Class Codes: B60W 2552/40 : Coefficient of friction
• CPC Class Codes: B60W 2552/50 : Barriers
• CPC Class Codes: B60W 2552/53 : Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 : Input parameters relating to objects
• CPC Class Codes: B60W 2554/4026 : Cycles
• CPC Class Codes: B60W 2554/4029 : Pedestrians
• CPC Class Codes: B60W 2554/4041 : Position
• CPC Class Codes: B60W 2554/4046 : Behavior, e.g. aggressive or erratic
• CPC Class Codes: B60W 2554/80 : Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/802 : Longitudinal distance
• CPC Class Codes: B60W 2556/10 : Historical data
• CPC Class Codes: B60W 2556/45 : External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/55 : using telemetry
• CPC Class Codes: B60W 2556/65 : Data transmitted between vehicles
• CPC Class Codes: B60W 2710/18 : Braking system
• CPC Class Codes: B60W 2710/20 : Steering systems
• CPC Class Codes: B60W 2756/10 : Involving external transmission of data to or from the vehicle
• CPC Class Codes: B60W 2900/00 : Indexing codes relating to the purpose of, or problem solved of road vehicle drive control systems not otherwise provided for in groups B60W30/00
• CPC Class Codes: B60W 30/08 : Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/085 : Taking automatic action to adjust vehicle attitude in preparation for collision, e.g. braking for nose dropping
• CPC Class Codes: B60W 30/09 : Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/095 : Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 30/0953 : the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 : the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 : Traffic conditions
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: B60W 40/09 : Driving style or behaviour
• CPC Class Codes: B60W 40/105 : Speed
• CPC Class Codes: B60W 50/0097 : Predicting future conditions
• CPC Class Codes: B60W 50/0098 : Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/08 : Interaction between the driver and the control system
• CPC Class Codes: B60W 50/082 : Selecting or switching between different modes of propelling
• CPC Class Codes: B60W 60/0027 : using trajectory prediction for other traffic participants
• CPC Class Codes: B62D 15/00 : Steering not otherwise provided for
• CPC Class Codes: B62D 15/0265 : Automatic obstacle avoidance by steering
• CPC Class Codes: G01C 21/3407 : specially adapted for specific applications
• CPC Class Codes: G01C 21/3461 : Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types, segments such as motorways, toll roads, ferries
• CPC Class Codes: G01C 21/3469 : Fuel consumption; Energy use; Emission aspects
• CPC Class Codes: G01C 21/3484 : Personalized, e.g. from learned user behaviour or user-defined profiles
• CPC Class Codes: G01C 21/3492 : employing speed data or traffic data, e.g. real-time or historical traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01C 21/3682 : output of POI information on a road map G01C21/3614, G01C21/3685 take precedence
• CPC Class Codes: G01C 21/3691 : Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions arrangements for giving variable traffic instructions G08G1/09
• CPC Class Codes: G01C 21/3697 : Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G01S 13/862 : Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 : Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 : Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 : Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 15/02 : using reflection of acoustic waves G01S15/66 takes precedence
• CPC Class Codes: G01S 17/89 : for mapping or imaging
• CPC Class Codes: G01S 19/13 : Receivers
• CPC Class Codes: G01S 2007/4977 : including means to prevent or remove the obstruction
• CPC Class Codes: G01S 2013/9316 : combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9318 : Controlling the steering
• CPC Class Codes: G01S 2013/93185 : Controlling the brakes
• CPC Class Codes: G01S 2013/9319 : Controlling the accelerator
• CPC Class Codes: G01S 2013/932 : using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9322 : using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/9325 : for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G01S 2013/93271 : in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 : in the back of the vehicles
• CPC Class Codes: G01S 2013/93273 : on the top of the vehicles
• CPC Class Codes: G01S 2013/93274 : on the side of the vehicles
• CPC Class Codes: G01S 2013/93275 : in the bumper area
• CPC Class Codes: G01S 7/4021 : of receivers
• CPC Class Codes: G01S 7/4043 : including means to prevent or remove the obstruction
• CPC Class Codes: G01S 7/497 : Means for monitoring or calibrating
• CPC Class Codes: G02B 27/0006 : with means to keep optical surfaces clean, e.g. by preventing or removing dirt, stains, contamination, condensation G02B1/18 takes precedence; cleaning in general B08B
• CPC Class Codes: G05D 1/0212 : with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0276 : using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G05D 1/228 : Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G05D 1/247 : using signals provided by artificial sources external to the vehicle, e.g. navigation beacons
• CPC Class Codes: G05D 1/617 : Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80
• CPC Class Codes: G05D 1/646 : Following a predefined trajectory, e.g. a line marked on the floor or a flight path
• CPC Class Codes: G05D 1/81 : Handing over between on-board automatic and on-board manual control
• CPC Class Codes: G06F 16/29 : Geographical information databases
• CPC Class Codes: G06F 16/95 : Retrieval from the web
• CPC Class Codes: G06Q 30/0266 : based on the position of the vehicle
• CPC Class Codes: G06Q 30/0269 : based on user profile or attribute
• CPC Class Codes: G07C 5/0808 : Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G08G 1/161 : Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 : involving continuous checking
• CPC Class Codes: G08G 1/164 : Centralised systems, e.g. external to vehicles
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Embodiments herein relate to an autonomous vehicle or self-driving vehicle. The level of comfort with autonomous driving offered to the user and the parameters of operation of the autonomous vehicle might increase or decrease a driver'"'"'s comfort (anxiety) with the vehicle. Sensors in the vehicle can detect a user'"'"'s driving style, comfort level and may propose an autonomy level to the user and/or parameters of operation.

Claims

1. A vehicle, comprising:

a sensor configured to sense driving functions associated with an operation of the vehicle;

a vehicle control system configured to autonomously control driving functions of the vehicle; and

a processor in communication with the sensor and the vehicle control system, the processor configured to;

receive information from the sensor when a driver of the vehicle is operating the vehicle in a manual operational mode, wherein the information corresponds to driving functions performed by the driver in the manual operational mode;

determine, based on the information received from the sensor, a manual control behavior for the driving functions performed by the driver in the manual operational mode, the manual control behavior defining a manual driving style of the driver;

determine, based on the determined manual control behavior, an autonomous operational mode for controlling driving functions of the vehicle during an autonomous operation of the vehicle, wherein the autonomous operational mode includes an autonomous control behavior that mimics the manual control behavior and matches the manual driving style of the driver;

send the autonomous operational mode to the vehicle control system to control driving functions of the vehicle during the autonomous operation of the vehicle using the determined autonomous control behavior that mimics the determined manual control behavior and matches the manual driving style of the driver;

monitor, while the vehicle control system is autonomously controlling a driving function of the vehicle using the determined autonomous control behavior, a biometric characteristic of the driver to determine a level of anxiety of the driver; and

present, in response to determining an increased level of anxiety of the driver above a baseline level of anxiety for the driver, an option for the driver to take control over the vehicle.

2. The vehicle of claim 1, wherein the determined manual control behavior is stored as a driving characteristic in a driver behavior data structure.

3. The vehicle of claim 2, wherein the driving characteristic is one or more of a speed characteristic, an acceleration or deceleration characteristic, a distance characteristic, a steering angle characteristic, and/or a lane change characteristic performed by the driver in the manual operational mode.

4. The vehicle of claim 3, wherein the autonomous operational mode establishes a level of autonomy for the vehicle control system to control the vehicle, and wherein the operational mode is one of a Level 4 or a High Automation Level and a Level 5 or a Full Automation Level.

5. The vehicle of claim 4, wherein the autonomous operational mode is represented by an autonomous operational mode data structure and wherein the autonomous operational mode data structure comprises an autonomy level indicator indicative of one of Level 0, Level 1, Level 2, Level 3, Level 4, and Level 5.

6. The vehicle of claim 5, wherein the autonomous operational mode data structure further comprises one or more of an inter-vehicle distance setting, an acceleration or deceleration setting, a speed setting, and/or a lane change setting.

7. The vehicle of claim 6, wherein, while the vehicle control system is autonomously controlling the driving function of the vehicle, sensors measure the biometric characteristic of the driver and/or a biometric characteristic of a passenger to determine the level of anxiety of the driver and/or a level of anxiety of the passenger.

8. The vehicle of claim 7, wherein the biometric characteristic of the driver and/or the passenger is stored in an anxiety level data structure and compared against a baseline level for the driver and/or the passenger to determine whether the driver and/or the passenger has an increased level of anxiety in response to a sensed driving event.

9. The vehicle of claim 8, wherein the biometric characteristic of the driver and/or the passenger is one or more of a heart rate characteristic, a breathing rate characteristic, a body temperature characteristic, and/or an eye characteristic.

10. The vehicle of claim 9, wherein the comparison of the biometric characteristic of the driver and/or the passenger to the baseline level for the biometric characteristic indicates that the driver and/or the passenger has an increased level of anxiety in response to the sensed driving event.

11. The vehicle of claim 10, wherein based on the increased level of anxiety of the passenger, the processor changes the level of autonomy used to control the vehicle.

12. A method, comprising:

receiving, from a sensor configured to sense driving functions associated with an operation of a vehicle, information corresponding to driving functions performed by a driver while operating the vehicle in a manual operational mode;

determining, via a processor and based on the information received from the sensor, a manual control behavior for the driving functions performed by the driver in the manual operational mode, the manual control behavior defining a manual driving style of the driver;

determining, via the processor and based on the determined manual control behavior, an autonomous operational mode for controlling driving functions of the vehicle during an autonomous operation of the vehicle, wherein the autonomous operational mode includes an autonomous control behavior that mimics the manual control behavior and matches the manual driving style of the driver;

sending, via the processor, the autonomous operational mode to a vehicle control system to control driving functions of the vehicle during the autonomous operation of the vehicle using the determined autonomous control behavior that mimics the determined manual control behavior and matches the manual driving style of the driver;

controlling, autonomously via a vehicle control system, driving functions of the vehicle during the autonomous operation of the vehicle based on the autonomous operational mode sent by the processor to mimic the determined manual control behavior and match the manual driving style of the driver;

monitoring, while the vehicle control system is autonomously controlling the driving functions of the vehicle using the determined autonomous control behavior, a biometric characteristic of the driver to determine a level of anxiety of the driver; and

presenting, in response to determining an increased level of anxiety of the driver above a baseline level of anxiety for the driver, an option for the driver to take control over the vehicle.

13. The method of claim 12, wherein the determined manual control behavior is stored as a driving characteristic in a driver behavior data structure, wherein the driving characteristic is one or more of a speed characteristic, an acceleration or deceleration characteristic, an inter-vehicle distance characteristic, and/or a lane change characteristic performed by the driver in the manual operational mode.

14. The method of claim 13, wherein the autonomous operational mode establishes a level of autonomy for the vehicle control system to control the vehicle, wherein the autonomous operational mode is represented by an autonomous operational mode data structure, wherein the autonomous operational mode data structure comprises one or more of an autonomy level, a distance setting, an acceleration setting, a speed setting, and/or a lane change setting, and wherein the operational mode data structure comprises an autonomy level indicator indicative of one of Level 0, Level 1, Level 2, Level 3, Level 4, and Level 5.

15. The method of claim 14, wherein, while the vehicle control system is autonomously controlling a driving function of the vehicle, sensors measure the biometric characteristic of the driver and/or a biometric characteristic of a passenger to determine a level of anxiety of the driver and/or a level of anxiety of the passenger, wherein the biometric characteristic of the driver and/or the passenger is stored in an anxiety level data structure, wherein the autonomy level is one of a Level 4 or a High Automation Level and a Level 5 or a Full Automation Level, and wherein the biometric characteristic of the driver and/or the passenger is one of a driver identifier, a passenger identifier, a heart characteristic, a breath characteristic, a temperature characteristic, and/or an eye characteristic.

16. The method of claim 15, wherein the biometric characteristic of the driver and/or the passenger indicates that the driver or the passenger has an increased level of anxiety, wherein the biometric characteristic of the driver and/or the passenger is compared against a baseline level for the driver and/or the passenger to determine whether the driver and/or the passenger has an increased level of anxiety in response to a sensed driving event, and wherein based on the increased level of anxiety of the passenger, the processor changes the level of autonomy used to control the vehicle.

17. A method of controlling a vehicle via a vehicle control system, comprising:

while the vehicle control system is autonomously controlling driving functions of the vehicle, measuring, by on board vehicle sensors, a biometric characteristic of a driver and/or a passenger to determine a level of anxiety of the driver and/or passenger;

comparing the biometric characteristic to a baseline level for the biometric characteristic to determine that the driver or passenger has an increased level of anxiety in response to the autonomously controlled driving functions; and

based on the increased level of anxiety, the vehicle control system proposes to the driver to take control over the vehicle and control the driving functions of the vehicle in a fully-manual level of operation.

18. The method of claim 17, wherein the biometric characteristic is one or more of a heart rate characteristic, a breathing rate characteristic, a body temperature characteristic, and/or an eye characteristic.

19. The method of claim 17, wherein the vehicle control system is autonomously controlling the driving function of the vehicle mimicking a learned manual driving style associated with the driver.

20. The method of claim 19, wherein the learned manual driving style is stored as driving characteristics in a driver behavior data structure, and wherein the learned manual driving style is determined by a processor of the vehicle receiving, from a sensor, information corresponding to driving functions performed by the driver while operating the vehicle in a manual operational mode, determining based on the information received from the sensor, a manual control behavior for the driving functions performed by the driver in the manual operational mode, and wherein the manual control behavior defines a manual driving style for the driver.