Apparatus for switching driving mode of vehicle and method thereof

• US 10,493,997 B2
• Filed: 01/29/2018
• Issued: 12/03/2019
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Hyun
• Primary Examiner(s): Ramesh, Krishnan
• Application Number:
  US15/882,206
• Publication Number:
  US 20180148071A1
• Time in Patent Office: 673 Days
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60W 2040/089 Driver voice
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/10 Accelerator pedal position
• CPC Class Codes: B60W 2540/12 Brake pedal position
• CPC Class Codes: B60W 2540/21 Voice
• CPC Class Codes: B60W 2540/215 Selection or confirmation of options
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/223 Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 50/082 Selecting or switching between different modes of propelling
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0053 : from vehicle to occupant
• CPC Class Codes: B60W 60/0054 : Selection of occupant to assume driving tasks
• CPC Class Codes: B60W 60/0059 : Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: Y02T 10/84 : Data processing systems or methods, management, administration

Abstract

Systems and techniques are disclosed for switching a vehicle driving mode. A sensing unit senses a state of a driver of a vehicle configured to be driven automatically or manually. An intention detecting unit detects whether the driver intends to switch from an automatic driving mode to a manual driving mode based on the state of the driver. An operation detecting unit detects whether the driver is able to operate the vehicle in the manual driving mode based on the state of the driver. A driving state predicting unit predicts a driving state of the vehicle in the manual driving mode based on detecting that the driver is able to operate the vehicle in the manual driving mode. A control unit determines that the predicted driving state of the vehicle meets a preset condition and switches from the automatic to the manual driving mode.

Claims

1. An apparatus configured to switch a driving mode of a vehicle, the apparatus comprising:

at least one sensing unit; and

at least one processor configured to;

detect, through the at least one sensing unit, at least one of information related to the vehicle or information related to a driver of the vehicle;

detect, using the information related to the driver of the vehicle, whether the driver intends to switch the driving mode of the vehicle, the driving mode being one of an autonomous driving mode or a manual driving mode;

determine whether the driving mode of the vehicle is to be switched, based on at least one of the information related to the vehicle, the information related to the driver, or detecting that the driver intends to switch the driving mode;

predict a driving state in which the vehicle will operate in the switched driving mode;

determine whether the predicted driving state of the vehicle meets a preset condition;

based on a determination that the predicted driving state of the vehicle does not meet the preset condition, maintain the driving mode of the vehicle until a determination that the predicted driving state of the vehicle meets the preset condition; and

based on a determination that the predicted driving state of the vehicle meets the preset condition, switch the driving mode of the vehicle and generate an output to the driver regarding switching the driving mode of the vehicle.

2. The apparatus of claim 1, wherein:

the information related to the vehicle comprises at least one of;

information related to a characteristic of the vehicle, an operational state of the vehicle, an environment around the vehicle, an interaction between a user and the vehicle, or a mobile terminal in the vehicle, and

the information related to the driver of the vehicle comprises at least one of;

a position of a hand or a foot of the driver, a posture of the driver, or a state of at least one eye of the driver.

3. The apparatus of claim 1, wherein the at least one processor is configured to detect whether the driver intends to switch the driving mode of the vehicle by:

detecting a selection, by the driver, of one of the autonomous driving mode or the manual driving mode, or

determining whether an actuation of a control unit of the vehicle by the driver meets a threshold actuation condition.

4. The apparatus of claim 3, wherein the at least one processor is configured to:

detect the selection, by the driver, of one of the autonomous driving mode or the manual driving mode by detecting the selection as an input on a mobile terminal connected to the vehicle or mounted in the vehicle, or

detect the actuation, by the driver, of the control unit of the vehicle by detecting, via the at least one sensing unit, at least one of an angle of steering by the driver on a steering control unit of the vehicle, or an amount of pressing by the driver on a speed control unit of the vehicle.

5. The apparatus of claim 3, wherein the at least one processor is configured to:

based on a determination that the actuation of the control unit by the driver meets the threshold actuation condition, implement the driving mode of the vehicle as the manual driving mode; and

based on a determination that the actuation of the control unit by the driver does not meet the threshold actuation condition, implement the driving mode of the vehicle as the autonomous driving mode.

6. The apparatus of claim 3, wherein determining that the actuation of the control unit of the vehicle by the driver meets the threshold actuation condition comprises at least one of:

determining that a hand of the driver is holding a steering wheel of the vehicle,

determining that a foot of the driver is placed on an accelerator pedal or a brake pedal of the vehicle, or

determining that an angle of steering on a steering wheel of the vehicle or an amount of pressing on a pedal of the vehicle exceeds a threshold amount.

7. The apparatus of claim 3, wherein the at least one processor is further configured to:

while in a state in which the vehicle operates in the autonomous driving mode, perform one or more operations for controlling the vehicle according to the actuation of the control unit of the vehicle by the driver.

8. The apparatus of claim 3, wherein the at least one processor is further configured to:

while in a state in which the vehicle operates in the autonomous driving mode, modify the autonomous driving mode by changing a steering or a speed of the vehicle according to a manual actuation of the control unit of the vehicle by the driver.

9. The apparatus of claim 1, wherein the at least one processor is further configured to:

determine whether the driving mode of the vehicle is to be switched by determining whether an interaction by the driver with a display in the vehicle meets a threshold condition; and

based on determining that the interaction by the driver with the display meets the threshold condition;

in a state in which the vehicle operates in the manual driving mode, determine that the driving mode of the vehicle is to be switched to the autonomous driving mode; and

in a state in which the vehicle operates in the autonomous driving mode, determine that the driving mode of the vehicle is not to be switched to the manual driving mode.

10. The apparatus of claim 9, wherein the at least one processor is further configured to, based on determining that the interaction by the driver with the display does not meet the threshold condition:

in a state in which the vehicle operates in the autonomous driving mode, and based on detecting that the driver intends to switch the driving mode to the manual driving mode, determine that the driving mode of the vehicle is to be switched to the manual driving mode.

11. The apparatus of claim 9, wherein the at least one processor is configured to determine that the interaction by the driver with the display in the vehicle meets the threshold condition by:

detecting that the driver controls a program displayed on the display to navigate to a menu navigation depth that exceeds a threshold depth, or

detecting that a line of sight in which the driver gazes at the display remains fixed for more than a threshold amount of time.

12. The apparatus of claim 9, wherein the display is part of a mobile terminal connected to the vehicle, and

wherein the at least one processor is further configured to determine whether the interaction by the driver with the display meets the threshold condition by:

detecting whether the threshold condition is met by at least one of a gesture by the driver detected by the mobile terminal or by a physical manipulation of the mobile terminal by the driver.

13. The apparatus of claim 12, wherein detecting that the threshold condition is met by at least one of the gesture by the driver detected by the mobile terminal or by the physical manipulation of the mobile terminal by the driver comprises:

detecting that the mobile terminal is inclined by more than a threshold angle in a direction in which the display faces the driver, or

detecting at least one of the driver handling the mobile terminal, the driver disconnecting a wire from the mobile terminal, or the driver releasing a passcode on the mobile terminal.

14. The apparatus of claim 1, further comprising a display unit configured to output information corresponding to the driving mode of the vehicle,

wherein the at least one processor is configured to determine whether a condition of the driver meets a threshold condition based on the at least one sensing unit sensing that eyes of the driver are looking toward the information output on the display unit or looking toward a particular object recognized by an advanced driving assist system (S) of the vehicle.

15. The apparatus of claim 1, wherein the at least one processor is further configured to:

detect, through the at least one sensing unit, an eye gaze of the driver; and

determine whether the driving mode of the vehicle is to be switched based on a degree to which the sensed eye gaze of the driver aligns with a driving route of the vehicle, or based on a degree of fatigue of the driver that is calculated based on a degree of blinking by the driver and a degree to which eyelids of the driver cover the eyes of the driver.

16. The apparatus of claim 1, wherein the at least one processor is configured to determine whether the predicted driving state of the vehicle meets the preset condition by determining whether the vehicle operating in the switched state interferes with another vehicle, a pedestrian, an obstacle, or follows road indications.

17. The apparatus of claim 1, wherein the at least one processor is further configured to, based on a determination that the predicted driving state of the vehicle does not meet the preset condition:

output a recommendation to the driver regarding operating the vehicle to achieve a driving state that satisfies the preset condition, the recommendation being based on a condition of the driver and the driving mode of the vehicle.

18. The apparatus of claim 17, wherein the recommendation output to the driver comprises information related to holding a steering wheel of the vehicle, pressing pedal of the vehicle, an eye gaze of the driver, or an interaction by the driver with a display in the vehicle.

19. A method of controlling a vehicle, the method comprising:

detecting, through at least one sensing unit, at least one of information related to the vehicle or information related to a driver of the vehicle;

detecting, using the information related to the driver of the vehicle, whether the driver intends to switch a driving mode of the vehicle, the driving mode being one of an autonomous driving mode or a manual driving mode;

determining whether the driving mode of the vehicle is to be switched based on at least one of the information related to the vehicle, the information related to the driver, or detecting that the driver intends to switch the driving mode,

predicting a driving state in which the vehicle will operate in the switched driving mode;

determining whether the predicted driving state of the vehicle meets a preset condition;

based on a determination that the predicted driving state of the vehicle does not meet the preset condition, maintaining the driving mode of the vehicle until a determination that the predicted driving state of the vehicle meets the preset condition; and

based on a determination that the predicted driving state of the vehicle meets the preset condition, switching the driving mode of the vehicle and generating an output to the driver regarding switching the driving mode of the vehicle.

20. The method of claim 19, wherein:

the information related to the vehicle comprises at least one of;

information related to a characteristic of the vehicle, an operational state of the vehicle, an environment around the vehicle, an interaction between a user and the vehicle, or a mobile terminal in the vehicle, and

the information related to the driver of the vehicle comprises at least one of;

a position of a hand or a foot of the driver, a posture of the driver, or a state of at least one eye of the driver.

21. The method of claim 19, wherein detecting whether the driver intends to switch the driving mode of the vehicle comprises:

detecting a selection, by the driver, of one of the autonomous driving mode or the manual driving mode, or

detecting that an actuation of a control unit of the vehicle by the driver meets a threshold actuation condition, wherein the actuation of the control unit of the vehicle comprises the driver'"'"'s control of a steering or speed of the vehicle.

22. The method of claim 21, further comprising:

while in a state in which the vehicle operates in the autonomous driving mode, performing one or more operations for controlling the steering or the speed of the vehicle according to the actuation of the control unit of the vehicle by the driver.

23. The method of claim 19, further comprising:

determining that the driving mode of the vehicle is to be switched from the manual driving mode to the autonomous driving mode or that the driving mode of the vehicle is to be maintained in the autonomous driving mode without switching to the manual driving mode by;

detecting that the driver controls a program displayed on a display in the vehicle to navigate to a menu navigation depth that exceeds a threshold depth, or detecting that a line of sight in which the driver gazes at the display remains fixed for more than a threshold amount of time.

24. The method of claim 19, further comprising:

detecting, through the at least one sensing unit, an eye gaze of the driver; and

determining whether the driving mode of the vehicle is to be switched based on a degree to which the sensed eye gaze of the driver aligns with a driving route of the vehicle, or based on a degree of fatigue of the driver that is calculated based on a degree of blinking by the driver and a degree to which eyelids of the driver cover the eyes of the driver.