Apparatus and method to train autonomous driving model, and autonomous driving apparatus

• US 10,791,979 B2
• Filed: 07/13/2018
• Issued: 10/06/2020
• Est. Priority Date: 11/16/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Dong Hwa, Kim, Chang Hyun, Shim, Eun Soo, Choi, Ki Hwan, Choi, Hyun Jin
• Primary Examiner(s): Kan, Yuri
• Application Number:
  US16/034,436
• Publication Number:
  US 20180325442A1
• Time in Patent Office: 816 Days
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/10 Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/225 of the fingers, e.g. by monitoring hand-grip force
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

Disclosed is an apparatus and method to train an autonomous driving model. The apparatus includes a driver information collection processor configured to collect driver information while a vehicle is being driven. The apparatus also includes a sensor information collection processor configured to collect sensor information from a sensor installed in the vehicle while the vehicle is being driven, and a model training processor configured to train the autonomous driving model based on the driver information and the sensor information.

Claims

1. An autonomous driving apparatus comprising:

a driving information collection processor configured to collect driving information of a vehicle; and

a model training processor configured to train an autonomous driving model based on the driving information.

2. The autonomous driving apparatus of claim 1, wherein the driving information comprises at least one of a driver information, a sensor information from a sensor installed in the vehicle, and a navigation setting information.

3. The autonomous driving apparatus of claim 2, wherein the driver information comprises at least one of a driver state information, a driver behavior information, and a driving pattern information.

4. The autonomous driving apparatus of claim 3, wherein:

the driver state information comprises at least one piece of information about a gaze direction, a heart rate, an electrocardiogram, brain waves, and grip strength of a hand holding a steering wheel of a driver;

the driver behavior information comprises at least one piece of manipulation information about an audio device, an air conditioner, a mirror device, and a navigation device installed in the vehicle; and

the driving pattern information comprises at least one of steering wheel manipulation information, accelerator pedal manipulation information, horn manipulation information, turn-signal manipulation information, emergency button manipulation information, and brake pedal manipulation information.

5. The autonomous driving apparatus of claim 4, wherein the model training processor is configured to:

train the autonomous driving model to learn about an area in which the driver has to be careful while driving based on the gaze direction of the driver; and

train the autonomous driving model to learn about at least one of dangerous situations and levels of danger in driving based on the driving pattern information.

6. The autonomous driving apparatus of claim 1, wherein the model training processor trains the autonomous driving model based on the driving information after collection of the driving information starts and a predefined period of time has elapsed.

7. The autonomous driving apparatus of claim 2, wherein the model training processor trains the autonomous driving model to learn about an integrated driving situation from a starting location to an ending location based on the navigation setting information.

8. The autonomous driving apparatus of claim 2, wherein the sensor information comprises at least one of surrounding image information, surrounding voice information, current location information, light detection and ranging (LiDAR) sensor information, radar sensor information, vehicle speed information, and acceleration information.

9. The autonomous driving apparatus of claim 1, further comprising a model determination processor configured to determine whether the autonomous driving model is appropriate for a driver and a current driving situation based on at least one of a driver information and a navigation setting information.

10. The autonomous driving apparatus of claim 9, further comprising a driving controller configured to automatically control an operation of the vehicle when a result of the model determination processor is the autonomous driving model is appropriate for the driver and the current driving situation.

11. The autonomous driving apparatus of claim 9, wherein the model determination processor is configured to:

check whether the autonomous driving model is trained for the driver based on the driver information; and

when a result of the check is that the autonomous driving model is not trained for the driver, determine that the autonomous driving model is not appropriate.

12. The autonomous driving apparatus of claim 9, wherein the model determination processor is configured to:

check whether the autonomous driving model is trained for a driving route from a starting location to an ending location based on the navigation setting information; and

when a result of the check is that the autonomous driving model is not trained for the driving route, determine that the autonomous driving model is not appropriate.

13. An autonomous driving method comprising:

collecting driving information of a vehicle; and

training, an autonomous driving model based on the driving information.

14. The autonomous driving method of claim 13, wherein the driving information comprises at least one of a driver information, a sensor information from a sensor installed in the vehicle, and a navigation setting information.

15. The autonomous driving method of claim 14, wherein:

the driver information comprises at least one of a driver state information, a driver behavior information, and a driving pattern information;

the driver state information comprises at least one piece of information about a gaze direction, a heart rate, an electrocardiogram, brain waves, and grip strength of a hand holding a steering wheel of a driver;

the driver behavior information comprises at least one piece of manipulation information about an audio device, an air conditioner, a mirror device, and a navigation device installed in the vehicle; and

the driving pattern information comprises at least one of steering wheel manipulation information, accelerator pedal manipulation information, horn manipulation information, turn-signal manipulation information, emergency button manipulation information, and brake pedal manipulation information.

16. The autonomous driving method of claim 15, wherein the training of the

autonomous driving model comprises:

training the autonomous driving model to learn about an area in which the driver has to be careful while driving based on the gaze direction of the driver; and

training the autonomous driving model to learn about at least one of dangerous situations and levels of danger in driving based on the driving pattern information.

17. The autonomous driving method of claim 13, wherein the training of the autonomous driving model comprises training, after the collecting of the driving information starts and a predefined period of time has elapsed, the autonomous driving model based on the driving information.

18. The autonomous driving method of claim 14, wherein the training of the autonomous driving model comprises training the autonomous driving model to learn about an integrated driving situation from a starting location to an ending location based on the navigation setting information.

19. The autonomous driving method of claim 13, further comprising determining whether the autonomous driving model is appropriate for a driver and a current driving situation based on at least one of driver information and navigation setting information.

20. The autonomous driving method of claim 19, further comprising automatically controlling, when a result of determining the autonomous driving model is appropriate for the driver and the current driving situation, an operation of the vehicle.