Electronic apparatus for determining a dangerous situation of a vehicle and method of operating the same

• US 10,745,009 B2
• Filed: 12/20/2017
• Issued: 08/18/2020
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jang, Jun-Ik, Kim, Ye-Hoon, Yun, So-jung, Lee, Jin-ho, Yang, Do-jun, Lee, Han-sung, Hong, Soon-hyuk
• Primary Examiner(s): Goldman, Richard A
• Application Number:
  US15/848,901
• Publication Number:
  US 20180170375A1
• Time in Patent Office: 972 Days
• Field of Search: 701301, 701117, 701 23, 701 25, 701 293
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2050/0071 Controller overrides driver automatically
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2520/105 Longitudinal acceleration
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

An electronic apparatus is provided. The electronic apparatus includes at least one processor configured to detect, by a sensor, a driving state of a vehicle or a driving state of another vehicle in a vicinity of the vehicle while the vehicle is being driven, obtain state data including the driving state of the vehicle and the driving state of the other vehicle, and use at least one data recognition model to determine a dangerous situation of the vehicle after a certain period of time while the vehicle is being driven, based on the obtained state data. The electronic apparatus may predict a dangerous situation of the vehicle by using a rule-based algorithm or an artificial intelligence (AI) algorithm. The electronic apparatus predicts the dangerous situation of the vehicle by using at least one of machine learning, a neural network, and a deep learning algorithm.

Claims

1. An electronic apparatus comprising:

a sensor configured to detect a first driving state of a first vehicle and a second driving state of a second vehicle in a vicinity of the first vehicle while the first vehicle is being driven;

a memory configured to store at least one instruction; and

at least one processor configured to execute the at least one instruction stored in the memory,

wherein, by executing the at least one instruction, the at least one processor is further configured to;

obtain state data based on the detected first driving state of the first vehicle and the detected second driving state of the second vehicle, and

predict, using a data recognition model, a dangerous situation of the first vehicle while the first vehicle is being driven, based on the obtained state data,

wherein the at least one processor is further configured to train the data recognition model by using, as a training data, the obtained state data and the predicted dangerous situation of the first vehicle based on the obtained state data.

2. The electronic apparatus of claim 1,

wherein the sensor is further configured to detect a state of a driver who drives the first vehicle,

wherein the state of the driver comprises at least one of a facial expression, an eye direction, or an action of the driver, and

wherein, by executing the at least one instruction, the at least one processor is further configured to obtain the state data further comprising the detected state of the driver.

3. The electronic apparatus of claim 1, wherein the first driving state of the first vehicle comprises at least one of a driving speed, driving acceleration, or a driving direction of the first vehicle.

4. The electronic apparatus of claim 1,

wherein the sensor is further configured to obtain a captured image of the second vehicle driving in the vicinity of the first vehicle, and

wherein, by executing the at least one instruction, the at least one processor is further configured to:

obtain second vehicle information of the second vehicle from the captured image, and

obtain the state data further comprising the obtained second vehicle information of the second vehicle.

5. The electronic apparatus of claim 1,

wherein the sensor is further configured to detect a surrounding environment state,

wherein the surrounding environment state comprises:

a state of a road on which the first vehicle is being driven, or

a weather condition, and

wherein, by executing the at least one instruction, the at least one processor is further configured to obtain the state data further comprising the detected surrounding environment state.

6. The electronic apparatus of claim 1, wherein, by executing the at least one instruction, the at least one processor is further configured to predict, by using the data recognition model, driving guidance information for preventing the predicted dangerous situation of the first vehicle.

7. The electronic apparatus of claim 1, further comprising a display,

wherein, by executing the at least one instruction, the at least one processor is further configured to display at least one of the predicted dangerous situation or driving guidance information for preventing the predicted dangerous situation.

8. The electronic apparatus of claim 1,

wherein, by executing the at least one instruction, the at least one processor is further configured to determine, by using the data recognition model, a level of risk of the second vehicle which is driving in the vicinity of the first vehicle and affecting to the first vehicle, based on the obtained state data.

9. The electronic apparatus of claim 1, wherein, by executing the at least one instruction, the at least one processor is further configured to control a driving operation comprising at least one of a driving speed or a driving direction of the first vehicle based on driving guidance information for preventing the predicted dangerous situation of the first vehicle.

10. The electronic apparatus of claim 9, wherein, by executing the at least one instruction, when a level of risk of the predicted dangerous situation of the first vehicle corresponds to a preset threshold value, the at least one processor is further configured to take over control of the driving operation from a driver.

11. The electronic apparatus of claim 1,

wherein, by executing the at least one instruction,

the at least one processor is further configured to update the data recognition model by using, as a training data, the dangerous situation predicted based on the state data and a driving guidance information for preventing the predicted dangerous situation.

12. The electronic apparatus of claim 1,

wherein, by executing the at least one instruction,

the at least one processor is further configured to update the data recognition model by using, as a training data, the dangerous situation predicted based on state data obtained by combining a plurality of detected states.

13. The electronic apparatus of claim 1, wherein the at least one processor is further configured to:

predict the dangerous situation of the first vehicle by using at least one of a rule-based algorithm or an artificial intelligence (AI) algorithm.

14. A method of operating an electronic apparatus, the method comprising:

detecting a first driving state of a first vehicle and a second driving state of a second vehicle in a vicinity of the first vehicle while the first vehicle is being driven;

obtaining state data based on the detected first driving state of the first vehicle and the detected second driving state of the second vehicle; and

predicting, using a data recognition model, a dangerous situation of the first vehicle while the first vehicle is being driven, based on the obtained state data,

wherein the method further comprises training the data recognition model by using, as a training data, the obtained state data and the predicted dangerous situation of the first vehicle based on the obtained state data.

15. The method of claim 14, further comprising:

detecting a state of a driver who drives the first vehicle,

wherein the state of the driver comprises at least one of a facial expression, an eye direction, or an action of the driver, and

wherein the obtaining of the state data comprises obtaining the state data further comprising the detected state of the driver.

16. The method of claim 14, further comprising:

obtaining an image of the second vehicle driving in the vicinity of the first vehicle; and

obtaining second vehicle information of the second vehicle from the image of the second vehicle,

wherein the obtaining of the state data comprises obtaining the state data further comprising the obtained second vehicle information of the second vehicle.

17. The method of claim 14, further comprising:

predicting, by using the data recognition model, driving guidance information for preventing the predicted dangerous situation of the first vehicle.

18. The method of claim 14, further comprising:

displaying at least one of the predicted dangerous situation of the first vehicle or driving guidance information for preventing the predicted dangerous situation of the first vehicle.

19. The method of claim 14, further comprising:

controlling a driving operation using at least one of a driving speed or a driving direction of the first vehicle, based on driving guidance information for preventing the predicted dangerous situation of the first vehicle.

20. The method of claim 14, further comprising:

updating the data recognition model by using, as a training data, the dangerous situation predicted based on the state data obtained by combining a plurality of detected states.