Method of effective driving behavior extraction using deep learning

• US 10,198,693 B2
• Filed: 10/24/2016
• Issued: 02/05/2019
• Est. Priority Date: 10/24/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dong, Wei Shan, Gao, Peng, Li, Jian, Li, Chang Sheng, Luo, Wen Han, Ma, Chun Yang, Yao, Renjie, Yuan, Ting, Zhu, Jun
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): Jha, Abdhesh K
• Application Number:
  US15/332,407
• Publication Number:
  US 20180113458A1
• Time in Patent Office: 834 Days
• Field of Search: None
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/06 Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Systems and methods for obtaining vehicle operational data and driving context data from one or more monitoring systems, including converting the obtained vehicle operational data and driving context data into sequential vehicle operational feature data and sequential driving context feature data, calibrating the sequential vehicle operational feature data and the sequential driving context feature data temporally to form calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data, constructing a sequence table of temporal sample points based on the calibrated sequential vehicle operational feature data and the calibrated sequential driving context feature data, feeding the sequence table into a deep neural network model for applying network learning to form a trained deep neural network model, extracting driving behavior features from the trained deep neural network model and analyzing the extracted driving behavior features to determine driving behavior characteristics of the driver.

Claims

1. A computer implemented method for determining driving behavior characteristics of a driver of a vehicle comprising:

obtaining vehicle operational data and driving context data from one or more monitoring systems over a plurality of temporal data points;

converting the obtained vehicle operational data and driving context data into sequential vehicle operational feature data and sequential driving context feature data;

associating the sequential vehicle operational feature data to the sequential driving context feature data of corresponding temporal data points of the plurality of temporal data points to form calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data;

constructing a sequence table of a sequence of the plurality of temporal data points based on the calibrated sequential vehicle operational feature data and the calibrated sequential driving context feature data associated with each of the plurality of temporal data points, the sequence table being constructed by combining segmented calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data from a plurality of windows, each window corresponding to a respective one temporal sample point;

feeding the sequence table into a deep neural network model for applying network learning to form a trained deep neural network model;

extracting driving behavior features from the trained deep neural network model; and

automatically determining driving behavior characteristics of the driver in the future based on the extracted driving behavior features.

2. The computer implemented method of claim 1, wherein the vehicle operational data comprises GPS vehicle location data and the sequential vehicle operational data comprises at least one member selected from the group consisting of sample points of instantaneous speed, acceleration, angular speed and a difference of each sample point from a normal instantaneous speed, acceleration, angular speed and wherein the vehicle operational data comprises OBD data obtained from the vehicle.

3. The computer implemented method of claim 1, wherein the driving context data is obtained from a plurality of IoT sensors detecting data from at least one member selected from the group consisting of weather conditions, driving route attributes and passenger information.

4. The computer implemented method of claim 1, wherein the associating step comprises applying one or more of data transformation and data interpolation.

5. The computer implemented method of claim 1, wherein the step of constructing the sequence table further comprises enframing the calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data to define statistical feature data, the statistical feature data comprising at least one member selected from the group consisting of mean, standard deviation and quantiles.

6. The computer implemented method of claim 1, wherein the step of feeding the sequence table into the deep neural network model comprises applying one dimensional convolutions and poolings along a temporal axis of the plurality of windows.

7. The computer implemented method of claim 1, wherein software is provided as a service in a cloud environment.

8. A system for determining driving behavior characteristics of a driver of a vehicle, comprising:

one or more storage devices;

one or more hardware processors coupled to the one or more storage devices;

one or more hardware processors operable to obtain vehicle operational data and driving context data from one or more monitoring systems over a plurality of temporal data points;

one or more hardware processors operable to convert the obtained vehicle operational data and driving context data into sequential vehicle operational feature data and sequential driving context feature data;

one or more hardware processors operable to associate the sequential vehicle operational feature data to the sequential driving context feature data of corresponding temporal data points of the plurality of temporal data points to form calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data;

one or more hardware processors operable to construct a sequence table of a sequence of the plurality of temporal data points based on the calibrated sequential vehicle operational feature data and the calibrated sequential driving context feature data associated with each of the plurality of temporal data points, the sequence table being constructed by combining segmented calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data from a plurality of windows, each window corresponding to a respective one temporal sample point;

one or more hardware processors operable to feed the sequence table into a deep neural network model for applying network learning to form a trained deep neural network model;

one or more hardware processors operable to extract driving behavior features from the trained deep neural network model; and

one or more hardware processors operable to automatically determine driving behavior characteristics of the driver in the future based on the extracted driving behavior features.

9. The system of claim 8, wherein the vehicle operational data comprises GPS vehicle location data and the sequential vehicle operational data comprises at least one member selected from the group consisting of sample points of instantaneous speed, acceleration, angular speed and a difference of each sample point from a normal instantaneous speed, acceleration, angular speed and wherein the vehicle operational data comprises OBD data obtained from the vehicle.

10. The system of claim 8, wherein the driving context data is obtained from a plurality of IoT sensors detecting data from at least one member selected from the group consisting of weather conditions, driving route attributes and passenger information.

11. The system of claim 8, wherein the one or more hardware processors operable to associate is operable to apply one or more of data transformation and data interpolation.

12. The system of claim 8 , wherein the one or more hardware processors operable to construct the sequence table is further operable to enframe the calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data to define statistical feature data, the statistical feature data comprising at least one member selected from the group consisting of mean, standard deviation and quantiles.

13. The system of claim 8 , wherein the one or more hardware processors operable to feed the sequence table into the deep neural network model is further operable to apply one dimensional convolutions and poolings along a temporal axis of the plurality of windows.

14. A computer readable storage medium storing a program of instructions executable by a machine to perform a method of determining driving behavior characteristics of a driver of a vehicle, the method comprising:

obtaining vehicle operational data and driving context data from one or more monitoring systems over a plurality of temporal data points;

converting the obtained vehicle operational data and driving context data into sequential vehicle operational feature data and sequential driving context feature data;

associating the sequential vehicle operational feature data to the sequential driving context feature data of corresponding temporal data points of the plurality of temporal data points to form calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data;

constructing a sequence table of a sequence of the plurality of temporal data points based on the calibrated sequential vehicle operational feature data and the calibrated sequential driving context feature data associated with each of the plurality of temporal data points, the sequence table being constructed by combining segmented calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data from a plurality of windows, each window corresponding to a respective one temporal sample point;

feeding the sequence table into a deep neural network model for applying network learning to form a trained deep neural network model;

extracting driving behavior features from the trained deep neural network model; and

automatically determining driving behavior characteristics of the driver in the future based on the extracted driving behavior features.

15. The computer readable storage medium of claim 14, wherein the associating step comprises applying one or more of data transformation and data interpolation.

16. The computer readable storage medium of claim 14 , wherein the step of constructing the sequence table further comprises enframing the calibrated sequential vehicle operational feature data and calibrated sequential driving context feature data to define statistical feature data, the statistical feature data comprising at least one member selected from the group consisting of mean, standard deviation and quantiles.

17. The computer readable storage medium of claim 14, wherein the step of feeding the sequence table into the deep neural network model comprises applying one dimensional convolutions and poolings along a temporal axis of the plurality of windows.