System and method for real-time recognition of driving patterns

• US 6,879,969 B2
• Filed: 01/19/2002
• Issued: 04/12/2005
• Est. Priority Date: 01/21/2001
• Status: Expired due to Fees

• Current Assignee: Volvo Technology Corporation
• Original Assignee: Volvo Technological Development Corporation
• Inventors: Engström, Johan, Victor, Trent
• Primary Examiner(s): Knight, Anthony
• Assistant Examiner(s): Holmes, Michael B.
• Application Number:
  US09/683,571
• Publication Number:
  US 20020128751A1
• Time in Patent Office: 1,179 Days
• Field of Search: 706 15- 44
• US Class Current: 706/20
• CPC Class Codes: B60T 2220/02 Driver type; Driving style; Driver adaptive features
• CPC Class Codes: B60W 2510/0638 Engine speed
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2530/00 Input parameters relating to vehicle conditions or values, not covered by groups B60W2510/00 or B60W2520/00
• CPC Class Codes: B60W 2540/12 Brake pedal position
• CPC Class Codes: B60W 2540/18 Steering angle
• CPC Class Codes: B60W 2540/20 Direction indicator values
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2552/05 Type of road, e.g. motorways, local streets, paved or unpaved roads
• CPC Class Codes: B60W 30/18181 Propulsion control with common controlling member for different functions
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/0104 Measuring and analyzing of parameters relative to traffic conditions

Abstract

System and method for real-time, automatic, recognition of large time-scale driving patterns employs a statistical pattern recognition framework, implemented by means of feed-forward neural network utilizing models developed for recognizing, for example, four classes of driving environments, namely highway, main road, suburban traffic and city traffic, from vehicle performance data. A vehicle control application effects changes in vehicle performance aspects based on the recognized driving environment.

Claims

1. A computer implemented method for optimizing driver-vechicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vechicle is presently being driven; and

wherein the measurement of the at least one driver characteristic is made by direct body scan of the driver.

2. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

wherein the measurement of the at least one vehicle performance characteristic is at least one of (i) non-GPS based and (ii) geographically unspecific.

3. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

wherein a resulting data set from the collection of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic has a capacity to be statistically segregated into a plurality of driving environment categories.

4. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

wherein the measurement of at least one vehicle characteristic comprises quantification of at least one of acceleration pedal position, gear selection, turn indicator activity, vehicle speed, steering angle, engine speed and brake activity.

5. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

utilizing as reference data, annotated data values incorporating a driver indication of driver environment existing at the time a respective annotated data value was collected thereby enabling look-up analysis of each real-time collected measurement.

6. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising,

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

wherein the at least one driver characteristic is driver eye movement.

7. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

wherein the at least one driver characteristic is driver head movement.

8. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

utilizing predetermined criteria for predicting driving environments, the predetermined criteria differentiating between at least two of a plurality of driving environments including highway driving, main road driving, suburban driving and city driving.

9. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is resents being driven; and

collecting and recording a plurality of measurements of a plurality of driver characteristics and a plurality of vehicle characteristics from a plurality of subjects thereby creating a collection of reference values for the driver and vehicle characteristics.

10. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

collecting and recording a plurality of measurements of a plurality of driver characteristics and a plurality of vehicle characteristics from a plurality of subjects driving a plurality of routes thereby creating a collection of reference values for the driver and vehicle characteristics.

11. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting on a substantially real-time basis a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

considering at least acceleration pedal position, gear selection, turn indicator activity, vehicle speed, steering angle, engine speed and brake activity in the evaluation.

12. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

considering at least one of acceleration pedal position, gear selection, turn indicator activity, vehicle speed, steering angle, engine speed and brake activity in the evaluation.

13. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

analyzing in a pre-processing step over a predetermined time window, a series of iteratively collected measurements of at least one of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic for purposes of feature extraction.

14. The method of claim 13, wherein said analysis in the preprocessing step comprises computing an average of the collected measurements of at least one of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic.

15. The method of claim 14, further comprising:

determining the probable driving environment occurring during the predetermined time window based on the computed average of the collected measurements of at least one of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic.

16. The method of claim 15, wherein said predetermined time window is sufficiently long to determine the driving environment occurring during the predetermined time window while avoiding identification of a small-time scale driving pattern.

17. The method of claim 15, wherein said predetermined time window is sufficiently short to determine a small-time scale driving pattern occurring during the predetermined time window.

18. The method of claim 17, wherein said determined small-time scale driving pattern identifies at least one of the conditions of turning the vehicle, changing lanes in the vehicle and passing another vehicle.

19. The method of claim 13, wherein said analysis in the preprocessing step comprises computing a standard deviation of the collected measurements of at least one of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic.

20. The method of claim 15, further comprising:

determining the probabilistic driving environment occurring during the predetermined time window based on the averaged value.

21. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic;

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

ascertaining a probability of a particular driving environment occurring during a predetermined time window utilizing a neural network to analyze the plurality of collected measurements.

22. The method of claim 21, further comprising:

performing statistical pattern recognition utilizing the neural network.

23. The method of claim 21, wherein the ascertainable driving environments include at least one of a highway driving environment, a main road driving environment, a suburban driving environment, and a city driving environment.

24. A computer implemented method for optimizing driver-vehicle performance in a driver operated vehicle, said method comprising:

collecting, on a substantially real-time basis, a plurality of measurements of at least one driver characteristic and at least one vehicle characteristic; and

evaluating said plurality of measurements to predict a current driving environment within which the vehicle is presently being driven; and

effecting changes in performance characteristics of the vehicle based on the evaluation of the plurality of measurements of at least one driver characteristic and at least one vehicle characteristic.

25. The method of claim 24, wherein effecting the changes in vehicle performance characteristics further comprises adapting the performance of at least one of (i) an engine and (ii) a chassis of the vehicle based on recognition of a particular driving environment.

26. The method of claim 25, wherein effecting the changes in performance characteristics of the vehicle aspects further comprises effecting real-time optimization of at least one of an engine parameter and a chassis parameter to a predicted current driving environment.

27. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of(i) a non-GPS-based, geographically unspecific vehicle characteristic and (ii) a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

computing statistical characteristics of the data set including at least one of the parameters average magnitude, variability and change rate of the data set.

38. The method as recited in claim 27, wherein the small-scale predetermined time window has a period of approximately 40 seconds.

28. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving, environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of (i) a non-GPS-based, geographically unspecific vehicle characteristic and (ii) a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set; and

ascertaining a large time-scale, driving pattern occurring during the collection of the data set based on the analysis; and

categorizing the ascertained large time-scale driving pattern occurring during the collection of the data set into one category, among a plurality of categories, that is representative of the driving environment occurring during the collection of the data set.

29. The method as recited in claim 28, further comprising:

defining the plurality of categories to differentiate between (i) a highway driving environment, (ii) a suburban driving environment and (iii) a city driving environment.

30. The method as recited in claim 28, further comprising:

classifying a highway driving environment as one of the plurality of possible categories of driving environment occurring during the collection of the data set.

31. The method as recited in claim 28, further comprising:

classifying a main road driving environment as one of the plurality of possible categories of driving environment occurring during the collection of the data set.

32. The method as recited in claim 28, further comprising:

classifying a suburban driving environment as one of the plurality of possible categories of driving environment occurring during the collection of the data set.

33. The method as recited in claim 28, further comprising:

classifying a city driving environment as one of the plurality of possible categories of driving environment occurring during the collection of the data set.

34. The method as recited in claim 28, further comprising:

basing the categorization on a probability calculated from the data set.

44. The method as recited in claim 28, further comprising:

effecting changes in vehicle performance aspects based on the ascertained large time-scale driving pattern for potentiating vehicle performance in the categorized driving environment.

35. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of (i) a non-GPS-based geographically unspecific vehicle characteristic and (ii) a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis;

producing a data stream constituted at least in part by iterative measurements of the at least one of (i) a non-GPS-based, (ii) geographically unspecific vehicle characteristic and (iii) a physical characteristic of an operator of an operator-driven vehicle; and

selecting members of the data set based on application of a large-scale predetermined time window to the data stream.

36. The method as recited in claim 35, wherein the large-scale predetermined time window has a period of approximately 400 seconds.

37. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of (i) a non-GPS-based, geographically unspecific vehicle characteristic and (ii) a physical characteristic of ail operator of an operator-driven vehicle and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

producing a data stream constituted at least in part by iterative measurements of the at least one of (i) a non-GPS-based, (ii) geographically unspecific vehicle characteristic and (iii) a physical characteristic of an operator of an operator-driven vehicle; and

ascertaining a small tune-scale driving pattern occurring during the collection of the data set by analysis of a sub-set sampling therefrom based on application of a small-scale predetermined time window to the data stream.

39. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of (i) a non-GPS-based, geographically unspecific vehicle characteristic and (ii) a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining, a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

wherein each member of the data set represents a discrete quantification of at least one vehicle characteristic selected from a group of vehicle characteristics including (i) accelerator pedal position, (ii) gear selection, (iii) turn indicator position, (iv) vehicle speed, (v) steering angle, (vi) engine speed and (vii) brake activity.

40. The method as recited in claim 39, wherein the physical characteristic of the operator is eye orientation.

41. The method of claim 40, wherein the measurement of the operator'"'"'s eye orientation is made by direct body scan of the driver.

42. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving, environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, at least one of (i) a non-GPS-based, geographically unspecific vehicle characteristic and (ii) a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

wherein the physical characteristic of the operator is head orientation.

43. The method of claim 42, wherein the measurement of the operator'"'"'s head orientation is made by direct body scan of the driver.

45. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, a vehicle characteristic and collecting therefrom a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

wherein said vehicle characteristic is at least one of (i) non-GPS-based and (ii) geographically unspecific.

46. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

wherein said physical characteristic of the operator is head movement.

47. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

wherein said physical characteristic of the operator is eye movement.

48. A computer implemented method for ascertaining, on an essentially real-time basis, large time-scale driving patterns indicative of the current driving environment of an operator-driven vehicle, said method comprising:

repetitively sensing, on an essentially real-time basis, a physical characteristic of an operator of an operator-driven vehicle, and therefrom collecting a data set for statistical pattern recognition analysis;

performing statistical pattern recognition analysis on the data set;

ascertaining a large time-scale driving pattern occurring during the collection of the data set based on the analysis; and

determining a current driving environment from said ascertained large time-scale driving pattern.