Method and system for inferring the behavior or state of the driver of a vehicle, use of the method and computer program for carrying out the method

• US 9,663,047 B2
• Filed: 02/01/2012
• Issued: 05/30/2017
• Est. Priority Date: 02/01/2012
• Status: Active Grant

• Current Assignee: FICO Mirrors SA (Panasonic Holdings Corporation)
• Original Assignee: FICO Mirrors SA (Panasonic Holdings Corporation)
• Inventors: Bande Martnez, Daniel, Bartra Cisa, Ariadna, Fernndez Chimeno, Mireya, Marco Cols, Santiago, Ramos Castro, Juan
• Primary Examiner(s): Blount, Eric M
• Application Number:
  US14/376,020
• Publication Number:
  US 20150029014A1
• Time in Patent Office: 1,945 Days
• Field of Search: 340438, 340439, 340575, 340576
• CPC Class Codes: B60R 16/0232 for measuring vehicle parameters and indicating critical, abnormal or dangerous conditions
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08B 21/18 Status alarms G08B21/02 takes precedence

Abstract

The invention relates to a method and system for inferring the behavior or state of the driver of a vehicle, use of the method and computer program for carrying out the method

The method comprises evaluating a series of descriptive functions of the conditions of interaction of the driver with the vehicle based at least on information from inertial sensors of an autonomous computation device, by comparing such evaluation with reference information associated with a behavior of the driver that is considered normal.

The system is suitable for implementing the method of the invention.

The use is for detecting behavioral changes of a driver.

The computer program is suitable for performing the steps of the method of the invention.

Claims

1. A method for inferring the behavior or state of the driver of a vehicle with the aid of an autonomous computation device located in the vehicle, integrating sensors that can report on the motion and operating state of the vehicle, wherein said sensors are inertial sensors comprising accelerometers and/or gyroscopes, and the method comprises:

a) processing at least part of the information from one or several of said inertial sensors of said autonomous computation device, one by one or in combination, to make an evaluation of at least one descriptive function of an interaction or behavior of the driver with the vehicle over time, in each phase of a trajectory and for a specific road; and

b) inferring said behavior or state by comparing the evaluation of said at least one descriptive function, or at least a value thereof, with reference information associated with a behavior of the driver that is considered normal for the specific road, said reference information being stored in at least one memory of said autonomous computation device, or in a remote device or database,

wherein said reference information comprises reference values obtained by evaluating said at least one descriptive function, in characterization periods and in zones in which specific conditions relating to the speed of the vehicle and/or radius of curvature are met, said method comprising performing the comparison of said step b) of the evaluation of said descriptive function in step a) and said reference values.

2. The method according to claim 1, wherein said reference information further comprises reference values, the method comprising performing said comparison of step b) of the value of said at least one descriptive function, and said reference values.

3. The method according to claim 1, wherein said step a) comprises obtaining a plurality of descriptive functions and said step b) comprises comparing the evaluation thereof with said reference information.

4. The method according to claim 1, wherein said descriptive function is a combined descriptive function, the method comprising evaluating it in step a) combining a plurality of individual descriptive functions.

5. The method according to claim 1, further comprising:

determining that the behavior of the driver of the vehicle is abnormal if the result of said comparison of step b) offers at least one discrepancy greater than a specific discrepancy value and generating a warning alarm signal if a discrepancy log shows an abnormal driving interval frequency greater than a specific frequency.

6. The method according to claim 5, further comprising:

sending the warning alarm signal to the driver and/or to a remote control center where it is collected in a selectively accessible database together with at least trajectory and date data.

7. The method according to claim 1 wherein said descriptive function is at least one of the following descriptive functions:

standard deviation of the lateral position of the car, entropy of the lateral deviation, number of zigzags, frequency of zigzags, maximum lateral speed, mean lateral speed, maximum acceleration in the direction of motion, mean speed, maximum speed variation, number of peaks in angular acceleration and number and intensity of pitches and rolls, or a combination thereof.

8. The method according to claim 1, wherein said autonomous computation device is a device with telecommunication capacity, selected from a mobile telephone, a personal digital assistant, a navigation device, a TCU, or telecommunications unit of the vehicle, and wherein the method comprises said autonomous computation device receiving information from at least one of the following additional sensors:

receiver/locator based on a global navigation satellite system, temperature sensor, sensor associated with speed, compass, sensor associated with detecting radii of curvature, sensor associated with detecting lane width, sensor associated with detecting the quality and/or type of road, altitude sensor, video camera integrated in the autonomous computation device for detecting the biological constants of the user, sensor associated with a system for detecting obstacles or lane departure or a combination thereof.

9. The method according to claim 8, wherein said autonomous computation device comprises at least one of said additional sensors.

10. The method according to claim 8, wherein said additional sensors are external to the autonomous computation device, and the method comprises the autonomous computation device receiving said information from the mentioned additional sensors through a telematic means using said telecommunication capacity, or setting up a connection with a communications network of the vehicle.

11. The method according to claim 1, wherein said determination of the behavior or state of the driver is inferred by additionally taking into account the processing of physiological information of the driver.

12. The method according to claim 11, further comprising:

obtaining said physiological information of the driver through at least one physiological sensing device, said physiological sensing device being in contact with or close to the driver and selected from;

a contact sensor comprising an inductive band, piezoelectric band or resistive band, electrodes, temperature sensor, or a contactless sensor comprising a resistive device integrated in the vehicle seat belt, a bioimpedance sensor or array of bioimpedance sensors, a camera aiming the user to detect his/her biological constants or a radar device.

13. The method according to claim 12, wherein said autonomous computation device and said at least one physiological sensor, or an electronic system connected thereto, have wireless communication capacity, the method comprising wirelessly acquiring by means of said autonomous computation device said physiological information of the driver from said at least one physiological sensor, or from said electronic system connected thereto, and in that said physiological sensor incorporates a driver movement and/or vibration compensation module.

14. The method according to claim 12, further comprising:

comparing said information about the physiological state of the driver with reference values relating to a physiological state of the driver associated with a behavior of the driver that is considered normal, said reference values being stored in a memory of the physiological sensing device, in a memory of the autonomous computation device, or in a remote device, and

establishing that the behavior of the driver of the vehicle is abnormal if the result of said comparison of information about the physiological state of the driver offers at least one discrepancy greater than a specific discrepancy value and generating a warning alarm signal at least for the driver if it is determined that a discrepancy log shows an abnormal sequence.

15. The method according to claim 14, wherein a warning signal is generated as a result of an evaluation of the information from the inertial sensors of the autonomous device, and/or from the physiological sensors or from the additional sensors.

16. The method according to claim 1, further comprising:

generating said reference values relating to operating states of the vehicle and/or said reference values relating to a physiological state of the driver by means of an initial characterization process comprising a user providing an indication of data about his/her state at a specific time and habits through an input of the autonomous computation device, or by means of the inclusion thereof by default in a computer program installed in the autonomous device, or through a database or a sensor associated with the user, comprising several previously established states, and

performing said generation of values for each of a plurality of drivers and/or driving profiles and/or operating environments comprising data about the road being traveled on.

17. The method according to claim 16, wherein the coherence of the data entered by a user is checked against the behavior of said user inferred through the dynamics and/or operating state of the vehicle or through his/her biological data for a check period having a predetermined duration.

18. The method according to claim 8, further comprising:

the autonomous computation device determining the operating state or environment of the vehicle from at least the information received by said and/or other sensors and/or by means of a user providing an indication through an input device of the autonomous computation device or by information relating to the road being traveled on stored in said autonomous computation device or received from an external storage device.

19. The method according to claim 1, wherein for carrying out said determination of the behavior or state of the driver, further comprising using additional information about the environment that can affect the behavior of the driver, chosen from at least one of the following types of information:

time of day, weather conditions and light information, said additional information about the environment being accessed by the autonomous computation device through internal sensors, external sensors and/or from a remote point accessible through communication means of the autonomous computation device.

20. The method according to claim 1, further comprising:

performing a prior step of calibrating at least the inertial sensors of the autonomous computation device, and if during normal operating mode the autonomous computation device identifies that one or more of the sensors provides values outside those expected, the values are temporarily assigned a lower weight in relation to the dynamics and/or operating state of the vehicle that a respective one of the sensors is reporting about.

21. The method according claim 1, wherein said autonomous computation device is fixed to a structure of the interior of the vehicle cabin chosen from the dashboard of the vehicle or a pillar thereof which reduces vibrations.

22. The method according to claim 1, further comprising:

detecting behavioral changes of a driver including;

drowsiness, distraction, stressed states, states of inebriation or under the influence of drugs, using different physiological variables chosen from heart rate, breathing, perspiration and the dynamics and/or operating state of the vehicle.

23. A system for inferring the behavior of the driver of a vehicle, comprising an autonomous computation device arranged in the vehicle with:

processing means (E);

internal sensors (Si1-Si14) connected with said processing means (E) and arranged to detect the state of the vehicle and to supply information relating to said state of the vehicle to said processing means (E); and

at least one physiological sensor (Sbs, Sbc) associated with the driver of the vehicle, intended for detecting a physiological state of the driver and supplying information relating to said physiological state to said autonomous computation device (D), and in that said autonomous computation device (D) and said at least one physiological sensor (Sbs, Sbc), or an electronic system connected thereto, have wireless communication capacity, said at least one physiological sensor (Sbs, Sbc), or said electronic system connected thereto, being intended for wirelessly supplying said information relating to said physiological state to said autonomous computation device (D),

the system wherein;

said autonomous computation device (D) comprises a memory (M) where at least reference information relating to states of the vehicle associated with a behavior of the driver that is considered normal is recorded;

said processing means (E) are communicated with said memory (M) to access said reference information, and they implement a method for inferring the behavior or state of the driver of a vehicle with the aid of an autonomous computation device located in the vehicle, integrating sensors that can report on the motion and operating state of the vehicle, wherein said sensors are inertial sensors comprising accelerometers and/or gyroscopes, and the method comprises;

a) processing at least part of the information from one or several of said inertial sensors of said autonomous computation device, one by one or in combination, to make an evaluation of at least one descriptive function of the interaction or behavior of the driver with the vehicle over time, in each phase of a trajectory and for a specific road; and

b) inferring said behavior or state by comparing the evaluation of said at least one descriptive function, or at least a value thereof, with reference information associated with a behavior of the driver that is considered normal for the specific road, said reference information being stored in at least one memory of said autonomous computation device, or in a remote device or database.

24. A non-transitory tangible computer program product comprising:

computer program code means suitable for performing the steps of a method for inferring behavior or state of a driver of a vehicle with aid of an autonomous computation device located in the vehicle;

integrating sensors that can report on the motion and operating state of the vehicle, wherein said sensors are inertial sensors comprising accelerometers and/or gyroscopes, and the method comprises;

a) processing at least part of the information from one or several of said inertial sensors of said autonomous computation device, one by one or in combination, to make an evaluation at least one descriptive function of the interaction or behavior of a driver with the vehicle over time, in each phase of a trajectory and for a specific road; and

b) inferring said behavior or state by comparing the evaluation of said at least one descriptive function, or at least a value thereof, with reference information associated with a behavior of the driver that is considered normal for the specific road, said reference information being stored in at least one memory of said autonomous computation device, or in a remote device or database, said steps including said evaluation of the at least one descriptive function of an interaction of the driver with the vehicle, in each phase of a trajectory, and treatment of the information about the physiological state of the driver and said determination of the behavior or state of the driver of the vehicle when the aforementioned program is run in a computer, a digital signal processor, an application-specific integrated circuit, a microprocessor, a microcontroller or programmable hardware.