Method and system for combining sensor data

• US 8,781,688 B2
• Filed: 11/07/2008
• Issued: 07/15/2014
• Est. Priority Date: 11/07/2008
• Status: Active Grant

• Current Assignee: Volvo Lastvagnar AB (Volvo AB)
• Original Assignee: Volvo Lastvagnar AB (Volvo AB)
• Inventors: Sandblom, Fredrik
• Primary Examiner(s): Olszewski, John R
• Assistant Examiner(s): Do, Truc M
• Application Number:
  US13/127,983
• Publication Number:
  US 20120083974A1
• Time in Patent Office: 2,076 Days
• Field of Search: 701/1, 701/45, 701/47, 701/301, 340/435, 340/436, 340/439, 340/576, 340/903
• US Class Current: 701/45
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60W 2050/0029 Mathematical model of the driver
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method is provided for combining sensor data collected by at least two sensors coupled to at least one application, wherein at least one of the sensors provides driver-related sensor data of a driver-related behavior and at least one external sensor provides sensor date not related to driver-related behavior. The sensor data of the at least two sensors are combined as the respective measurement errors of the data are uncorrelated in time with respect to the at least one application.

Claims

1. A method for combining sensor data collected by at least two sensors coupled to at least one application, comprising

monitoring, using at least one of the at least two sensors, driver behaviour and providing driver-related sensor data of the driver-related behaviour,

monitoring, using at least one external sensor of the at least two sensors that is not monitoring the driver behaviour and providing sensor data not related to driver-related behaviour,

feeding sensor data of the at least two sensors to a sensor fusion unit and combining and/or exchanging the sensor data of the at least two sensors in the fusion unit when respective measurement errors of the data are uncorrelated in time and to each other and

transmitting the sensor data to one or more applications subsequent to the sensor fusion unit, and correlating in time estimation errors after processing in the sensor fusion unit.

2. The method according to claim 1, wherein the sensor data are pre-processed before fed into the sensor fusion unit.

3. The method according to claim 1, wherein driver-related sensor data. comprises at least driver attention and/or driver gaze direction data.

4. The method according to claim 1, wherein the driver-related sensor data and data derived from the application are used to estimate the driver'"'"'s field of view.

5. The method according to claim 1, wherein a reaction time of the driver and/or a distraction level of the driver is derived from a comparison between driver-related sensor data indicating cognitive detection of an object and detection of the object by the at least one external sensor.

6. The method according to claim 1, wherein a reaction time is extracted from a time difference between a detection of an object by the at least one external sensor and driver-related sensor data indicating cognitive detection of the same object (60) by the driver.

7. The method according to claim 6, wherein the at least one application is a vehicular safety system and the reaction time of one or more drivers is extracted and stored for use in the vehicular safety system.

8. The method according to claim 6, wherein the reaction time is used to adapt a sensitivity level of the safety system, and/or the reaction time is used to evaluate a level of driver attention.

9. The method according to claim 1, wherein the at least one application is a vehicular safety system.

10. The method according to claim 1, wherein the driver-related sensor data are combined with tracking data of one or more objects, wherein in a tracking system a sensor model is built which at least comprises one or more of

a probability that the driver notices one or more object;

an accuracy of a driver monitoring camera presenting a driver'"'"'s head and/or gaze direction;

a relation between head/gaze direction and position of one or more objects (60) viewed by the driver;

a probability that the driver looks at non-objects;

an impact on track confidence as a function of driver attention.

11. The method according to claim 10, wherein a track is recognized as valid if eternal sensor data and driver-related sensor data coincide.

12. A safety system of a vehicle employing the method according to claim 1.

13. A tracking system of a vehicle employing the method according to claim 1.

14. Computer comprising a. computer program code adapted to perform a method or for use in a method according to claim 1.

15. Computer program product stored on a non-transitory computer readable medium, comprising a program code for use in a method according to claim 1.