SYSTEMS AND METHODS FOR DETECTING SURPRISING EVENTS IN VEHICLES

• US 20170113664A1
• Filed: 10/21/2016
• Published: 04/27/2017
• Est. Priority Date: 10/23/2015
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Nix, Axel
• Granted Patent: US 10,427,655 B2
• Time in Patent Office: Days
• CPC Class Codes: B60T 7/12 for automatic initiation; for initiation not subject to will of driver or passenger limiting speed of vehicles other than rail vehicles B60K31/00
• CPC Class Codes: B60T 8/171 Detecting parameters used in the regulation; Measuring values used in the regulation
• CPC Class Codes: B60T 8/17551 determining control parameters related to vehicle stability used in the regulation, e.g. by calculations involving measured or detected parameters
• CPC Class Codes: B60T 8/17558 specially adapted for collision avoidance or collision mitigation road vehicle drive control systems for collision avoidance otherwise than by controlling a particular sub-unit B60W30/09
• CPC Class Codes: B60T 8/241 Lateral vehicle inclination
• CPC Class Codes: B60T 8/32 responsive to a speed condition, e.g. acceleration or deceleration using electrical circuitry or regulation means B60T8/17 ; B60T8/28 takes precedence; electric devices on electrically propelled vehicles indicating the wheel slip B60L3/10; measuring linear or angular speed per se G01P3/00
• CPC Class Codes: B60W 2050/0018 Method for the design of a control system
• CPC Class Codes: B60W 2050/041 Built in Test Equipment [BITE]
• CPC Class Codes: B60W 2050/046 involving external transmission of data to or from the vehicle, e.g. via telemetry, satellite, Global Positioning System [GPS]
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 60/0016 : of the vehicle or its occupants
• CPC Class Codes: B60W 60/0051 : from occupants to vehicle
• CPC Class Codes: G06F 11/3072 : where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
• CPC Class Codes: G06V 20/41 : Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 : Event detection
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G07C 5/008 : communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0841 : Registering performance data recording measured values G01D; information storage G11B
• CPC Class Codes: G07C 5/0866 : the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: G07C 5/0891 : Video recorder in combination with video camera
• CPC Class Codes: G08B 13/19669 : Event triggers storage or change of storage policy
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 : Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A driver assistance system is presented, including an event detector which comprises a set of rules defining a surprising event based on signals reflecting a vehicle operator input signals from an object detection sensor. An event data file generator is configured to generate an event data file according to rules comprised in the event detector, the event data file comprising a video signal received from a camera, a signal from at least one dynamic vehicle sensor, and target object information received from the object detection sensor. The event data file generator is further configured to initiate data file generation responsive to a surprising event being detected by the event generator, and wherein the contents of the data file are specified by the rules comprised in the event detector. In this way, surprising events may be collected and analyzed off-board in order to generate updates for the driver assistance system.

Claims

1. A surprising event detection system for a vehicle having a driver assistance system that controls lateral and/or longitudinal movement of the vehicle, comprising:

an event detector comprising a set of rules defining a surprising event based on a combination of at least two input signals;

an event data file generator configured to generate an event data file comprising;

a video signal received from a camera;

a signal from at least one dynamic vehicle sensor; and

object information received from an object detection sensor,

wherein the event data file generator is further configured to initiate event data file generation responsive to a surprising event being detected by the event generator; and

a fusion and control module configured to periodically export event data to a server.

2. The surprising event detection system as in claim 1, wherein the combination of at least two input signals comprises at least one signal reflecting a vehicle operator input and at least one signal from the object detection sensor, and wherein the driver assistance system includes an automatic emergency braking mechanism.

3. The surprising event detection system as in claim 1, wherein the combination of at least two input signals comprises at least one signal received over an extra-vehicle communication system from a second vehicle.

4. The surprising event detection system as in claim 1, wherein the combination of at least two input signals are transmitted on a vehicle data bus and are selected from the group consisting of a vehicle speed signal, a vehicle yaw rate signal, a steering wheel angle signal, a vehicle brake pedal position signal, a vehicle accelerator pedal position signal, a preceding vehicle distance signal, a preceding vehicle relative velocity signal, a turn indicator signal, a steering torque request signal, a vehicle acceleration request signal, and an airbag deployment signal.

5. The surprising event detection system as in claim 1, wherein the combination of at least two input signals comprises at least one signal from the object detection sensor and at least one signal from a second object detection sensor.

6. The surprising event detection system as in claim 1, further comprising a video compressor, wherein the video signal in the event data file has been compressed by the video compressor.

7. The surprising event detection system as in claim 1, wherein the combination of at least two input signals comprises:

at least one signals selected from the group consisting of a steering wheel angle signal, a vehicle brake pedal position signal, and a vehicle accelerator pedal position signal; and

at least one signal selected from the group consisting of a steering torque request, a steering angle request, and an acceleration request.

8. The surprising event detection system as in claim 1, wherein the rules defining the surprising event are associated with rules defining the event data file, and wherein the rules defining the event data file comprise at least one of:

a length of time to be recorded before detection of the surprising event,

a length of time to be recorded after detection of the surprising event,

a frame rate of video to be recorded, and

an identification of signals to be recorded.

9. The surprising event detection system as in claim 1,

further comprising a rolling event data buffer which receives signals from a vehicle data bus,

wherein the event data file comprises data present in the rolling event data buffer at the time of detecting a surprising event and data received by the surprising event detection system after the time of the surprising event.

10. The surprising event detection system of claim 1, further comprising an event data uploader configured to upload an event data file to the server based on the set of rules comprised in the event detector.

11. The surprising event detection system of claim 10, wherein the rules comprised in the event detector indicate a priority for the detected surprising event.

12. The surprising event detection system of claim 10, wherein the rules comprised in the event detector indicate a preferred data transfer network for uploading the event data file to the driver assistance server.

13. The surprising event detection system of claim 1, wherein the object detection sensor is a radar sensor, a lidar sensor, or a machine vision camera.

14. The surprising event detection system of claim 1, wherein the event data file is asymmetrically encrypted with the public key of a designed recipient.

15. The surprising event detection system of claim 1, further comprising a driver assistance module updater configured to receive an update patch from the server, and to apply the update patch to the surprising event detection system, and wherein the update patch comprises updated rules defining a surprising event to be stored at the event detector.

16. A method for detecting surprising events in a vehicle having a driver assistance system that controls lateral and/or longitudinal movement of the vehicle, comprising:

receiving signal data from one or more onboard vehicle sensors and one or more object identification sensors;

receiving video data from one or more video cameras;

storing the signal data and the video data in an event data buffer;

identifying a surprising event from the received signal data based on surprise event rules stored at an event detector;

deriving an event data file for the surprising event based on the surprise event rules;

uploading the event data file to a server;

receiving an update module from the server, comprising updated surprise event rules; and

applying the update module to change the surprise event rules.

17. The method of claim 16, wherein receiving signal data from one or more onboard vehicle sensors and one or more object identification sensors further comprises at least one of:

receiving data from an on-board radar sensor;

receiving data from an on-board machine vision camera;

receiving data from a dynamic vehicle sensor;

receiving data from a vehicle navigation system;

receiving data from a vehicle-to-vehicle communication module; and

receiving data from a fusion and control module configured to adjust a parameter of one or more vehicle actuators responsive to signals received from the object identification sensors.

18. The method of claim 16, wherein receiving video data from one or more video cameras comprises at least one of:

receiving video from a forward facing camera mounted to windshield;

receiving video from a forward facing camera mounted near the front of the vehicle;

receiving video data from a rearward facing side mirror replacement camera; and

receiving video from a rearward facing camera mounted near the rear of the vehicle.

19. The method of claim 16, further comprising the steps

executing, on the server, a machine vision algorithm to video data contained within the event data file;

and

associating annotation information with the event data file.

20. The method of claim 16, wherein the surprise event rules are selected to cause, on average, no more than one event per week to be uploaded to the server.