Camera-based precrash detection system

• US 6,838,980 B2
• Filed: 05/22/2001
• Issued: 01/04/2005
• Est. Priority Date: 05/24/2000
• Status: Active Grant

• Current Assignee: Daimler AG
• Sponsoring Entity: Daimler Chrysler Company LLC (Fiat Chrysler Automobiles N.V.)
• Inventors: Woltermann, Bernd, Gloger, Joachim, Oberlaender, Matthias
• Primary Examiner(s): Wu, Daniel
• Assistant Examiner(s): TANG, SON M
• APPLICATION: US 20010045981A1
• Time in patent office: 1323 Days
• Field of Search: 340/435, 340/436, 340/903, 340/937, 340/928, 382/104, 382/107, 382/224, 348/148, 348/171, 348/149, 348162-164, 348/135, 348/140, 348/142, 348/143, 348/150, 348/169, 348/159
• US Class Current: 340/435
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/0134 responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: G06K 9/00 Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints processing or analysis of tracks of nuclear particles G01T5/02 ; information retrieval G06F16/00; radio frequency identification G06K7/00; recognition of barcodes and similar code images G06K7/10; computer systems based on specific computational models G06N; image analysis, inspection, positioning or tracking G06T7/00; recognition of acoustic speech signals G10L15/00; acoustic speaker identification G10L17/00
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A method and a device for detecting road users and obstacles on the basis of camera images, in order to determine their distance from the observer and to classify them. In a two-step classification, potential other parties involved in a collision are detected and identified. In so doing, in a first step, potential other parties involved in a collision are marked in the image data of a mono-image camera; their distance and relative velocity are subsequently determined so that endangering objects can be selectively subjected to a type classification in real time. By breaking down the detection activity into a plurality of steps, the real-time capability of the system is also rendered possible using conventional sensors already present in the vehicle.

Claims

1. A method for detecting road users and obstacles as a function of camera images so as to determine their distance from an observer and to classify them, comprising the steps of:

identifying regions within a two-dimensional camera image that is not resolved with respect to distance using a classifier designed for detecting road users and obstacles;

marking and ranging, in a subsequent step, the identified regions using a distance-measuring sensor with respect to their distance from the observer; and

subsequently type classifying the identified regions using a type classifier.

2. The method as recited in claim 1 wherein the camera image contains only two-dimensional image information without any distance resolution.

3. The method as recited in claim 1 wherein information obtained the marking and ranging step is used to determine a relative velocity of the individual road users or obstacles.

4. The method as recited in claim 1 wherein the classifier designed for recognizing road users is a hyperpermutation network.

5. The method as recited in claim 1 wherein a box algorithm is used to mark regions identified as road users.

6. The method as recited in claim 1 wherein the distance-measuring sensor is a radar sensor.

7. The method as recited in claim 1 wherein the distance-measuring sensor is a stereo-camera system.

8. The method as recited in claim 1 wherein the distance-measuring sensor is a mono-camera system, which, by using suitable image processing, is able to make distance estimates.

9. The method as recited in claim 8 wherein the mono-camera system is used in the identifying step to generate the two-dimensional camera image.

10. The method as recited in claim 1 wherein the type classifier is a radial-basis function classifier.

11. The method as recited in claim 1 wherein the type classifier is a support vector machine.

12. The method as recited in claim 1 wherein the regions to be subjected to a type classification are selected as a function of at least one of distance and relative velocity in relation to the observer.

13. The method as recited in claim 1 wherein the selection of the regions to be subjected to a type classification includes all regions identified as road users or obstacles.

14. The method as recited in claim 1 wherein the result of the type classification is transmitted to a risk calculator to decide on reactions to be possibly initiated.

15. The method as recited in claim 1 further comprising using the method for early detection of accident situations.

16. A device for detecting road users and obstacles as a function of camera images to determine their distance from an observer, and to classify them, comprising:

a distance-measuring sensor unit;

a mono-image camera coupled to the distance-measuring sensor unit;

a first classifying unit interposed between the sensor unit and the camera; and

a second classifying unit downstream from the sensor unit and the camera.

17. The device as recited in claim 16 further comprising a selection unit and wherein the mono-image camera is linked to the first classifying unit, the first classifying unit containing a module for identifying image regions to be assigned to road users and obstacles, and marking these regions and making available corresponding data at an output of the module for a further processing;

and coupled to the output of the module is the distance-measuring sensor unit, which is able to measure the marked regions with respect to their distance from the observer, and which makes available these measured data via a connection to the selection unit, via which the second classifying unit is linked and classifies the regions supplied to it by the selection unit with respect to type of road user or obstacle.

18. The device as recited in claim 16 further comprising a risk calculator is connected to the output of the second classifying unit.

19. The device as recited in claim 16 wherein the device is located in a vehicle and used for early detection of accident situations.