Redundant object detection for driver assistance systems

• US 9,592,764 B2
• Filed: 05/07/2012
• Issued: 03/14/2017
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Randler, Martin, Baratoff, Gregory
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US14/116,478
• Publication Number:
  US 20140160252A1
• Time in Patent Office: 1,772 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A device for redundant object detection includes a stereo camera having two image sensors, a first processor for detecting objects by stereo image evaluation of image data from the two image sensors, and a second processor for classifying the detected objects by evaluation of image data from at least one of the two image sensors. A method and a program for such object detection and classification are also provided.

Claims

1. A device for redundant object detection comprising:

a stereo camera that comprises first and second image sensors configured and arranged to respectively produce, at outputs of the first and second image sensors, first and second mono image data of a scene including objects viewed by the image sensors,

a stereo image processor that is connected to the outputs of the first and second image sensors, and that is configured to detect the objects by a stereo image evaluation of stereo image data represented by a combination of the first and second mono image data from the outputs of the first and second image sensors, and that is configured to provide object data regarding the detected objects at an output of the stereo image processor, and

a mono image processor that has an object data input connected to the output of the stereo image processor, and that has at least one of a first image input connected to the output of the first image sensor and/or a second image input connected to the output of the second image sensor, and that is configured to classify the detected objects represented in the object data received at the object data input, by performing a redundant detection and a

classification of the objects by a mono image evaluation of at least one of the first mono image data from the output of the first image sensor or the second mono image data from the output of the second image sensor.

2. The device according to claim 1, wherein the stereo image processor is configured to generate and provide at the output thereof, as the object data, a detected object list of the objects that have been detected as obstacles which project from a road in the scene, based on disparities and/or optical flow in the stereo image data during the stereo image evaluation.

3. The device according to claim 2, wherein the mono image processor is configured to classify the obstacles in the detected object list by the mono image evaluation of the first mono image data from the first image sensor, and to generate at an output of the mono image processor a classified object list of classified obstacles.

4. The device according to claim 1, further comprising a printed circuit board, wherein the first and second image sensors, the stereo image processor and the mono image processor are all installed in common together on the printed circuit board.

5. The device according to claim 1, wherein the stereo camera including the first and second image sensors, the stereo image processor and the mono image processor are integrated together in a unitary stereo camera module.

6. The device according to claim 5, wherein the output of the stereo image processor and an output of the mono image processor are both lead out of the unitary stereo camera module.

7. The device according to claim 5, wherein the unitary stereo camera module does not include and is not connected to receive any input from any other sensor in addition to the first and second image sensors of the stereo camera.

13. The device according to claim 1, wherein the mono image processor has only the first image input connected to the output of the first image sensor and not the second image input connected to the output of the second image sensor, and is configured to perform the mono image evaluation only on the first mono image data and not on the second mono image data.

14. The device according to claim 1, wherein the mono image evaluation of the first mono image data by the mono image processor is limited to pixels of the first mono image data correlated to object pixels of the detected objects that are detected in the stereo image evaluation of the stereo image data by the stereo image processor.

15. The device according to claim 1, wherein the mono image evaluation by the mono image processor includes the redundant detection of the objects in the first mono image data, correlating the detected objects in the first mono image data with the detected objects represented in the object data provided by the stereo image processor to the object data input, and the classification of the detected objects.

20. The device according to claim 1,

wherein the mono image processor has the first image input connected to the output of the first image sensor and the second image input connected to the output of the second image sensor, and

wherein the mono image processor is configured to perform the mono image evaluation as a purely mono image evaluation of only either the first mono image data from the output of the first image sensor or the second mono image data from the output of the second image sensor at a given time.

8. A method of redundant object detection comprising the steps:

a) with first and second image sensors of a stereo camera respectively producing first and second mono image data of a scene including objects viewed by the image sensors;

b) in a stereo image processor, detecting the objects, by performing a stereo image evaluation of stereo image data represented by a combination of the first and second mono image data produced by the first and second image sensors, and producing object data regarding the detected objects; and

c) in a mono image processor, performing a redundant detection and a classification of the objects by a mono image evaluation of at least one of the first mono image data produced by the first image sensor and or the second mono image data produced by the second image sensor.

9. The method according to claim 8, wherein the step b) comprises generating, as the object data, a detected object list of the objects that have been detected as obstacles which project from a road in the scene, based on disparities and/or optical flow in the stereo image data during the stereo image evaluation.

10. The method according to claim 9, wherein the step c) comprises classifying the obstacles in the detected object list by the mono image evaluation of the first mono image data from the first image sensor, and generating at an output of the mono image processor, a classified object list of classified obstacles.

11. The method according to claim 8, further comprising outputting the object data produced by the stereo image evaluation and a result of the classifying of the detected objects by the mono image evaluation to a driver assistance system of a motor vehicle, and via the driver assistance system influencing an operation of the motor vehicle in response to and dependent on the object data and/or the result.

12. A non-transitory tangible machine-readable storage medium encoded with a computer program which, when read out of the storage medium and executed by the processors connected to the stereo camera, carries out the method according to claim 8.

16. The method according to claim 8, wherein the mono image evaluation is performed only on the first mono image data and not on the second mono image data.

17. The method according to claim 8, wherein the mono image evaluation of the first mono image data by the mono image processor is performed limited to pixels of the first mono image data correlated to object pixels of the detected objects that are detected in the stereo image evaluation of the stereo image data by the stereo image processor.

18. The method according to claim 8, wherein the mono image evaluation in the mono image processor includes the redundant detection of the objects in the first mono image data, correlating the detected objects in the first mono image data with the detected objects represented in the object data provided by the stereo image processor, and the classification of the detected objects.

21. The method according to claim 8, wherein the mono image evaluation is a purely mono image evaluation of only either the first mono image data produced by the first image sensor or the second mono image data produced by the second image sensor at a given time.

19. A method of obstacle detection for motor vehicle comprising:

a) with first and second image sensors of a stereo camera, respectively producing first and second mono image data of a scene including an object viewed by the image sensors;

b) in a processor arrangement, performing a stereo image evaluation of stereo image data represented by a combination of the first and second mono image data, and based thereon detecting the object as an obstacle for the motor vehicle, and producing object data regarding the object detected as an obstacle, wherein the object data includes at least one parameter selected from a position, a distance, a dimension, and a speed of the object; and

c) in the processor arrangement, performing a mono image evaluation of at least one of the first mono image data or the second mono image data, and based thereon redundantly detecting the object in the at least one of the first mono image data and/or the second mono image data classifying the object as to an object type thereof, and outputting corresponding classified object data including the at least one parameter and the object type of the object.

22. The method according to claim 19, wherein the mono image evaluation is a purely mono image evaluation of only either the first mono image data or the second mono image data at a given time.