OBJECT MOTION DETECTION SYSTEM BASED ON COMBINING 3D WARPING TECHNIQUES AND A PROPER OBJECT MOTION DETECTION

• US 20100315505A1
• Filed: 05/17/2010
• Published: 12/16/2010
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Michalke, Thomas, Kastner, Robert
• Granted Patent: US 8,564,657 B2
• Time in Patent Office: Days
• US Class Current: 348/118
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/251 involving models

Abstract

The invention proposes a method for detecting dynamic objects in the scene in a driver assistance system of a vehicle, comprising the steps of:

• • feeding signals from sensors (internal sensors, 3D sensors, cameras) of the vehicle to the driver assistance system,
  • generation of a surrounding model using 3D world positions based on the sensor signals and,
  • combination of 3D Warping-based approaches and optical flow based approaches, whose gained novel key points are:
  • a) Restriction of the search space of an optical flow method based on the computation results of the 3D warping method,
  • b) Additionally, to a) also an optimal parameterization of the optical flow approach in terms of search direction, amplitude, etc.,
  • c) Refinement and verification of the detection results of one of the methods based on the computation results of the other method (if both approaches are running in parallel),
  • storing the detected dynamic objects and their measured motion parameters for their use in, e.g. a collision avoidance or path planning system.

Claims

1. A method for detecting dynamic objects in the visually sensed scene of a driver assistance system of a vehicle or robot with ego-motion, comprising the steps of:

providing visual signals of an environment of a vision sensor attached to a vehicle or robot with ego-motion,

detecting a proper motion of objects on the input field of the vision sensor based on a detected optical flow,

detecting the proper motion of objects based on a 3D representation model of the environment and using a 3D warping on the basis of predicted and sensed 3D data, the predicted 3D data being generated based on measured 3D data and data representing the ego-motion, and

combining the 3D warping-based and the optical flow based object motion detection, and

storing information on the detected dynamic objects and their measured motion parameters.

2. The method according to claim 1, wherein the information on the detected dynamic objects is used for collision avoidance or path planning.

3. The method according to claim 1, wherein the 3D Warping-based and the optical-flow-based object motion detection are combined such that the search space of the respectively other detection method is restricted and the control parameters are determined and supplied to the respectively other detection method.

4. The method according to claim 1, wherein the 3D Warping based object motion detection parameterizes, e.g. as to the direction and amplitude, the optical-flow-based object motion recognition as to the regions, which indicate the lateral motion of objects.

5. The method according to claim 1, wherein only those regions in the 3D Warping based object motion detection which indicate a lateral motion relative to the ego-motion are processed by the optical flow based object motion recognition.

6. The method according to claim 1, wherein the 3D-Warping-based and the optical-flow-based object motion detection are combined such that the detection results of the respectively other detection method is refined and verified.

7. The method according to claim 1, wherein information from a 3D depth sensor, such as e.g. rotating laser scanner is used to generate 3D data.

8. The method according to claim 1, wherein information from a 2D depth sensor such as e.g. Photonic Mixer Device is used to generate 3D data.

9. The method according to claim 1, wherein information from a 1D depth sensor (e.g., laser scanner) is used to generate dense 3D data.

10. The method according to claim 1, wherein a dense depth sensor provides 3D data and information of the ego-motion computed based on a single track model.

11. The method according to claim 1, wherein a dense depth sensor provides 3D data and information of the ego-motion computed based on a single track model.

12. The method according to claim 1, wherein input data for the single track model comes from additional sensors for vehicle velocity and/or yaw-rate.

13. The method according to claim 1, wherein both methods are running in parallel.

14. A driver assistance computing unit, designed to carry out a method according to claim 1.

15. A vehicle being equipped with a driver assistance computing unit according to claim 14.

16. An autonomous robot being equipped with a computing unit carrying out a method according to claim 1.

17. A computer program product, implementing a method according to claim 1 when run on a computing unit.

18. A driver assistance system with a driving path and/or surrounding model generation apparatus, the model generation apparatus comprising:

means for providing visual signals of the environment of a vision sensor with ego-motion,

computing means detecting the proper motion of objects on the input field of the vision sensor based on the detected optical flow,

computing means for detecting the proper motion of objects based on a 3D representation model of the environment and using a 3D warping on the basis of predicted and sensed 3D data, the predicted 3D data being generated based on measured 3D data and data representing the ego-motion, and

computing means for combining the 3D warping-based and the optical flow based object motion detection, and

means for storing information on the detected dynamic objects and their measured motion parameters.

19. The system according to claim 18, wherein the sensors comprise video cameras.

20. The system according to claim 18, comprising a satellite-based navigation system for generating environmental information.

21. The system according to claim 18, wherein the vision sensor is a dense depth sensor providing 3D data and information on ego-motion.

22. The system according to claim 18, wherein the vision sensor is accessible on a CAN bus and/or from an additional sensor.