ESTIMATING OBJECTS PROPER MOTION USING OPTICAL FLOW, KINEMATICS AND DEPTH INFORMATION

• US 20110243390A1
• Filed: 08/21/2008
• Published: 10/06/2011
• Est. Priority Date: 08/22/2007
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Rebhan, Sven, Willert, Volker, Eggert, Julian, Schmudderich, Jens
• Granted Patent: US 8,422,741 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 7/269 using gradient-based methods

Abstract

The invention relates to a method for detecting the proper motion of an real-world object, comprising the steps of acquiring, by an image capturing device (ICD), a first image (I₁) of the object at a first point in time (t₁) and a second image (I₂) at a second point in time (t₂); obtaining a third (hypothetical) image (I₃), based on an estimated effect of the motion of the image capturing device (ICD) itself (EMF) between the first and the second point in time (t₁, t₂), wherein the effect of the motion of the image capturing device (ICD) itself is estimated based on the forward kinematics of the image capturing device; determining an optical flow (OF) between the second image (I₂) and the third image (I₃); and evaluating the optical flow (OF) by incorporating uncertainties of the optical flow (OF) and the ego-motion-flow (EMF) in order to determine the proper motion of the object.

Claims

1-8. -8. (canceled)

9. A method for detecting the proper motion of real-world objects in an image from a image capturing device, the method comprising the steps of:

acquiring, by an image capturing device (ICD), a first image (I₁) of the object at a first point in time (t₁) and a second image (I₂) at a second point in time (t₂);

acquiring depth information for each pixel of the first image (I₁) and the second image,

computing a third image (I₃), based on the first or the second image as well as the estimated ego-motion (EMF) of the image capturing device (ICD) itself between the first and the second point in time (t₁, t₂), wherein ego-motion is estimated based on the forward kinematics of the image capturing device and depth information for each pixel of the first image (I₁);

computing an ego motion-compensated optical low (OF) by comparing the second image (I₂) and the third image (I₃);

determining the proper motion of objects by evaluating the optical flow (OF) based on the likelihood that the optical flow (OF) and the estimated effect (EMF) are consistent with the same physical velocity.

10. The method according to claim 9, wherein the depth information for the first image (I₁) is estimated based on binocular disparity.

11. The method according to claim 10, wherein an invalid estimated depth is replaced by a depth model gained from valid estimated depths.

12. Method according to claim 9, wherein the step of obtaining the third image comprises warping the first image (I₁).

13. Method according to claim 9, wherein the step of obtaining the third image comprises warping the second image (I₂).

14. Use of a method according to claim 9 for controlling a mobile autonomous vehicle.

15. Use according to claim 14, wherein the mobile autonomous vehicle is an autonomous robot.

16. A method for detecting the proper motion of an real-world objects in an image from a image capturing device, the method comprising the steps of:

means for acquiring, by an image capturing device (ICD), a first image (I1) of the object at a first point in time (t1) and a second image (I2) at a second point in time (t2);

means for acquiring depth information for each pixel of the first image (I1) and the second image,

computing a third image (I3), based on an estimated effect of the motion of the image capturing device (ICD) itself (EMF) between the first and the second point in time (t1, t2), wherein the effect of the motion of the image capturing device (ICD) itself is estimated based on the forward kinematics of the image capturing device and depth information for each pixel of the first image (I1);

means for determining an ego motion-compensated optical flow (OF) by comparing the second image (I2) and the third image (I3); and

means for determining the proper motion of objects by evaluating the optical flow (OF) based on the likelihood that the optical flow (OF) and the estimated effect (EMF) are consistent with the same physical velocity by incorporating uncertainties of the optical flow (OF) and ego-motion-flow (EMF) in order to determine the proper motion of the object.