SINGLE-CAMERA DISTANCE ESTIMATION

• US 20130222589A1
• Filed: 02/28/2012
• Published: 08/29/2013
• Est. Priority Date: 02/28/2012
• Status: Active Grant

• Current Assignee: NXP USA, Inc. (NXP Semiconductors NV)
• Original Assignee: CogniVue Corporation (NXP Semiconductors NV)
• Inventors: LALONDE, Jeffrey, MARTEL, Luc, LAGANIERE, Robert
• Granted Patent: US 10,008,002 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space

Abstract

Systems and methods for estimating distances to objects or features are described. The distance estimation tracks features across images captured from a single camera and uses an estimation of the ground motion to determine the distance estimate to the tracked features.

Claims

1. A method for determining a distance to an obstacle from a camera mounted to an automobile, the method comprising:

tracking a plurality of features across two or more successive images;

identifying off-ground features from the tracked plurality of features based on a ground motion estimate and respective motion of the plurality of features tracked across two or more successive images; and

calculating for each off-ground feature of the plurality of features a distance to the camera based on motion of the feature in the images captured by the camera projected to a ground-plane and the ground motion estimate.

2. The method of claim 1, further comprising:

for each image captured by the camera generating the corresponding successive image by projecting the captured image to the ground-plane.

3. The method of claim 1, further comprising:

determining the ground motion estimate based on motion of a plurality of tracked features across the two or more successive images.

4. The method of claim 1, further comprising:

identifying one of the successive images as a snapshot for use in determining the ground motion estimate.

5. The method of claim 4, further comprising identifying new snapshot comprises:

determining if an average displacement of tracked features between a next one of the successive images and a latest identified snapshot is above a threshold; and

identifying the next one of the successive images as the new snapshot when the average displacement is above a threshold.

6. The method of claim 5, wherein the average distance is determined using tracked features with a displacement above a minimum threshold.

7. The method of claim 5, further comprising:

determining the ground motion estimate based on motion of a plurality of tracked features across two or more successive snapshots.

8. The method of claim 7, wherein determining the ground motion estimate comprises:

determining an estimate of ground motion parameters describing motion between a current snapshot and one or more previous snapshots, the ground motion parameters defining a 2-dimensional rigid transformation based on {θ

, tx, ty},

wherein;

θ

is a rotation of the camera;

tx is a translation of the camera in a first direction; and

ty is a translation of the camera in a second direction perpendicular to the first direction.

9. The method of claim 8, wherein determining an estimate of the ground motion parameters comprises:

approximating the ground motion as an approximate affine transformation using a linear least squares fit of the plurality of tracked features used to determine the ground motion estimate; and

refining the approximate affine transformation using an iterative Newton-Gauss method.

10. The method of claim 1, wherein identifying off-ground features comprises identifying a feature as an off-ground feature if the feature'"'"'s motion across successive images is greater than the ground motion estimate.

26. The method of claim 2, further comprising:

determining the ground motion estimate based on motion of a plurality of tracked features across the two or more successive images.

27. The method of claim 2, further comprising:

identifying one of the successive images as a snapshot for use in determining the ground motion estimate.

28. The method of claim 6, further comprising:

determining the ground motion estimate based on motion of a plurality of tracked features across two or more successive snapshots.

29. The method of claim 3, wherein identifying off-ground features comprises identifying a feature as an off-ground feature if the feature'"'"'s motion across successive images is greater than the ground motion estimate.

11. A processor for executing instructions stored in memory, the processor for determining a distance to an obstacle from a camera mounted to an automobile, the instructions configuring the processor to:

track a plurality of features across two or more successive images;

identify off-ground features from the tracked plurality of features based on a ground motion estimate and respective motion of the plurality of features tracked across two or more successive images; and

for each off-ground feature, calculate a distance to the camera based on motion of the feature in the images captured by the camera projected to a ground-plane and the ground motion estimate.

12. The processor of claim 11, wherein the instructions when executed by the processing unit, further configuring the processor to for each image captured by the camera, generate the corresponding successive image by projecting the image to the ground-plane.

13. The processor of claim 11, wherein the instructions when executed by the processor, further configuring the processor to:

determine the ground motion estimate based on motion of a plurality of tracked features across the two or more successive images.

14. The processor of claim 11, wherein the instructions when executed by the processor, further configuring the processor to:

generate one or more snapshots from one or more of the successive images.

15. The processor of claim 14, wherein generation of a new snapshot comprises:

determining if an average displacement of tracked features between a next one of the successive images and a latest snapshot is above a threshold; and

generating the new snapshot of the next one of the successive image when the average displacement is above a threshold.

16. The processor of claim 15, wherein the average distance is determined using tracked features with a displacement above a minimum threshold.

17. The processor of claim 15, wherein the instructions when executed by the processor, further configuring the processor to determine the ground motion estimate based on motion of a plurality of tracked features across two or more successive snapshots.

18. The processor of claim 17, wherein determining the ground motion estimate comprises:

determining an estimate of ground motion parameters describing motion between a current snapshot and one or more previous snapshots, the ground motion parameters defining a 2-dimensional rigid transformation based on {θ

, tx, ty},

wherein;

θ

is a rotation of the camera;

tx is a translation of the camera in a first direction; and

ty is a translation of the camera in a second direction perpendicular to the first direction.

19. The processor of claim 18, wherein determining an estimate of the ground motion parameters comprises:

approximating the ground motion as an approximate affine transformation using a linear least squares fit of the plurality of tracked features used to determine the ground motion estimate; and

refining the approximate affine transformation using an iterative Newton-Gauss method.

20. The processor of claim 11, wherein identifying off-ground features comprises identifying a feature as an off-ground feature if the feature'"'"'s motion across successive images is greater than the ground motion estimate.

22. The processor of claim 12, wherein the instructions when executed by the processor, further configuring the processor to:

determine the ground motion estimate based on motion of a plurality of tracked features across the two or more successive images.

23. The processor of claim 12, wherein the instructions when executed by the processor, further configuring the processor to:

generate one or more snapshots from one or more of the successive images.

24. The processor of claim 16, wherein the instructions when executed by the processor, further configuring the processor to determine the ground motion estimate based on motion of a plurality of tracked features across two or more successive snapshots.

25. The processor of claim 15, wherein identifying off-ground features comprises identifying a feature as an off-ground feature if the feature'"'"'s motion across successive images is greater than the ground motion estimate.

21. A system for determining a distance to an obstacle, the system comprising:

a camera to be mounted to an automobile;

a memory for storing instructions; and

a processor for executing instructions, the processor for determining a distance to an obstacle from the camera when mounted to the automobile, the instructions configuring the system to;

track a plurality of features across two or more successive images;

identify off-ground features from the tracked plurality of features based on a ground motion estimate and respective motion of the plurality of features tracked across two or more successive images; and

for each off-ground feature, calculate a distance to the camera based on motion of the feature in the images captured by the camera projected to a ground-plane and the ground motion estimate.