Three-dimensional road map estimation from video sequences by tracking pedestrians

• US 20090028384A1
• Filed: 04/18/2005
• Published: 01/29/2009
• Est. Priority Date: 04/18/2005
• Status: Active Grant

• Current Assignee: ABB Patent GmbH (ABB Limited), Aurora Operations Incorporated
• Original Assignee: Intel Corporation
• Inventors: Bovyrin, Alexander, Rodyushkin, Konstantin
• Granted Patent: US 7,751,589 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

Estimation of a 3D layout of roads and paths traveled by pedestrians is achieved by observing the pedestrians and estimating road parameters from the pedestrian'"'"'s size and position in a sequence of video frames. The system includes a foreground object detection unit to analyze video frames of a 3D scene and detect objects and object positions in video frames, an object scale prediction unit to estimate 3D transformation parameters for the objects and to predict heights of the objects based at least in part on the parameters, and a road map detection unit to estimate road boundaries of the 3D scene using the object positions to generate the road map.

Claims

1. A method of analyzing video frames capturing a 3D scene over time to automatically generate a road map of the 3D scene comprising:

detecting positions of objects in the video frames;

estimating 3D transformation parameters for the objects;

predicting heights of the objects based at least in part on the 3D transformation parameters; and

estimating road boundaries of the 3D scene using the object positions to generate the road map.

2. The method of claim 1, further comprising estimating a height map for objects moving on a road of the road map.

3. The method of claim 1, further comprising removing outliers from the predicted heights of objects and repeating estimation of the 3D transformation parameters and prediction of the heights of objects.

4. The method of claim 1, wherein detecting positions of objects comprises applying a foreground object detection process to the video frames.

5. The method of claim 1, further comprising removing outlier pixels from the road map.

6. The method of claim 1, wherein estimating road boundaries comprises applying a region growing process to object positions to find pixels of the video frames belonging to a road surface in the 3D scene.

7. An article comprising:

a machine accessible medium containing instructions, which when executed, result in analyzing video frames capturing a 3D scene over time to automatically generate a road map of the 3D scene by

detecting positions of objects in the video frames;

estimating 3D transformation parameters for the objects;

predicting heights of the objects based at least in part on the 3D transformation parameters; and

estimating road boundaries of the 3D scene using the object positions to generate the road map.

8. The article of claim 7, further comprising instructions for estimating a height map for objects moving on a road of the road map.

9. The article of claim 7, further comprising instructions for removing outliers from the predicted heights of objects and repeating estimation of the 3D transformation parameters and prediction of the heights of objects.

10. The article of claim 7, wherein instructions for detecting positions of objects comprises instructions for applying a foreground object detection process to the video frames.

11. The article of claim 7, further comprising instructions for removing outlier pixels from the road map.

12. The article of claim 7, wherein instructions for estimating road boundaries comprises instructions for applying a region growing process to object positions to find pixels of the video frames belonging to a road surface in the 3D scene.

13. A system comprising:

a foreground object detection unit to analyze video frames of a 3D scene and detect objects and object positions in the video frames;

an object scale prediction unit to estimate 3D transformation parameters for the objects and to predict heights of the objects based at least in part on the 3D transformation parameters; and

a road map detection unit to generate the road map by estimating road boundaries of the 3D scene using the object positions.

14. The system of claim 13, wherein the system estimates a height map for objects moving on a road of the road map.

15. The system of claim 13, wherein the object scale prediction unit is configured to remove outliers from the predicted heights of objects and repeat estimation of the 3D transformation parameters and prediction of the heights of objects.

16. The system of claim 13, wherein the road map detection unit is configured to remove outlier pixels from the road map.

17. The system of claim 13, wherein the road map estimation unit estimates road boundaries by applying a region growing process to object positions to find pixels of the video frames belonging to a road surface in the 3D scene.