Method and apparatus for automatically generating a site model

• US 20030014224A1
• Filed: 07/08/2002
• Published: 01/16/2003
• Est. Priority Date: 07/06/2001
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Kumar, Rakesh, Guo, Yanlin, Sawhney, Harpreet Singh, Hsu, Stephen Charles
• Granted Patent: US 7,509,241 B2
• Time in Patent Office: Days
• US Class Current: 703/1
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A method and apparatus for automatically combining aerial images and oblique images to form a three-dimensional (3D) site model. The apparatus or method is supplied with aerial and oblique imagery. The imagery is processed to identify building boundaries and outlines as well as to produce a depth map. The building boundaries and the depth map may be combined to form a 3D plan view model or used separately as a 2D plan view model. The imagery and plan view model is further processed to determine roof models for the buildings in the scene. The result is a 3D site model having buildings represented rectangular boxes with accurately defined roof shapes.

Claims

1. A method for generating a three-dimensional site model of a site that contains objects comprising:

(a) providing at least one of aerial imagery and oblique imagery of the site;

(b) processing at least one of the aerial imagery or the oblique imagery to generate a plan view model comprising object outlines; and

(c) producing a three-dimensional site model comprising blocks that represent objects within the site.

2. The method of claim 1 further comprising:

(d) processing the oblique imagery to generate elevation information regarding the objects within the site.

3. The method of claim 1 wherein step (b) comprises:

(b1) generating an elevation map;

(b2) identifying object regions; and

(b3) processing the object regions by classifying color, features and texture within said object regions.

4. The method of claim 1 wherein step (b) comprises:

(b1) producing an elevation map;

(b2) computing local orientation;

(b3) aggregating surface patches for key images;

(b4) aggregating elevation information amongst key images to produce a three-dimensional structure; and

(b5) processing the three-dimensional structure into a plan view model.

5. The method of claim 1 wherein the objects in the site are buildings and the method further comprises:

(d) processing the oblique imagery to produce a roof model for each building in the site.

6. The method of claim 5 wherein step (d) further comprises:

(d1) selecting a building within the site;

(d2) detecting edges of the building within the oblique imagery;

(d3) identifying a set of roof model hypotheses that correspond to the edges; and

(d4) selecting and fitting each roof model hypothesis to the oblique imagery to identify a roof model hypothesis as the roof model.

7. The method of claim 6 wherein the step (d1) selects a building based upon visibility within the oblique imagery.

8. The method of claim 5 wherein step (d) further comprises:

(d1) selecting a building within the site;

(d2) extracting an range map from the oblique imagery;

(d3) mapping pixel locations within the range map to a common orthographic coordinate system;

(d4) extracting multiple planes from the range map using regression to fit the planes to form a roof model.

9. The method of claim 1 further comprising receiving elevation information from an elevation information source.

10. The method of claim 1 further comprising:

(d) deriving elevation information from the oblique imagery;

(e) sweeping a horizontal plane through a point cloud in the elevation information;

(f) processing a point density to identify object locations within the site.

11. The method of claim 10 wherein step (f) further comprises:

(f1) fitting rectangular boxes to the point density edges, where the rectangular boxes represent objects within the site.

12. The method of claim 10 wherein step (f) further comprises:

(f1) analyzing the point density to identify object shapes and sizes.

13. The method of claim 12 wherein step (f1) further comprises:

(f11) sweeping at least one addition plane through the point cloud in the elevation information to produce point densities representing planes of objects within the site.

14. The method of claim 1 wherein the plan view model is produced using graphical analysis.

16. The apparatus of claim 15 further comprising:

(d) means for processing the oblique imagery to generate depth information regarding the objects within the site.

17. The apparatus of claim 15 wherein means (b) comprises:

(b1) means for generating a range map;

(b2) means for identifying object regions; and

(b3) means for processing the object regions by classifying color, features and texture within said object regions.

18. The apparatus of claim 15 wherein means (b) comprises:

(b1) means for producing an elevation map;

(b2) means for computing local orientation;

(b3) means for aggregating surface patches for key images;

(b4) means for aggregating elevation information amongst key images to produce a three-dimensional structure; and

(b5) means for processing the three-dimensional structure into a plan view model.

19. The apparatus of claim 15 wherein the objects in the site are buildings and the apparatus further comprises:

(d) means for processing the oblique imagery to produce a roof model for each building in the site.

20. The apparatus of claim 19 wherein means (d) further comprises:

(d1) means for selecting a building within the site;

(d2) means for detecting edges of the building within the oblique imagery;

(d3) means for identifying a set of roof model hypotheses that correspond to the edges;

(d4) means for selecting and fitting each roof model hypothesis to the oblique imagery to identify a roof model hypothesis as the roof model.

21. The apparatus of claim 20 wherein the means (d1) selects a building based upon visibility within the oblique imagery.

22. The apparatus of claim 19 wherein means (d) further comprises:

(d1) means for selecting a building within the site;

(d2) means for extracting a range map from the oblique imagery;

(d3) means for mapping pixel locations within the range map to a common orthographic coordinate system;

(d4) means for extracting multiple planes from the range map using regression to fit the planes to form a roof model.

23. The apparatus of claim 15 further comprising an elevation information source.

24. The apparatus of claim 15 further comprising:

(d) means for deriving range information from the oblique imagery;

(e) means for sweeping a horizontal plane through a point cloud in the range information;

(f) means for processing a point density to identify object locations within the site.

25. The apparatus of claim 24 wherein means (f) further comprises:

(f1) means for fitting rectangular boxes to the point density edges, where the rectangular boxes represent objects within the site.

26. The apparatus of claim 24 wherein means (f) further comprises:

(f1) means for analyzing the point density to identify object shapes and sizes.

27. The apparatus of claim 26 wherein means (f1) further comprises:

(f11) means for sweeping at least one addition plane through the point cloud in the range information to produce point densities representing planes of objects within the site.

28. The apparatus of claim 27 further comprising a means for producing a plan view model using graphical analysis.

30. The method of claim 29 further comprising:

sweeping at least one additional plane through the point cloud in the elevation information to produce point densities representing planes of objects within the site.

15. Apparatus for generating a three-dimensional site model of a site that contains objects comprising:

(a) means for providing at least one of aerial imagery and oblique imagery of the site;

(b) means for processing at least one of the aerial imagery or the oblique imagery to generate a plan view model comprising object outlines; and

(c) means for producing a three-dimensional site model comprising blocks that represent objects within the site.

29. A method for generating a three-dimensional site model of a site that contains objects comprising:

providing oblique imagery of the site;

deriving elevation information from the oblique imagery;

sweeping a horizontal plane through a point cloud in the elevation information;

processing a point density to identify object locations within the site; and

analyzing the point density to identify object shapes and sizes.