System and Method for Cost-Effective, High-Fidelity 3D-Modeling of Large-Scale Urban Environments

• US 20080221843A1
• Filed: 08/31/2006
• Published: 09/11/2008
• Est. Priority Date: 09/01/2005
• Status: Active Grant

• Current Assignee: Alexander Harari, Victor Shenkar
• Original Assignee: Alexander Harari, Victor Shenkar
• Inventors: Shenkar, Victor, Harari, Alexander
• Granted Patent: US 8,818,076 B2
• Time in Patent Office: Days
• US Class Current: 703/1
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

A method, a system, and a program for high-fidelity three-dimensional modeling of a large-scale urban environment, performing the following steps:

• • acquiring imagery of the urban environment, containing vertical aerial stereo-pairs, oblique aerial images; street-level imagery; and terrestrial laser scans,
  • acquiring metadata pertaining to performance, spatial location and orientation of imaging sensors providing the imagery;
  • identifying pixels representing ground control-points and tie-points in every instance of the imagery where the ground control-points and tie-points have been captured;
  • co-registering the instances of the imagery using the ground control-points, the tie-points and the metadata, and
  • referencing the co-registered imagery to a common, standard coordinate system.

The referenced co-registration obtained enables:

• • extraction of ground coordinates for each pixel located in overlapping segments of the imagery, representing a 3D-point within the urban environment; and
  • applying data pre-processing and 3D modeling procedures;
  • to create the high-fidelity 3D model of a large-scale urban environment.

Claims

1. A method for three-dimensional (3D) modeling of an urban environment, said method comprising the steps of:

a) acquiring imagery of said urban environment comprising;

1) aerial imagery comprising at least one of;

i) stereo-pair of vertical aerial images; and

ii) oblique aerial images taken in four main directions; and

2) street-level imagery comprising;

i) a plurality of terrestrial photographs; and

ii) a plurality of terrestrial laser scans;

b) acquiring metadata comprising information pertaining to;

performance, spatial location and orientation of imaging sensors providing said aerial imagery and street-level imagery;

c) identifying pixels representing ground control-points and tie-points in every instance of said imagery, in which said ground control points and tie-points have been captured;

d) co-registering said instances of said imagery using said ground control-points, said tie-points and said metadata, and

e) referencing said co-registered imagery to a common, standard coordinate system;

wherein said referenced and co-registered imagery enables extraction of ground coordinates for each pixel located in overlapping segments of said imagery, representing a 3D-point within said urban environment.

2. A three-dimensional modeling method according to claim 1, additionally comprising the steps of:

f) determining ground coordinates for a collection of selected pixels of said imagery;

g) forming a three dimensional box-model of selected modeling units spanned over said ground coordinates of said collection of selected pixels; and

h) incorporating geometric details and textural details into said modeling units forming a high-fidelity 3D-model of said urban environment.

3. A three-dimensional modeling method according to claim 2, wherein said step of incorporating geometric details and textural details into said modeling units forming a high-fidelity 3D-model of said urban environment further comprises:

1) dividing said 3D-model into data-layers comprising;

i) a building-models data-layer comprising at least one 3D building model;

ii) a terrain-skin data-layer comprising a 3D terrain skin model; and

iii) a street-level-culture data-layer comprising at least one 3D street-level-culture model;

2) developing said 3D building models, said 3D terrain skin models, and said 3D street-level-culture models;

3) creating several levels-of-detail for said data layers; and

4) merging corresponding instances of said at least one 3D building model, terrain skin model, and street-level-culture models, forming a high-fidelity 3D-model of said urban environment.

4. A three-dimensional modeling method according to claim 3, further comprising:

i) storing said 3D-model in a database.

5. A three-dimensional modeling method according to claim 4, further comprising:

j) dividing said urban environment into city-block segments;

k) applying said steps (f) through (h) to each city-block segment to form a 3D model of said city-block segment, independently of other city-block segments; and

l) integrating said 3D models of said city-block segments to form an integrated 3D model of said urban environment.

6. A three-dimensional modeling method according to claim 1, wherein said vertical aerial imagery comprises a plurality of stereo-pairs of vertical images providing a continuous coverage of said urban environment and a plurality of oblique aerial images providing a continuous coverage of said urban environment in four main directions.

7. A three-dimensional modeling method according to claim 3, wherein said step of developing said 3D building models, said 3D terrain skin model, and said 3D street-level-culture models comprises at least one of the steps selected from the group of steps comprising:

1) automatically importing selected segments of said imagery and associated metadata that are pertinent to modeling a specific city-block segment;

2) automatically identifying best image quality photographic textures of said selected segments;

3) automatically pasting said best image quality photographic textures onto corresponding 3D-geometry;

4) interactively determining spatial location of repetitive 3D-geometric elements;

5) automatically inserting said repetitive 3D-geometric elements into said 3D-model;

6) interactively determining at least one of;

precise spatial location,

physical size, and

cross-sections of specific details extracted from said 3D street-level imagery;

7) automatically inserting said specific details into said 3D model of said urban environment;

8) interactively creating templates of building details;

9) selectively implanting said templates into at least one of said 3D building models, terrain skin model and street-level-culture models; and

10) automatically generating a 3D-mesh representing said terrain skin surface, using a predetermined set of 3D-points and 3D-lines (“

polylines”

) belonging to said 3D terrain skin model, without affecting existing 3D-details.

8. A computer program product, stored on one or more computer-readable media, comprising instructions operative to cause a programmable processor to:

a) acquire imagery of said urban environment comprising;

1) aerial imagery comprising at least one of;

i) stereo-pair of vertical aerial images; and

ii) oblique aerial images taken in four main directions; and

2) street-level imagery comprising;

i) a plurality of terrestrial photographs; and

ii) a plurality of terrestrial laser scans;

b) acquire metadata comprising information pertaining to;

performance, spatial location and orientation of imaging sensors providing said aerial imagery and street-level imagery;

c) identify pixels representing ground control-points and tie-points in every instance of said imagery, in which said ground control-points and tie-points have been captured;

d) co-register said instances of said imagery using said ground control points, said tie-points and said metadata; and

e) reference said co-registration of said imagery to a common, standard coordinate system, enabling thereby extraction of ground coordinates for each pixel located in overlapping segments of said imagery, representing a 3D-point within said urban environment.

9. A computer program product according to claim 8, comprising additional instructions operative to cause said programmable processor to:

f) determine ground coordinates for a collection of selected pixels of said imagery;

g) form a three dimensional box-model of selected modeling units spanned over said ground coordinates of said collection of selected pixels; and

h) incorporate geometric details and textural details into said modeling units while forming a high-fidelity 3D-model of said urban environment.

10. A computer program product according to claim 9, comprising additional instructions operative to cause said programmable processor to:

1) divide said 3D-model into data layers comprising;

i) a building-models data-layer comprising at least one 3D building model;

ii) a terrain-skin data-layer comprising 3D terrain-skin model; and

iii) a street-level-culture data-layer comprising at least one 3D street-level-culture model; and

2) develop said 3D building models, said 3D terrain skin models, and said 3D street-level-culture models;

3) create several levels-of-detail for said data layers; and

4) merge corresponding instances of at least one said 3D building model, terrain skin model, and street-level-culture models, forming a high-fidelity 3D-model of said urban environment.

11. A computer program product according to claim 10, comprising additional instructions operative to cause said programmable processor to:

i) storing said 3D-model in a database.

12. A computer program product according to claim 11, comprising additional instructions operative to cause said programmable processor to:

j) divide said urban environment into city-block segments;

k) apply said steps f to h to each city-block segment to form a 3D model of said city-block segment, independently of other city-block segments; and

l) integrate said 3D models of said city-block segments to form an integrated 3D model of said urban environment.

13. A computer program product according to claim 10, comprising additional instructions operative to cause said programmable processor to perform at least one function selected from the group of functions consisting of:

1) import selected segments of said imagery and associated metadata that are pertinent to modeling a specific city-block segment;

2) identify best image quality photographic textures of said selected segments;

3) paste said best image quality photographic textures onto corresponding 3D-geometry;

     4) enable a user to interactively determine spatial location of repetitive 3D-geometric elements;

5) insert said repetitive 3D-geometric element into said 3D-model;

6) enable a user to interactively determine at least one of;

precise spatial location,

physical size, and

cross-sections,

of specific details extracted from said 3D street-level imagery;

7) automatically insert said specific details into said 3D model of said urban environment;

8) enable a user to interactively create templates of building details;

9) selectively implant said templates into at least one of said 3D building models, terrain skin model and street-level-culture models; and

10) automatically generate a 3D-mesh representing said terrain skin surfaces, using a predetermined set of 3D-points and 3D-lines (“

polylines”

) belonging to said 3D terrain skin model, without affecting existing 3D-details.

14. A method for managing development of a three-dimensional (3D) model of an urban environment, said method comprising:

a) assigning at least one task of modeling a city-block segment of said urban environment to at least one 3D modeler;

b) importing imaging data associated with said city-block segment to a workstation operated by said 3D modeler;

c) monitoring progress of 3D modeling process of said city-block segment by a production manager; and

d) progressively collecting and storing products of said 3D modeling work process.

15. A method for managing development of a three-dimensional (3D) model of an urban environment according to claim 14, further comprising:

e) assigning quality control reviews of said 3D modeling processes according to progress.

16. A method for managing development of a three-dimensional (3D) model of an urban environment according to claim 14 further additionally comprising:

e) providing statistical reports of said 3D modeling processes.

17. A computer program product, stored on one or more computer-readable media, comprising instructions operative to cause a programmable processor to:

a) assign at least one task of modeling a city-block segment of said urban environment to at least one 3D modeler;

b) import imaging data associated with said city-block segment to a workstation operated by said 3D modeler;

c) monitor progress of 3D modeling process of said city-block segment by a production manager; and

d) progressively collect and store products of said 3D modeling process.

18. A computer program product according to claim 17, comprising additional instructions operative to cause said programmable processor to:

e) assign quality control reviews of said 3D modeling processes according to work progress.

19. A computer program product according to claim 17, comprising additional instructions operative to cause said programmable processor to:

f) provide statistical reports of said 3D modeling processes.

20. A mobile terrestrial device operative to acquire street-level imaging information for three-dimensional (3D) modeling of an urban environment, said mobile terrestrial device comprising:

a) at least one photographic imaging device;

b) at least one laser scanner; and

c) at least one location/orientation measuring device operative to provide ground positioning information of said mobile terrestrial device.

21. A mobile terrestrial device according to claim 20, wherein said photographic imaging device (“

camera”

) comprising at least one of;

1) a horizontal (“

azimuth”

) rotation manipulator;

2) an elevation manipulator; and

3) at least one camera capable of being pointed to different azimuth and elevation angles.

22. A mobile terrestrial device according to claim 20, wherein said location/orientation measuring device comprising at least one of:

1) a GPS receiver;

2) an attitude sensor;

3) at least one fast frame-rate, body-fixed digital camera; and

4) a rotary encoder coupled to a wheel of said mobile terrestrial device.

23. A computer program product, stored on one or more computer-readable media, comprising instructions operative to cause a programmable processor to collect:

a) street-level imagery comprising;

1) a plurality of terrestrial photographs; and

2) a plurality of terrestrial laser scans; and

b) acquiring metadata comprising information pertaining to;

performance, spatial location and orientation of imaging sensors providing said street-level imagery.