SYSTEM, COMPUTER PROGRAM AND METHOD FOR 3D OBJECT MEASUREMENT, MODELING AND MAPPING FROM SINGLE IMAGERY

• US 20080089610A1
• Filed: 01/18/2005
• Published: 04/17/2008
• Est. Priority Date: 01/16/2004
• Status: Active Grant

• Current Assignee: Microsoft Corporation
• Original Assignee: Geotango International Corp. (Microsoft Corporation)
• Inventors: Hu, Yong, Tao, C. Vincent, Jiang, Wanshou, Lenson, Peter, Croitoru, Arie
• Granted Patent: US 7,376,284 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 7/50 Depth or shape recovery

Abstract

A method for deriving three-dimensional measurement information and/or creating three-dimensional models and maps, from single images of at least one three-dimensional object is provided. The method includes the steps of: (a) obtaining at least one two-dimensional single image of the object, the image consisting of image data and being associated with an image geometry model (IGM); (b) deriving three-dimensional coordinate information associated with the image, based on the IGM, and associating the three-dimensional coordinate information with the image data; (c) analyzing the image data so as to: (i) measure the projection of the object using the IGM to derive measurement data including the height and/or point-to-point distances pertaining to the object; and/or (ii) measure the shadow of the object to derive measurement data including the height and/or point-to-point distance pertaining to the object; and (d) obtaining three-dimensional measurements based on the projection and/or shadow measurements of the object. In another aspect of the method, the method includes the further step of creating three-dimensional models or maps based on the projection and/or shadow measurements. A series of algorithms are also provided for processing the method of the invention. A computer system and related computer program for deriving three-dimensional measurement information and/or creating three-dimensional models and maps, from single images of at least one three-dimensional object is provided based on the disclosed method is also provided.

Claims

1. A method for deriving three-dimensional measurement information and/or creating three-dimensional models and maps, from single images of at least one three-dimensional object, the method comprising the steps of:

(a) obtaining at least one two-dimensional single image of the object, the image consisting of image data and being associated with an image geometry model (IGM);

(b) deriving three-dimensional coordinate information associated with the image, based on the IGM, and associating the three-dimensional coordinate information with the image data;

(c) analyzing the image data so as to;

(i) measure the projection of the object using the IGM to derive measurement data including the height and/or point-to-point distances pertaining to the object; and

/or

(ii) measure the shadow of the object to derive measurement data including the height and/or point-to-point distance pertaining to the object; and

(d) obtaining three-dimensional measurements based on the projection and/or shadow measurements of the object.

2. The method of claim 1, whereby the image data is analyzed by operation of a measurement utility.

3. The method claimed in claim 1, comprising the further step of creating three-dimensional models or maps based on the projection and/or shadow measurements.

4. The method of claim 1, comprising the further step of creating three-dimensional models or maps by operation of a model generation utility that is operable to utilize data provided by the measurement utility to create three-dimensional models or maps.

5. The method of claim 1, whereby a Rational Function Model (RFM) is used as the IGM.

6. The method of claim 1, whereby the projection of the object is measured using a projection ruler, and the shadow thereof is measured using a shadow ruler, each of the projection ruler and shadow ruler being accessible via a graphic user interface linked to the measurement utility.

7. The method of claim 6, comprising the further step of an operator using the projection ruler, whereby the projection ruler fits the object'"'"'s projection in the image, and whereby:

(a) where the projection is measurable, the projection is fully cast in the image from the base point to the true tip of the object; and

(b) where the projection is not measurable, the method comprises the further step of casting the unmeasurable projection.

8. The method of claim 6, comprising the further step of an operator using the shadow ruler, whereby the shadow ruler fits the object'"'"'s shadow in the image, and whereby:

(a) where the shadow is measurable, the shadow is fully cast in the image from the base point to the endpoint of the shadow; and

(b) where the shadow is not measurable, the method comprises the further step of casting the unmeasurable shadow.

9. The method of claim 8, comprising the following further steps for measuring the shadow of the object where the shadow of the object is measurable:

(a) a user selecting a base point in the image by operation of a graphic user interface associated with the image;

(b) calculating the ground coordinates of the base point using the IGM by operation of a measurement utility;

(c) adjusting the height of the object by selecting and adjusting the height of the object by operation of the graphic user interface;

(d) the measurement utility calculating the offsets of the estimated shadow endpoint relative to the base point; and

(e) the user casting a shadow endpoint of the object to the image by operation of the measurement utility using the IGM.

10. The method of claim 8, comprising the following further steps for measuring of the shadow of the object where the shadow of the object is not measurable:

(a) a user selecting a shadow endpoint in the image by operation of a graphic user interface associated with the image;

(b) calculating the ground coordinates of the shadow endpoint u sing the IGM by operation of a measurement utility;

(c) adjusting the height of the object by selecting and adjusting the height of the object by operation the graphic user interface;

(d) the measurement utility calculating the offsets of the estimated shadow endpoint relative to the base point;

(e) the measurement utility calculating the position of the corresponding projection ruler; and

(f) the user casting a base point of the object to the image when the projection ruler reaches the true tip of the object by operation of the measurement utility using the IGM.

11. The method as claimed in claim 9, whereby the measurement utility calculates the ground coordinates (X, Y) of the base point by processing the following equation:

12. The method as claimed in claims 9, whereby the offsets of the shadow are calculated for flat ground in the vicinity of the object by processing the following equation:

Δ

X=X₁−

X₀=l·

sin(α

)=h·

sin α

/tan θ

Δ

Y=Y₁−

Y₀=l·

cos(α

)=h·

cos α

/tan θ

where h is the height of the object, θ

is the sun'"'"'s altitude at the time the image was taken, and l is the shadow length.

13. The method as claimed in claims 9, whereby the offsets of the shadow are calculated for non-flat ground in the vicinity of the object by processing the following equation:

Δ

X=X₁−

X₀=s·

cos(ψ

)sin(α

)

Δ

Y=Y₁−

Y₀=s·

cos(ψ

)cos(α

)

where

14. The method of claims 9, whereby the length of the shadow of the object, where the object is positioned on a flat surface, and where the coordinates of the shadow endpoint are (X₁, Y₁, Z₁), and Z₁ is equal to Z₀ for the flat surface, is calculated by processing the following equation:

l=h/tan θ

=(Z₂−

Z₀)/tan θ

where h is the height of the object, θ

is the sun'"'"'s altitude at the time the image was taken.

15. The method of claim 1, whereby the three-dimensional measurements include one or more of:

(a) height, (b) distance in 3D, (c) line of sight distance, (d) volume, or (e) bearings in 3D.

16. The method of claim 1, the method further comprising steps for deriving measurement information from a single image of at least one three-dimensional object, where the full projection of the object is visible in the image, the method comprising the further steps of:

(a) a user selecting a base point in the image by operation of a graphic user interface associated with the image;

(b) calculating the ground coordinates of the base point using the IGM by operation of a measurement utility;

(c) adjusting a selected length of the object by selecting and adjusting the selected length of the object by operation of the graphic user interface; and

(d) the measurement utility calculating the selected length of the object.

17. The method of claim 1, the method further comprising steps for deriving measurement information from a single image of at least one three-dimensional object, where the full shadow of the three-dimensional object is visible in the image, the method comprising the further steps of:

(a) a user drawing a shadow ruler on the image by operation of a graphic user interface associated with the image;

(b) the measurement utility calculating the coordinates of the shadow endpoint and the base endpoint using the IGM;

(c) the measurement utility also calculating the coordinates of the endpoint and the base point of the corresponding projection ruler using the IGM;

(d) the user iteratively adjusting height information associated with the object, by operation of the graphic user interface, until the shadow ruler fits substantially the entire shadow of the object and/or the projection ruler reaches the true tip of the object; and

(e) the measurement utility calculating the height of the object using the IGM.

18. The method of claim 1, the method further comprising steps for deriving measurement information from a single image of a three-dimensional object whose base point is not visible or does not exist comprising the further steps of:

(a) a user drawing a shadow ruler on the image by operation of a graphic user interface associated with the image;

(b) the measurement utility calculating the coordinates of the shadow endpoint and the base endpoint using the IGM;

(c) the measurement utility also calculating the coordinates of the endpoint and the base point of the corresponding projection ruler using the IGM;

(d) the user iteratively adjusting height information associated with the object, by operation of the graphic user interface, until the projection ruler reaches the true tip of the object; and

(e) the measurement utility calculating the height of the object using the IGM.

19. The method of claim 1, comprising the further step of compensating for systemic biases occurring in horizontal and vertical directions when capturing the single image.

20. The method claimed in claim 3, comprising the further step of establishing the shape geometry for the object, and using the object shape geometry for creating the three-dimensional models and/or maps.

21. The method claimed in claim 3, comprising the further step of creating 3D models of varying shapes by operation of a model generation utility.

22. The method claimed in claim 3, comprising the further step of creating 3D models of varying roof structures by operation of a model generation utility.

25. The system as claimed in claim 21, wherein the system further comprises a server computer, wherein the computer program is loaded on a server computer, the server computer being accessible by the computer via a network, the server computer providing access to the functions of the measurement utility via the network at the computer.

23. A system for deriving three-dimensional measurement information and/or creating three-dimensional models and maps, from single images of at least one three-dimensional object, the system comprising:

(a) At least one computer; and

(b) A computer program operably linked to the computer so as to enable the computer to;

(i) obtain at least one two-dimensional single image of the object, the image consisting of image data and being associated with an image geometry model (IGM);

(ii) derive three-dimensional coordinate information associated with the image, based on the IGM, and associating the three-dimensional coordinate information with the image data;

(iii) analyze the image data so as to;

(A) measure the projection of the object using the IGM to derive measurement data including the height and/or point-to-point distances pertaining to the object; and

/or

(B) measure the shadow of the object to derive measurement data including the height and/or point-to-point distance pertaining to the object; and

(iv) obtain three-dimensional measurements based on the projection and/or shadow measurements of the object.

24. The system as claimed in claim 23, wherein the computer program is further operably linked to the computer so as to define a model generation utility that is operable to utilize data provided by the measurement utility to create three-dimensional models or maps.

26. A computer usable medium, the computer useable medium comprising instructions for defining a measurement utility on a computer, the measurement utility being operable to:

(i) obtain at least one two-dimensional single image of the object, the image consisting of image data and being associated with an image geometry model (IGM);

(ii) derive three-dimensional coordinate information associated with the image, based on the IGM, and associating the three-dimensional coordinate information with the image data;

(iii) analyze the image data so as to;

(A) measure the projection of the object using the IGM to derive measurement data including the height and/or point-to-point distances pertaining to the object; and

/or

(B) measure the shadow of the object to derive measurement data including the height and/or point-to-point distance pertaining to the object; and

(iv) obtain three-dimensional measurements based on the projection and/or shadow measurements of the object.