Photogrammetry engine for model construction

• US 7,720,276 B1
• Filed: 06/27/2005
• Issued: 05/18/2010
• Est. Priority Date: 06/26/1998
• Status: Expired due to Term

• Current Assignee: Carl P. Korobkin
• Original Assignee: Carl P. Korobkin
• Inventors: Korobkin, Carl P.
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): LIEW, ALEX KOK SOON
• Application Number:
  US11/168,141
• Time in Patent Office: 1,786 Days
• Field of Search: 348/14.16, 382/154
• US Class Current: 382/154
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes

Abstract

A system and methods for accessing 2D digital images and 3D geometric models over a network (preferably the Internet) and transforming and composing that media along with 2D digital image and 3D geometric model media, acquired by other means, into enhanced 2D image and 3D model representations for virtual reality visualization and simulation is disclosed. Digital images and models from a network and other sources are incorporated and manipulated through an interactive graphical user interface. A photogrammetric media processing engine automatically extracts virtual sensor (camera) and geometric models from imagery. Extracted information is used by a reconstruction processor to automatically and realistically compose images and models. A rendering system provides real-time visualization and simulation of the constructed media. A client-server processing model for deployment of the media processing engine system over a network is disclosed.

Claims

1. A computer implemented method of determining camera parameters in a three-dimensional space from a two-dimensional image of objects in the three-dimensional space, said computer comprising one or more processor, memory, one or more user input device, one or more data output device, and one or more data storage device, wherein data relating to said two-dimensional image of objects in the three dimensional space is transformed into a calculation of said camera'"'"'s parameters, the method comprising the steps of:

identifying a polygon, in perspective, in the image that corresponds to a planar facet in the space; and

calculating camera parameters including a camera position from the polygon;

wherein the step of calculating comprises;

identifying parameterized geometric primitives in the space;

determining the elements of the primitives, wherein an element is a spatial structure projectable onto the image;

mapping features of the image to the elements to form correspondences; and

solving for an objective function that minimizes an aggregate error between features of the image and projections of elements onto the image;

wherein said steps are performed on said computer.

2. The computer implemented method according to claim 1, further comprising:

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the polygon and the dimension.

3. The computer implemented method according to claim 1, wherein the camera parameters further include a focal length.

4. The computer implemented method according to claim 3 further comprising:

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the polygon and the dimension.

10. The computer implemented method according to claim 1, wherein calculating the camera parameters does not include prior knowledge of a focal length of a camera.

11. The computer implemented method according to claim 10, further comprising:

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

5. A computer implemented method of interactively determining camera parameters of a camera in a three-dimensional space from a two-dimensional image of objects in the three-dimensional space, said computer comprising one or more processor, memory, one or more user input device, one or more data output device, and one or more data storage device, wherein data relating to said two-dimensional image of objects in the three dimensional space is transformed into a calculation of said camera'"'"'s parameters, the method comprising the steps of:

interactively modifying a cursor graphic, superimposed on the image such that it appears to lie in perspective on a planar facet in the space; and

calculating camera parameters including a camera position the cursor graphic,

wherein the step of calculating comprises;

identifying parameterized geometric primitives in the space;

determining the elements of the primitives, wherein an element is a spatial structure projectable onto the image;

mapping features of the cursor graphic to the elements to form correspondences; and

solving for an objective function that minimizes an aggregate error between features of the cursor graphic and projections of elements onto the image;

wherein said steps are performed on said computer.

6. The computer implemented method according to claim 5, wherein the cursor graphic is an n-sided polygon.

7. The computer implemented method according to claim 5, wherein the camera parameters further includes a focal length.

8. The computer implemented method according to claim 5, further comprising:

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

9. The computer implemented method according to claim 7, further comprising:

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

19. The computer system according to claim 5, wherein the cursor graphic is an n-sided polygon.

12. A computer system for determining camera parameters in a three-dimensional space from a two-dimensional image of objects in the three-dimensional space, said computer system comprising:

one or more processor for performing instructions for;

identifying a polygon, in perspective, in the image that corresponds to a planar facet in the space; and

calculating camera parameters including a camera position from the polygon;

wherein the instruction for calculating camera parameters comprises;

identifying parameterized geometric primitives in the space;

determining the elements of the primitives, wherein an element is a spatial structure projectable onto the image;

mapping features of the image to the elements to form correspondences; and

solving for an objective function that minimizes an aggregate error between features of the image and projections of elements onto the image;

one or more data storage device for storing said instructions.

13. The computer system according to claim 12, further comprising:

instructions for;

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the polygon and the dimension.

14. The computer system according to claim 12, wherein the camera parameters calculations further include instructions for calculating a focal length.

15. The computer system according to claim 14, further comprising:

instructions for;

identifying at least one dimension of an edge of the planar facet to be executed prior to calculating the camera parameters; and

calculating the camera parameters including the camera position from the polygon and the dimension.

16. The computer system according to claim 12, wherein instructions for calculating the camera parameters do not include prior knowledge of focal length of the camera.

17. The computer system according to claim 16, further comprising:

instructions for;

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

18. A computer system for interactively determining camera parameters in a three-dimensional space from a two-dimensional image of objects in the three-dimensional space, said computer system comprising:

one or more processor for performing instructions for;

interactively modifying a cursor graphic superimposed on the image such that it appears to lie in perspective on a planar facet in the space; and

calculating camera parameters including a camera position from the cursor graphic,

wherein the instruction for calculating comprises;

identifying parameterized geometric primitives in the space;

determining the elements of the primitives, wherein an element is a spatial structure projectable onto the image;

mapping features of the cursor graphic to the elements to form correspondences; and

solving for an objective function that minimizes an aggregate error between features of the cursor graphic and projections of elements onto the image;

one or more data storage device for storing said instructions.

20. The computer system according to claim 18, wherein the camera parameters calculations further includes instructions for calculating a focal length.

21. The computer system according to claim 18, further comprising:

instructions for;

prior to calculating the camera parameters, identifying at least one dimension of an edge of the planar facet; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

22. A computer implemented method of interactively determining camera parameters of a camera in a three-dimensional space from a two-dimensional image of objects in the three-dimensional image of space, said computer comprising one or more processor, memory, one or more user input device, one or more data output device, and one or more data storage devices, wherein data relating to said two-dimensional image of objects in the three dimensional space is transformed into a calculation of said camera'"'"'s parameters, the method comprising the steps of:

interactively modifying a cursor graphic, superimposed on the image such that it appears to lie in perspective relative to at least one planar facet in the space; and

calculating camera parameters including a camera position from the cursor graphic;

wherein the cursor graphic corresponds to a geometric structure that does not exist in the physical space or is fully obscured in the two-dimensional image.

23. The method according to claim 22, wherein the cursor graphic is a rectangle.

24. The method according to claim 22, wherein the cursor graphic is an n-sided polygon.

25. The method according to claim 22, wherein the cursor graphic is a cube.

26. A computer system for interactively determining camera parameters in a three-dimensional space from a two-dimensional image of the three-dimensional space, said computer system comprising:

one or more processors for performing instructions for;

interactively modifying a cursor graphic superimposed on the image, said cursor graphics representing a geometric structure that does not exist in the physical space or is fully obscured in the two-dimensional image,

such that the cursor graphic appears to lie in perspective relative to at least one planar facet in the space; and

calculating camera parameters including a camera position from the cursor graphic, one or more data storage device for storing said instructions.

27. The computer system according to claim 26, wherein the cursor graphic is a rectangle.

28. The computer system according to claim 26, wherein the cursor graphic is an n-sided polygon.

29. The computer system according to claim 26, wherein the cursor graphic is a cube.

30. The computer system according to claim 26, further comprising:

instructions for;

prior to calculating the camera parameters, identifying at least one dimension of an edge of the cursor graphic; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

31. The computer system according to claim 26, wherein the camera parameters calculations further include instructions for calculating a focal length.

32. The computer system according to claim 31 further comprising:

instructions for;

identifying at least one dimension of an edge of the cursor graphic to be executed prior to calculating the camera parameters; and

calculating the camera parameters including the camera position from the cursor graphic and the dimension.

33. A computer system for interactively determining camera parameters in a three-dimensional space from a two-dimensional image of the three-dimensional space, said computer system comprising:

a display device comprising,

a means for receiving at least one image of the space;

a means for displaying the at least one image of the space on the display; and

a means for user input, operable to interactively modify cursor graphic superimposed on the at least one image of the space on the display;

a first memory for storing the at least one image of the space;

one or more processors for performing instructions for;

interactively modifying a cursor graphic superimposed on the image;

said cursor graphic representing a geometric structure that does not exist in the space or is fully obscured in the two-dimensional image

such that the cursor graphic appears to lie in perspective relative to at least one planar facet in the space; and

calculating camera parameters including a camera position from the cursor graphic;

a second memory for storing said instructions;

a third memory for storing said camera parameters;

wherein said first, second and third memory are implementable in one or more data repository.

34. The computer system according to claim 33, configured to function as client-server system

wherein the display device is a client connected over a network to a server,

and the calculating of camera parameters is performed by the client.

35. The computer system according to claim 33, configured to function as client-server system

wherein the display device is a client connected over a network to a server,

and the calculating of camera parameters is performed by the server.