Photogrammetry engine for model construction

• US 8,542,911 B1
• Filed: 12/24/2012
• Issued: 09/24/2013
• Est. Priority Date: 06/26/1998
• Status: Expired due to Fees

• Current Assignee: Carl Phillip Korobkin
• Original Assignee: Carl Phillip Korobkin
• Inventors: Korobkin, Carl Phillip
• Primary Examiner(s): Liew, Alex
• Application Number:
  US13/726,490
• Time in Patent Office: 274 Days
• Field of Search: 382/106, 382/154, 345419-427, 348 42- 60
• US Class Current: 382/154
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes

Abstract

A system and methods for accessing 2D digital images and 3D geometric models over a network (preferably the Internet) and transforming and composing that media along with 2D digital image and 3D geometric model media, acquired by other means, into enhanced 2D image and 3D model representations for virtual reality visualization and simulation is disclosed. Digital images and models from a network and other sources are incorporated and manipulated through an interactive graphical user interface. A photogrammetric media processing engine automatically extracts virtual sensor (camera) and geometric models from imagery. Extracted information is used by a reconstruction processor to automatically and realistically compose images and models. A rendering system provides real-time visualization and simulation of the constructed media. A client-server processing model for deployment of the media processing engine system over a network is disclosed.

Claims

1. A computer implemented method of generating remote presentation of products in situ for a user, said computer comprising one or more processor, memory, one or more user input device, one or more data output device, and one or more data storage device, the method comprising the steps of:

(a) prompting the user to acquire a digital image of a space in which a product not of the space is to be viewed in context;

(b) accepting a product representation representing a product for which a three-dimensional geometric model exists or can be created;

(c) determining at least one dimension reference, wherein a dimension reference is a measurement of a distance in the space corresponding to a separation of two points on the digital image;

(d) calculating, from the image, the at least one dimension reference and reference information, camera parameters for a camera in the space from which the image was captured, wherein the camera parameters include at least a camera position and a focal length;

(e) accepting an input of a product location, the product location being a location in the space where the product is to be displayed in situ;

(f) transforming the geometric model of the product not of the space based on at least the camera position and the product location to form a transformed geometric model; and

(g) combining the transformed geometric model and the digital image to form a modified image of space where the modified image has been modified to show the identified product in situ in the image with a proper proportion and perspective;

wherein said steps are performed on said computer.

2. The method of claim 1, wherein the step of accepting a product representation comprises the steps of:

prompting the user to identify a product of interest from a set of products;

searching a collection of product representations to locate a member of the collection;

accepting the product representation.

3. The method of claim 1, wherein the step of accepting a product representation is a step of accepting a product representation from a server that serves product representations.

4. The method of claim 1, wherein the step of accepting a product representation is performed using a client application on a user computer that receives product representations and further comprising a step of providing the digital image of a space to the client application.

5. The method of claim 4, wherein a client application is delivered to a user computer from a server.

6. A computer system for generating remote presentation of products in situ for a user, said computer system comprising:

one or more processor for performing instructions for;

(a) prompting the user to acquire a digital image of a space in which a product not of the space is to be viewed in context;

(b) prompting the user to select a product from a plurality of products

(c) prompting the user to indicate a product location, the product location being a location in the space where the product is to be displayed in situ;

(d) accepting the selected product representation representing a product for which a three-dimensional geometric model exists or can be created;

(e) calculating, from the image and reference information, camera parameters for a camera in the space from which the image was captured, wherein the camera parameters include at least a camera position and a focal length;

(f) transforming the geometric model of the product not of the space based on at least the camera position and the product location to form a transformed geometric model; and

(g) combining the transformed geometric model and the digital image to form a modified image of space where the modified image has been modified to show the identified product in situ in the image with a proper proportion and perspective;

and one or more data storage device for storing said instructions.

7. The computer system according to claim 6, wherein the instructions further comprise:

determining, prior to calculating camera parameters, at least one dimension reference, wherein a dimension reference is a measurement of a distance in the space corresponding to a separation of two points on the digital image; and

calculating, from the image, the at least one dimension reference and reference information, camera parameters for a camera in the space from which the image was captured, wherein the camera parameters include at least a camera position and a focal length.

8. The computer system according to claim 6, wherein the system further comprises a server and a user client computer connected over a network, and the instructions further comprise:

retrieving a product representation as a step of retrieving a product representation from the server.

9. The computer system according to claim 6, wherein the system further comprises a server and a user client computer connected over a network, and the instructions further comprise:

retrieving a product representation as a step of retrieving a product representation from the server.

10. The computer system according to claim 6, wherein the system further comprises a server and a user client computer over a network, and the instructions further comprise:

(h) sending a client application from the server to the user client computer;

(i) executing the client application on the user client computer, wherein the client application performs steps a b, and c; and

sends the image of the space to the server; and

(j) executing a server application on the server, wherein the server application performs steps d, e, f, and g; and

sends the modified image of the space to the user client computer for display.

11. The computer system according to claim 6, wherein the system further comprises a server and a user client computer connected over a network, and the instructions further comprise;

(h) sending a client application from the server to the user computer; and

(i) executing the client application on the user client computer, wherein the client application performs steps a thru g;

wherein the product representation is accepted from the server.

12. The computer system according to claim 7, wherein the system further comprises a server and a user client computer connected over a network, and the instructions further comprise:

(h) sending a client application from the server to the user computer; and

(i) executing the client application on the user client computer, wherein the client application performs steps a thru g;

wherein the product representation is accepted from the server.

13. An electronic commerce merchandise visualization and information system for generating remote presentations of products in situ for a user comprising:

space representation storage for storing a space representation, wherein a space representation comprises at least one digital image of a space in which a product is to be viewed in situ, operable to prompt the user to acquire a digital image of a space in which a product not of the space is to be viewed in context;

product representation storage for storing a product representation, wherein a product representation is at least a product image taken of a three-dimensional product operable to prompt the user to accept a product representation representing a product for which a three-dimensional geometric model exists or can be created;

parameter storage for storing at least one dimension reference, wherein a dimension reference is a measurement of a distance in the space corresponding to a separation of two points on the digital image, operable to determine at least one dimension reference;

a photogrammetric modeling unit operable to calculate from the image, the at least one dimension reference and reference information, camera parameters for a camera in the space from which the image was captured, wherein the camera parameters include at least a camera position and a focal length;

product location storage, wherein a product location is a location in the space where the product is to be displayed in situ, and operable to accept an input of a product location;

a visual reconstruction unit operable to transform the geometric model of the product not of the space based on at least the camera position and the product location to form a transformed geometric model; and

a rendering unit operable to combine the transformed geometric model and the digital image to form a modified image of space where the modified image has been modified to show the identified product in situ in the image with a proper proportion and perspective.

14. The electronic commerce merchandise visualization and information system of claim 13 using a computer network having at least a server and a client computer, wherein:

the client computer includes at least;

the space representation storage, the parameter storage, product location storage, the photogrammetric modeling unit, the visual reconstruction unit, and the rendering unit;

the server includes at least;

the product representation storage;

the modified image of a space is displayed on the client computer.

15. The electronic commerce merchandise visualization and information system of claim 13 using a computer network having at least a server and a client computer, wherein:

the server includes at least the space representation storage, the product representation storage, the parameter storage, the photogrammetric modeling unit, product location storage, the visual reconstruction unit, and the rendering unit;

the modified image of a space is displayed on the client computer.

16. The electronic commerce merchandise visualization and information system of claim 13 using a computer network having at least a server and a client computer, wherein:

the client computer includes at least the space representation storage, the parameter storage, and the rendering unit;

the server includes at least the product representation storage, the photogrammetric modeling unit, and the visual reconstruction unit;

the modified image of a space is displayed on the client computer.