Method and apparatus for image assisted modeling of three-dimensional scenes

• US 7,602,404 B1
• Filed: 04/27/2000
• Issued: 10/13/2009
• Est. Priority Date: 04/17/1998
• Status: Expired due to Term

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Reinhardt, Tilman, Seidl, Robert
• Primary Examiner(s): Amini; Javid A
• Application Number:
  US09/562,211
• Time in Patent Office: 3,456 Days
• Field of Search: 345/419, 345/420, 345/427, 345/619, 345/672, 345/677, 345/679, 345/629, 359/470
• US Class Current: 345/629
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A computer-assisted technique for constructing a three-dimensional model on top of one or more images (e.g., photographs) such that the model'"'"'s parameters automatically match those of the real world object depicted in the photograph(s). Camera parameters such as focal length, position, and orientation in space may be determined from the images such that the projection of a three-dimensional model through the calculated camera parameters matches the projection of the real world object through the camera onto the image surface. Modeling is accomplished using primitives, such as boxes or pyramids, which may be intuitively manipulated to construct the three-dimensional model on a video display or other display screen of a computer system with a two-dimensional input controller (e.g., a mouse, joystick, etc.) such that the displayed three-dimensional object manipulation emulates physical three-dimensional object manipulation. Camera and primitive parameters are incrementally updated to provide visual feedback of the effect of additional constraints on the three-dimensional model, making apparent which user action may have been responsible for any failure to provide a modeling solution and, thus, allowing for rapid reversal and correction thereof. Surface properties (i.e., textures) may be extracted from the images for use in the three-dimensional model.

Claims

1. A method, comprising:

using a computer to implement;

determining, from a single image obtained from a single real-world camera and as a result of manipulating one or more geometric primitives superimposed over the single image, one or more camera parameters of the single real-world camera from which the image was obtained such that a projection of a computer-generated three-dimensional model, which includes the one or more geometric primitives, through the determined camera parameters approximates the projection of a real-world object depicted in the image as seen through the real-world camera,

wherein the one or more camera parameters include focal length, wherein the one or more camera parameters are refined based upon manipulating the one or more geometric primitives, wherein the one or more geometric primitives are three-dimensional objects, and wherein the single image is obtained from a scanner, or from memory.

2. The method of claim 1 wherein the camera parameters comprise focal length, position or orientation in space.

3. The method of claim 1, wherein the one or more camera parameters are further determined using one or more constraints to associate the one or more geometric primitives to the image.

4. The method of claim 3, wherein determining the one or more camera parameters of the real-world camera from the image as a result of manipulating the primitives, comprises:

setting the one or more camera parameters to respective default values; and

incrementally updating the one or more camera parameters to new values for each additional geometric primitive used to generate the three-dimensional model that approximates the real-world object depicted in the image.

5. The method of claim 4, wherein the one or more camera parameters are further incrementally updated to the new values for each additional constraint used.

6. The method of claim 4, wherein the one or more camera parameters are made available after each update.

7. The method of claim 1, wherein each geometric primitive corresponds to a wireframe rendering, and wherein the wireframe rendering is used to generate the three-dimensional model.

8. The method of claim 1, wherein the image is obtained from a digital camera, or a video capture device.

9. A computer-readable storage medium storing a computer-based model, comprising one or more virtual cameras each having parameters determined as a result of manipulating one or more geometric primitives superimposed over a digital representation of a single respective image derived from a corresponding single real-world camera, wherein a projection of the computer-based model through the parameters of the one or more virtual cameras approximates the projection of one or more real-world objects depicted in the digital representation of the single image as seen through the single real-world camera, wherein the one or more camera parameters include focal length, wherein the one or more camera parameters are refined based upon manipulating the one or more geometric primitives, wherein the one or more geometric primitives are three-dimensional objects, and wherein the single image is obtained from a scanner, or from memory.

10. The computer-readable storage medium of claim 9, wherein the parameters of the virtual cameras are further determined using one or more constraints associated with the primitives.

11. The computer-readable storage medium of claim 9, wherein the one or more geometric primitives include simple primitives and compound primitives.

12. The computer-readable storage medium of claim 11, wherein the simple primitives include box primitives, pyramid primitives, cylinder primitives, linear extrusion primitives, and rotation symmetric primitives.

13. The computer-readable storage medium of claim 11, wherein the each of the compound primitives include two or more simple primitives.

14. The computer-readable storage medium of claim 9, wherein each of the primitives is represented as a wireframe rendering.

15. The computer-readable storage medium of claim 9, wherein the camera parameters are determined incrementally.

16. The computer-readable storage medium of claim 9, wherein the one or more images are obtained from a digital camera, or a video capture device.

17. A computer-readable storage medium, comprising program instructions computer-executable to:

determine from a single image obtained from a single real-world camera and in response to manipulation of one or more geometric primitives superimposed over the single image, one or more camera parameters of the single real-world camera from which the image was obtained such that a projection of a computer-generated three-dimensional model through the determined camera parameters approximates a projection of a real-world object depicted in the image as seen through the real-world camera,

wherein the one or more camera parameters include focal length, wherein the one or more camera parameters are refined based upon manipulating the one or more geometric primitives, wherein the one or more geometric primitives are three-dimensional objects, and wherein the single image is obtained from a scanner, or from memory.

18. The storage medium of claim 17, wherein the program instructions are computer-executable to determine the one or more camera parameters using one or more constraints to associate the one or more geometric primitives to the image.

19. The storage medium of claim 17, wherein the one or more camera parameters are incrementally determined with each additional geometric primitive or constraint used.

20. The storage medium of claim 17, wherein the image is obtained from a digital camera, or from a video capture device.