Method and apparatus for image assisted modeling of three-dimensional scenes

• US 6,333,749 B1
• Filed: 04/17/1998
• Issued: 12/25/2001
• Est. Priority Date: 04/17/1998
• Status: Expired due to Fees

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Seidl, Robert, Reinhardt, Tilman
• Primary Examiner(s): Luu, Matthew
• Assistant Examiner(s): HARRISON, CHANTE E
• Application Number:
  US09/062,512
• Time in Patent Office: 1,348 Days
• Field of Search: 345/438, 345/420, 345/421, 345/423, 345/444, 345/442, 345/443, 345/418, 345/440, 345/437, 345/326, 345/419, 345/348, 345/964, 345/502, 345/506, 345/524, 345/513, 382/154
• US Class Current: 345/629
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A computer-assisted technique for constructing a three-dimensional model on top of one or more images (e.g., photographs) such that the model'"'"'s parameters automatically match those of the real world object depicted in the photograph(s). Camera parameters such as focal length, position, and orientation in space may be determined from the images such that the projection of a three-dimensional model through the calculated camera parameters matches the projection of the real world object through the camera onto the image surface. Modeling is accomplished using primitives, such as boxes or pyramids, which may be intuitively manipulated to construct the three-dimensional model on a video display or other display screen of a computer system with a two-dimensional input controller (e.g., a mouse, joystick, etc.) such that the, displayed three-dimensional object manipulation emulates physical three-dimensional object manipulation. Camera and primitive parameters are incrementally updated to provide visual feedback of the effect of additional constraints on the three-dimensional model, making apparent which user action may have been responsible for any failure to provide a modeling solution and, thus, allowing for rapid reversal and correction thereof. Surface properties (i.e., textures) may be extracted from the images for use in the three-dimensional model.

Claims

1. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives and a virtual camera for each of the digital representations of the image, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the simple primitives are chosen from a group comprising box primitives, pyramid primitives, cylinder primitives, linear extrusion primitives, and rotation symmetric primitives.

2. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two dimensional image, wherein the parametric scene comprises one or more geometric primitives and a virtual camera for each of the digital representations of the image, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the complex primitives comprise primitives which are constructed from more than one of the simple primitives associated by parameters, wherein the complex primitives are chosen from a group comprising application domain specific primitives.

3. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives and a virtual camera for each of the digital representations of the image, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the imported primitives comprise polyhedral models specified by one or more three-dimensional file formats.

4. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein creating a parametric scene comprises creating a series of geometric primitives and a series of constraints which associate the geometric primitives to the digital representations of the image, wherein the constraints are chosen from a group comprising pin constraints, bead constraints, glue constraints, measurement constraints, on-top constraints, on-face constraints, align orientation constraints, linear alignment constraints, equidistance constraints and shared parameter constraints.

5. The computer-assisted method of claim 4 wherein the glue constraints are chosen from a group comprising point-to-point glue constraints, point-to-edge glue constraints, and edge-to-edge glue constraints.

6. The computer-assisted method of claim 4 wherein the measurement constraints are chosen from a group comprising ruler constraints, compass constraints and datum constraints.

7. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein creating the scene directly on top of one or more displayed digital representations of the image comprises periodically displaying a rendering of the scene through a virtual camera associated with the image superimposed on the image.

8. The computer-assisted method of claim 7 wherein the rendering of the scene comprises a wireframe rendering.

9. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein estimating parameters of the scene is performed interactively and incrementally using an incremental solution mechanism, interleaved with scene redrawing, and with collecting and incorporating changes to constraints associated with primitives that comprise the scene, wherein the incremental solution mechanism is configured to continuously integrated minimal rates of change to the parameters of the scene as the scene is aligned with the displayed digital representations of the image and defined by the constraints.

10. A computer model comprising a parametric scene having estimated parameters, the scene having been created directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the simple primitives are chosen from a group comprising box primitives, pyramid primitives, cylinder primitives, linear extrusion primitives, and rotation symmetric primitives.

11. A computer model comprising a parametric scene having estimated parameters, the scene having been created directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the complex primitives are each constructed from more than one of the simple primitives, wherein the complex primitives are chosen from a group comprising a table primitive.

12. A computer model comprising a parametric scene having estimated parameters, the scene having been created directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives, wherein the geometric primitives comprise one or more simple, compound or imported primitives, wherein the imported primitives comprise polyhedral models specified by one or more three-dimensional file formats.

13. A computer model comprising a parametric scene having estimated parameters, the scene having been created directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises one or more geometric primitives;

a plurality of constraints which associate the geometric primitives to the digital representations, wherein the constraints are chosen from a group comprising pin constraints, bead constraints, glue constraints, measurement constraints, on-top constraints, on-face constraints, equidistance constraints and shared parameter constraints.

14. The computer model of claim 13 wherein the glue constraints are chosen from a group comprising point-to-point glue constraints, point-to-edge glue constraints, and edge-to-edge glue constraints.

15. The computer model of claim 13 wherein the measurement constraints are chosen from a group comprising ruler constraints, compass constraints and datum constraints.

16. A computer model comprising a parametric scene having estimated parameters, the scene having been created directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene comprises a periodically displayed rendering of the scene as seen through one or more virtual cameras associated with the digital representations of the image superimposed on the image.

17. The computer model of claim 16 wherein the rendering of the scene comprises a wireframe rendering.

18. A computer-assisted method, comprising interactively and incrementally creating a parametric scene and estimating parameters of the scene directly on top of one or more displayed digital representations of a two-dimensional image, wherein the parametric scene is represented as a polyhedral model with rectangular image textures in a general file format.