Modeling and editing image panoramas

• US 20040196282A1
• Filed: 02/17/2004
• Published: 10/07/2004
• Est. Priority Date: 02/14/2003
• Status: Abandoned Application

• Current Assignee: EveryScape Incorporated
• Original Assignee: EveryScape Incorporated
• Inventors: Oh, Byong Mok
• Application Number:
  US10/780,500
• Publication Number:
  US 20040196282A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

Three-dimensional models are created from one or more image panoramas. One or more image panoramas representing a visual scene and having one or more objects is received. A directional vector for each image panorama is determined, the directional vector indicating an orientation of the visual scene with respect to a reference coordinate system. The image panoramas are transformed such that the directional vectors are aligned relative to the reference coordinate system. The transformed image panoramas are aligned to each other. A three dimensional model of the visual scene is created using the reference coordinate system, the model comprising depth information describing the one or more objects contained in the scene.

Claims

1. A computerized method for creating a three dimensional model from one or more image panoramas, the method comprising:

receiving one or more image panoramas representing a visual scene and having one or more objects;

determining a directional vector for each image panorama, the directional vector indicating an orientation of the visual scene with respect to a reference coordinate system;

transforming the image panoramas such that the directional vectors are substantially aligned relative to the reference coordinate system;

aligning the transformed image panoramas to each other; and

creating a three dimensional model of the visual scene from the transformed image panoramas using the reference coordinate system and comprising geometry information describing the one or more objects contained in the scene.

2. The method of claim 1 wherein the directional vector is determined based, at least in part, on instructions identifying elements of the image panorama received from a user.

3. The method of claim 2 wherein the instructions from the user identify two or more substantially parallel features in the image.

4. The method of claim 2 wherein the instructions from the user identify two or more sets of substantially parallel features in the image.

5. The method of claim 2 wherein the instructions from the user identifying a horizon line of the image panorama.

6. The method of claim 2 wherein the instructions comprise the identification of two or more areas of the image, each area containing one or more elements and further comprising automatically identifying the two elements contained in the two or more areas.

7. The method of claim 6 further comprising using edge detection to automatically identify the two elements.

8. The method of claim 1 wherein the image panoramas are aligned relative to the reference coordinate system such that the directional vector is at least substantially parallel to one axis of the reference coordinate system.

9. The method of claim 1 wherein the image panoramas are aligned relative to the reference coordinate system such that the directional vector is at least substantially orthogonal to one axis of the reference coordinate system.

10. The method of claim 1 wherein the image panoramas are aligned according to instructions received from a user.

11. A computerized method of interactively editing objects in a panoramic image, the method comprising:

receiving an image panorama representing a visual scene, the image panorama having one or more objects and a point source;

creating a three dimensional model of the visual scene using features of the visual scene and the point source;

receiving an edit to one or more of the objects in the panorama;

transforming the edit relative to a viewpoint defined by the point source; and

projecting the transformed edit onto the objects.

12. The method of claim 11 wherein the three-dimensional model comprises one or more of depth information and geometry information.

13. The method of claim 11, further comprising receiving an edit to color information associated with the objects of the image.

14. The method of claim 11, further comprising receiving an edit to alpha information associated with the objects of the image.

15. The method of claim 11, further comprising receiving an edit to depth information associated with the objects of the image.

16. The method of claim 11, further comprising receiving an edit to geometry information associated with the objects of the image.

17. The method of claim 11 further comprising:

providing a user with an interactive drawing tool that specifies edits for one or more objects of the image; and

receiving the edits made by the user using the interactive drawing tool.

18. The method of claim 17 wherein the interactive drawing tool is one of an extrusion tool, a ground plane tool, a depth chisel tool or a non-uniform rational B-spline tool.

19. The method of claim 17, wherein the interactive drawing tool specifies a selected value for depth for objects of the image.

20. The method of claim 17, wherein the interactive drawing tool incrementally adds to the depth for objects of the image.

21. The method of claim 17, wherein the interactive drawing tool incrementally subtracts from the depth for objects of the image.

22. A method for projecting texture information onto a geometric feature within an image panorama, the method comprising:

receiving instructions from a user identifying a three-dimensional geometric surface within an image panorama, the image panorama containing features having one or more textures;

determining a directional vector from the three-dimensional geometric surface;

creating a geometric model of the image panorama based at least in part on the three-dimensional geometric surface and the directional vector; and

applying the one or more textures to the features in the image panorama based on the geometric model.

23. The method of claim 22 wherein the instructions are received using an interactive drawing tool.

24. The method of claim 22 wherein the three-dimensional geometric surface is one of a floor, a wall, or a ceiling.

25. The method of claim 22 wherein the directional vector is orthogonal to the planar surface.

26. The method of claim 22 wherein the geometric model comprises depth information.

27. The method of claim 22 wherein the texture information comprises color information.

28. The method of claim 22 wherein the texture information comprises luminance information.

29. A computerized method for creating a three-dimensional model of a visual scene from a set of image panoramas, the method comprising:

receiving multiple image panoramas;

arrange each image panorama to a common reference system;

receiving information identifying features common to two or more of the arranged panoramas;

aligning the two or more image panoramas to each other using the identified features; and

creating a three-dimensional model from the aligned image panoramas.

30. The method of claim 29 wherein the instructions are received using an interactive drawing tool.

31. The method of claim 30 wherein the interactive drawing tool is used to identify four or more features common to the two or more image panoramas.

32. A system for creating a three dimensional model from one or more image panoramas, the system comprising:

means for receiving one or more image panoramas representing a visual scene having one or more objects;

means for allowing a user to interact with the system to determine a directional vector for each image panorama;

means for aligning the image panoramas relative to each other; and

means for creating a three dimensional model from the aligned panoramas.

33. The system of claim 32, wherein the input images comprise two-dimensional images.

34. The system of claim 32, wherein the input images comprise three-dimensional images including geometry information.

35. The system of claim 32 wherein the image panoramas are aligned according to instructions received from a user.

36. A system for interactively editing objects in a panoramic image, the system comprising:

a receiver for receiving one or more image panoramas representing a visual scene having one or more objects and a point source;

a modeling module for creating a three dimensional model of the visual scene including depth information describing the objects

one or more interactive editing tools for providing an edit to one or more objects in the panorama;

a transformation module for transforming the edit relative to a viewpoint defined by the point source; and

a rendering module for projecting the transformed edit onto the objects.

37. The system of claim 36 wherein the one or more editing tools comprises a ground plane tool, an extrusion tool, a depth chisel tool, and a non-uniform rational B-spline tool.