Method and system for reconstructing 3D interactive walkthroughs of real-world environments

• US 7,027,049 B2
• Filed: 05/07/2004
• Issued: 04/11/2006
• Est. Priority Date: 04/16/2001
• Status: Expired due to Term

• Current Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Aliaga, Daniel G., Carlbom, Ingrid B.
• Primary Examiner(s): Chauhan, Ulka J.
• Assistant Examiner(s): PAPPAS, PETER
• Application Number:
  US10/840,402
• Publication Number:
  US 20050018058A1
• Time in Patent Office: 704 Days
• Field of Search: None
• US Class Current: 345/427
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching

Abstract

An omnidirectional video camera captures images of the environment while moving along several intersecting paths forming an irregular grid. These paths define the boundaries of a set of image loops within the environment. For arbitrary viewpoints within each image loop, a 4D plenoptic function may be reconstructed from the group of images captured at the loop boundary. For an observer viewpoint, a strip of pixels is extracted from an image in the loop in front of the observer and paired with a strip of pixels extracted from another image on the opposite side of the image loop. A new image is generated for an observer viewpoint by warping pairs of such strips of pixels according to the 4D plenoptic function, blending each pair, and then stitching the resulting strips of pixels together.

Claims

1. A method of gathering image data of an environment, comprising:

recording images of the environment at each of various points of along intersecting paths within the environment with a camera being conveyed at a fixed height to each of the various points along the intersecting paths, the intersecting paths forming an irregular grid in only two dimensions within the environment, at least a portion of the intersecting paths defining boundaries of an image loop; and

determining a three-dimensional position and orientation relative to the environment corresponding to each image.

2. The method of claim 1, wherein the camera is

a hemispherical field-of-view omnidirectional camera with an effective center-of-projection.

3. The method of claim 1, wherein operation of the camera to record images and conveying of the camera are performed by wired or wireless control.