Seaming polygonal projections from subhemispherical imagery

• US 6,683,608 B2
• Filed: 10/03/2001
• Issued: 01/27/2004
• Est. Priority Date: 11/02/1995
• Status: Expired due to Term

• Current Assignee: Immersive Licensing, Inc.
• Original Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Inventors: Baker, Thomas, Gilbert, Scott, Golin, Stuart J.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/970,418
• Publication Number:
  US 20020015049A1
• Time in Patent Office: 846 Days
• Field of Search: 345/419, 345/420, 345/422, 345/427, 345/428, 345/619, 345/629
• US Class Current: 345/419
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

A method and apparatus for generating views of an environment with proper perspective and parallax. This invention captures panoramic views at many points in the environment, preferably by using fisheye lenses with subhemispherical fields of views to yield overlapping fisheye images that are seemed together, and stores them as planar polygons, which have been extended to include imagery occluded at their viewing position but visible at nearby points. Three-dimensional information on the environmental source of these polygons is determined and stored, enabling the playback system to simulate parallax as well as to produce stereographic images. Given a location and orientation in the environment, the playback system finds a nearby capture point and translates the polygons at that point to the specified location, producing an image. It then rotates that image to the desired orientation which it then displays.

Claims

1. A method for forming a panoramic image from multiple fisheye images comprising the steps of:

capturing a plurality of fisheye images from a keypoint, and

seaming said plurality of fisheye images together to form at least a portion of a spherical view.

2. The method according to claim 1 wherein said plurality of images includes, a first image, a second image, a third image and a fourth image captured from the keypoint.

3. The method according to claim 2 wherein said first direction of said first image is straight upward from said keypoint, and the respective second, third and fourth directions of said second, third and fourth images are 0°

, 120° and

240°

about a horizon but tilted at least about 19.5 degrees below horizontal.

4. The method according to claim 2 wherein said plurality of images also includes a fifth fisheye image taken in a fifth direction and each of said images has a view angle of at least about 127°

.

5. The method according to claim 4 wherein said first direction of said first image is straight upward from said keypoint, said fifth direction of said fifth image is straight downward and respective second, third and fourth directions of said second, third and fourth images are 0°

, 120° and

240°

about a horizontal, respectively.

6. The method according to claim 4 wherein said first direction of said first image is straight upward from said keypoint, said sixth direction of said sixth image is straight downward and respective second, third, fourth and fifth directions of said second, third, fourth, and fifth images are 0°

, 90°

, 180° and

270°

about a horizontal, respectively.

7. The method according to claim 2 wherein said plurality of images includes a fifth fisheye image taken in a fifth direction and a sixth fisheye image taken in a sixth direction, each of said images having a view angle of at least about 109.5°

.

8. The method according to claim 1 wherein each of said images has a view angle greater than 141 degrees.

9. The method of claim 1, wherein the keypoint is a viewer-selected viewpoint.

10. A method for forming a full panorama image from multiple fisheye images comprising the steps of:

capturing a plurality of fisheye images each comprising a k-projection,

creating a database of images together with their k-projections, their k-points, predetermined key orientations, and domains, and

utilizing the information in said database to seam said plurality of fisheye images together to form a full image spherical view, said seaming step including seaming overlapping portions of at least two of said plurality of fisheye images.

12. The system of claim 10, wherein the keypoint is a viewer-selected viewpoint.

11. A system for forming a full panorama image from multiple fisheye images comprising the steps of:

a camera for capturing a plurality of fisheye images from a keypoint, and

a computer program for seaming said plurality of fisheye images together to form a full image spherical view, said seaming step including seaming overlapping portions of at least two of said plurality of fisheye images.

13. A method for forming a panoramic image from multiple fisheye images comprising the steps of:

capturing a plurality of fisheye images from a keypoint, and

seaming said plurality of fisheye images together to form at least a portion of a spherical view, said seaming step including seaming overlapping portions of at least two of said plurality of fisheye images.

14. A method of forming a panoramic image from multiple fisheye images, comprising:

receiving a viewpoint request from a user;

determining a nearest keypoint to the viewpoint, wherein the keypoint is associated with keypoint data stored in a storage medium, the keypoint data including at compressed polygons of imagery from at least one fisheye image, wherein the compressed polygons of imagery are visible at the keypoint;

responsive to the keypoint being a predetermined distance from the viewpoint,

reading the keypoint data from the storage medium;

decompressing the compressed polygons;

seaming the decompressed polygons together into an image that is visible at the keypoint; and

displaying the image to the user.

15. A method of forming a panoramic image from multiple fisheye images, comprising:

receiving a viewpoint request;

determining a nearest keypoint to the viewpoint, wherein the keypoint is associated with keypoint data stored in a storage medium, the keypoint data including compressed polygons of imagery from at least one fisheye image, wherein the compressed polygons of imagery are visible at the keypoint;

responsive to the keypoint being a predetermined distance from the viewpoint, generating polygons of imagery that are visible at the viewpoint;

compressing the polygons; and

storing the compressed polygons as keypoint data.

16. A computer-readable medium having stored thereon instructions which, when executed by a processor in a system for forming a panoramic image from multiple fisheye images, causes the system to perform the operations of:

capturing a plurality of fisheye images from a keypoint; and

seaming the plurality of fisheye images together to form at least a portion of a spherical view.

17. The computer-readable medium of claim 16, wherein the keypoint is a viewer-selected viewpoint.