Panoramic movie which utilizes a series of captured panoramic images to display movement as observed by a viewer looking in a selected direction

• US 6,654,019 B2
• Filed: 10/22/2001
• Issued: 11/25/2003
• Est. Priority Date: 05/13/1998
• Status: Expired due to Term

• Current Assignee: Immersive Licensing, Inc.
• Original Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Inventors: Kaiman, David J., Park, Michael C., Gilbert, Scott, Ripley, G. David
• Primary Examiner(s): Padmanabhan, Mano
• Assistant Examiner(s): SANTIAGO, ENRIQUE L
• Application Number:
  US10/003,399
• Publication Number:
  US 20020063709A1
• Time in Patent Office: 764 Days
• Field of Search: 345/418, 345/629, 345/474, 348/36, 348/38, 348/37, 348/39
• US Class Current: 345/474
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 9/8047 using transform coding
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Movement through multi-dimensional space is simulated using a series of panoramic images which are projected or displayed in sequence. The user'"'"'s direction of view, that is the selected view window, is maintained as the series of images is projected or displayed. Motion in directions other than forward or reverse is simulated by utilizing “branch” points in the sequence. Each path from a branch point simulates motion in a different direction. Branch points are generally indicated to a viewer by visual indicators called “hot spots”; however, branch points may also be hidden and activated in response to the viewer'"'"'s selected direction of view. If a branch point is indicated by a visual indicator, a user can select motion in a desired direction by clicking on a “hot spot”.

In order to conserve storage space, the image representing each panorama can be stored in a compressed format. Only the portion of the panorama necessary to create a “view window” that is, the portion of the image displayed in response to the user'"'"'s direction of view, is decompressed at view time. An index methodology is used to store the panoramic images. Use of the indexing methodology allows the images to be retrieved in both the forward and backward direction to simulate movement in either direction. Sound is provided in a special format, so that special effects can be provided based on the user'"'"'s point of view and dependent upon the direction of motion selected by the user.

Claims

1. A system for displaying panoramic images which displays movement as observed by a viewer looking in a selected direction, comprising:

a digital image capture device for capturing digital images;

a processor for seaming the digital images into panoramic images and linking the panoramic images into a sequence of panoramic images; and

a viewer for displaying a view window into selected ones of the sequence of panoramic images at a frame rate that simulates motion.

2. The system recited in claim 1 wherein the digital image capture device is a camera that has six lenses, one positioned on each side of a cube.

3. The system recited in claim 1 wherein the digital image captured device is moving in space while said images are being captured.

4. The system in claim 1 wherein said images are displayed at a frame rate above 15 frames per second.

19. The system of claim 1, wherein the digital images are compressed at the digital capture device, and portions of the panoramic images which correspond to the view window are decompressed and displayed in the view window.

20. The system of claim 1, wherein the orientation of the panoramic images are adjusted to provide a desired motion effect.

21. The system of claim 1, wherein at least one hot spot is included in at least one panoramic image.

22. The system of claim 21, wherein the hot spot is automatically triggered by a user'"'"'s location in the panoramic image relative to the hot spot.

23. The system of claim 1, wherein a portion of a panoramic image is selected for a view window based on a current user position and direction of travel in the panoramic images.

24. The system of claim 1, wherein segments of multiple sequences of panoramic images are linked to allow a user to follow a selected path in the multiple sequences of panoramic images.

25. The system of claim 1, wherein control information is associated with at least one panoramic image to control special effects in the sequence of panoramic images.

5. A method of displaying panoramic images, comprising:

capturing a plurality of digital images;

seaming the images into panoramic images and linking the panoramic images into a sequence of panoramic images; and

displaying a window into selected ones of the sequence of panoramic images at a frame rate that simulates motion.

6. The method recited in claim 5 wherein the plurality of digital images are captured by a camera that has six lenses, one positioned on each side of a cube.

7. The method recited in claim 5 wherein the portion of each panorama image that is viewed is selected by an operator who indicates a direction of view.

8. The method recited in claim 5 wherein said sequence of panoramic images has break points which provide at least two alternative sequences of images.

9. The method recited in claim 5 wherein the sequence of panoramic images displayed after a break point is selected in response to input from a user.

10. The method recited in claim 5 wherein sound is associated with at least one of the panoramic images.

11. A three dimensional (3-D) panorama movie for enabling a user interactively to view movement through a three-dimensional space along a path through a series of viewpoints and to view in any viewing direction in the three-dimensional space, the 3-D panorama movie comprising:

a computer storage medium for storage of machine readable image data;

a file of machine readable image data stored on the storage medium, the image data including a plurality of panorama frames forming a sequence of images of the three dimensional space in which each image in the sequence has a spatially different viewpoint; and

a panframe directory stored on the storage medium in association with the file of machine readable image data, containing a set of frame indexes, each frame index identifying a location in the file of one of the panorama frames,

whereby said panorama frames can be displayed in a sequence determined by said frame index to simulate motion.

12. A method of displaying movement of objects in multidimensional space comprising the steps of:

capturing a series of sets of individual images with a multi lens camera, each set of images covering at least a portion of a spherical view, said camera moving relative to said objects as said images are captured,

transferring said images to a computer,

seaming the images in each set of images into a panorama,

said panoramas being indexed in the sequence in which the images which formed the panoramas were captured, and,

displaying a view window into said panoramas in the same sequence as the sequence in which they were captured at a frame rate that simulates motion.

13. A computer-readable medium having stored thereon instructions, which, when executed by a processor in a system for displaying panoramic images, cause the processor to perform the operations of:

capturing a plurality of digital images;

seaming the images into panoramic images and linking the panoramic images into a sequence of panoramic images; and

displaying a view window into selected ones of the sequence of panoramic images at a frame rate that simulates motion.

14. The computer-readable medium of claim 13, wherein the plurality of digital images are captured by a camera that has six lenses, one positioned on each side of a cube.

15. The computer-readable medium of claim 13, wherein the portion of each panorama image that is viewed is selected by an operator who indicates a direction of view.

16. The computer-readable medium of claim 13, wherein said sequence of panoramic images has break points which provide at least two alternative sequences of images.

17. The computer-readable medium of claim 13, wherein the sequence of panoramic images displayed after a break point is selected in response to input from a user.

18. The computer-readable medium of claim 13, wherein sound is associated with at least one of the panoramic images.

26. A system for displaying partial panoramic images, comprising:

a digital image capture device for capturing digital images;

a processor for seaming portions of selected ones of the digital images into partial panoramic images; and

a viewer for displaying a view window into the partial panoramic images at a frame rate that simulates motion.

27. A method of displaying partial panoramic images, comprising:

capturing a plurality of digital images;

seaming portions of selected ones of the digital images into partial panoramic images; and

displaying a view window into the partial panoramic images at a frame rate that simulates motion.

28. A computer-readable medium having stored thereon instructions, which, when executed by a processor in a system for displaying partial panoramic images, cause the processor to perform the operations of:

capturing a plurality of digital images;

seaming portions of selected ones of the digital images into partial panoramic images; and

displaying a view window into the partial panoramic images at a frame rate that simulates motion.