Panoramic movies which simulate movement through multidimensional space

• US 6,337,683 B1
• Filed: 05/12/1999
• Issued: 01/08/2002
• Est. Priority Date: 05/13/1998
• Status: Expired due to Term

• Current Assignee: Immersive Licensing, Inc.
• Original Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Inventors: Park, Michael C., Gilbert, Scott, Ripley, G. David, Kaiman, David J.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Padmanabhan, Mano
• Application Number:
  US09/310,715
• Time in Patent Office: 972 Days
• Field of Search: 345/418, 345/435, 345/474, 345/629, 348/36
• US Class Current: 345/418
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 9/8047 using transform coding
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Movement through multi-dimensional space is simulated using a series of panoramic images which are projected or displayed in sequence. The user'"'"'s direction of view, that is the selected view window, is maintained as the series of images is projected or displayed. Motion in directions other than forward or reverse is simulated by utilizing “branch” points in the sequence. Each path from a branch point simulates motion in a different direction. Branch points are generally indicated to a viewer by visual indicators called “hot spots”; however, branch points may also be hidden and activated in response to the viewer'"'"'s selected direction of view. If a branch point is indicated by a visual indicator, a user can select motion in a desired direction by clicking on a “hot spot”. In order to conserve storage space, the image representing each panorama can be stored in a compressed format. Only the portion of the panorama necessary to create a “view window” that is, the portion of the image displayed in response to the user'"'"'s direction of view, is decompressed at view time. An index methodology is used to store the panoramic images. Use of the indexing methodology allows the images to be retrieved in both the forward and backward direction to simulate movement in either direction. Sound is provided in a special format, so that special effects can be provided based on the user'"'"'s point of view and dependent upon the direction of motion selected by the user.

Claims

1. A system for simulating movement through multidimensional space comprising in combination,

a multi-lens camera for simultaneously capturing a plurality of digital images that cover the entire spherical view field,

compression units for individually compressing said images into compressed images, means for transferring said images to a computer,

a program operated by said computer which seams said images into panoramas and which links said panoramas into a fixed sequence of panoramas,

a viewer which selectively automatically displays at a frame rate that creates a simulation of motion a portion of each of said linked panoramas in said fixed sequence whereby movement is simulated.

2. The system recited in claim 1 wherein said camera has six lenses, one positioned on each side of a cube.

3. The system recited in claim 1 wherein said computer compresses said images after seaming the images.

4. The system recited in claim 3 wherein said compression is single frame compression with no inter-frame compression.

5. The system recited in claim 1 wherein said program adds at least one sound track to each panorama.

6. The system recited in claim 1 wherein said sequence of images includes break points where said sequence can continue in at least one of two directions.

7. The system recited in claim 1 wherein the portion of said linked images displayed is determined by the direction of view selected by a uses and wherein said direction of view remains constant at break points until changed by a user.

8. A method of simulating movement through multidimensional space comprising the steps of:

capturing a series of sets of individual images, each set of images covering at least a portion of a spherical view,

individually compressing said images,

transferring said images to a computer,

decompressing said images,

seaming the images in each set of images into a panorama,

linking said panoramas into a fixed sequence of panoramas,

compressing said panoramas without using any inter-frame compression,

de-compressing at least a portion of each panorama which corresponds to a view window, and

automatically displaying at a frame rate of at least 15 frames per second, said view windows in said fixed sequence of panoramas,

whereby movement is simulated.

9. The method recited in claim 8 wherein each set of images comprises six images taken from the six sides of a cube.

10. The method recited in claim 8 where the portion of each panorama de-compressed for viewing is selected by an operator who indicates a direction of view.

11. The method recited in claim 8 wherein said sequence of images has break points which provide at least two alternative sequences of images.

12. The method recited in claim 8 wherein said the sequence of images displayed after a break point is selected in response to input from a user.

13. The method recited in claim 8 wherein sound is associated with each of said panoramas.

14. A three dimensional (3-D) panorama movie for enabling a user interactively to view movement through a three-dimensional space along a path through a series of viewpoints and to view in any viewing direction in the three-dimensional space, the 3-D panorama movie comprising:

a computer storage medium for storage of machine readable image data;

a file of machine readable image data stored on the storage medium, the image data including a plurality of panorama frames forming a fixed sequence of panoramic images of the three dimensional space in which each panoramic image in the sequence has a spatially different viewpoint;

a panframe directory stored on the storage medium in association with the file of machine readable image data, containing a set of frame indexes, each frame index identifying a location in the file of one of the panorama frames, and

automatically displaying in sequence at a frame rate of at least 15 frames per second, a portion of each panoramic image is said sequence whereby motion is simulated.

15. A panorama movie according to claim 14 which there exists a direction of ravel in three-dimensional space from each viewpoint, the direction of travel being stored in the panorama frame associated with each viewpoint as a predetermined point within the panorama frame to define a frame of reference for a viewing direction during playback.

16. A panorama movie according to claim 15 in which the predetermined point is a centerpoint of the panorama frame.

17. A panorama movie according to claim 14 in which the panorama frames are compressed using intraframe compression, defining a key frame, so that each frame can be decompressed during playback independently of each other frame.

18. A panorama movie according to claim 14 in which the file includes a hot spot associated with at least one of the frames, the hotspot being operable during playback to superimpose a function on the playback of that frame.

19. A panorama movie according to claim 18 in which the hotspot includes an image having a predetermined geometric shape to be superimposed over a feature in one or more of the panorama frames.

20. A panorama movie according to claim 18 in which the hotspot image is stored with an orientation in each panorama frame such that the superimposed image appears in each frame at playback with a position and shape that conforms to a perspective corrected shape and position of the feature over which it is superimposed.

21. A panorama movie according to claim 14 which includes two or more of said files, each forming a segment of the movie and having its own respective panframe directory which includes terminal file indices defining a firstfile index and a lastfile index for the respective terminal frames of each file, the movie further including a control file containing linking data for linking terminal indices to link segments together.

22. A panorama movie according to claim 21 in which the control file includes orientation information defining a new direction of travel to proceed from a terminal frame of first segment along a second segment linked to the first segment.

23. A panorama movie according to claim 22 in which a third segments is also linked to the terminal frame of first segment, the control file orientation information including an alternative new direction of travel selectable by the user during playback upon reaching the terminal frame of first segment, to branch the movie.

24. A panorama movie according to claim 22 in which the new direction of travel is selectable by the user selecting a viewing direction that approximately coincides with the new direction of travel.

25. A panorama movie method for simulating movement through multidimensional space comprising the steps of:

capturing a plurality of images consisting of imagery taken from a plurality of spatial positions in multidimensional space;

seaming said images into key-frame panoramas,

storing said key-frame panoramas in a file in a storage medium;

indexing said key-frame panoramas within a key-frame directory according to a position of each key-frame within the file and storing said key-frame directory on said storage medium;

displaying a portion of a first key-frame panorama according to a user position and viewing direction within the multidimensional space;

accessing said key-frame directory to determine a next key-frame image to be displayed according to a user travel speed and travel direction; and

automatically displaying at a frame rate that is fast enough to simulate motion a portion of a second key-frame panorama subsequent to the display of said first key-frame panorama according to a user position and viewing direction whereby motion is simulated.

26. The method of claim 25 wherein the step of capturing the plurality of key-frame images includes taking photographs.

27. The method of claim 25 wherein the step of capturing the plurality of key-frame images includes rendering the images within a computer system.

28. The method of claim 25 wherein a forward or reverse travel direction is determined by the order in which the keyframes are accessed.

29. The method of claim 25 wherein a travel speed is determined by accessing each nth key frame where n is an integer.

30. The method of claim 25 wherein a first segment of key-frame images is linked to a second segment of key-frame images by a control file stored in the storage medium, the control file including stored travel direction for the second segment which is selected by the user selecting a viewing direction at a terminal frame of the fist segment that approximately coincides with the stored travel direction for the second segment.

31. The method of claim 25 wherein the user actuates an input device to move from a first position through a series of subsequent positions, the method including display successive views of imagery from each of the successive positions.

32. The method of claim 25 including changing the viewing direction using the input device.

33. The method of claim 25 including superimposing objects into the display of multiple successive frames at a location that coincides with apparent movement in the 3-D space.

34. The method of claim 25 including compressing each panorama image using intraframe compression to form the key-frame images, and during playback selecting a portion of the key frame that includes the desired viewed selected by the user for local decompression.

35. The method of claim 25 including transforming the panorama images to a rectangular projection prior to compression.