Immersive imaging method and apparatus

• US 6,141,034 A
• Filed: 12/15/1995
• Issued: 10/31/2000
• Est. Priority Date: 12/15/1995
• Status: Expired due to Term

• Current Assignee: Staar SA
• Original Assignee: Immersive Media Company
• Inventors: McCutchen, David
• Primary Examiner(s): Ton, Dang
• Assistant Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US08/572,785
• Time in Patent Office: 1,782 Days
• Field of Search: 348/36, 348/38, 348/39, 348/47-49, 348/52-4, 348/42, 348/51, 348/135, 348/139, 348/169-171, 348/159, 348/153-155, 348/214, 350/502, 352/70, 354/94, 395/119, 395/125, 386/15, 386/37, 386/38, 386/106
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G06T 3/067 Reshaping or unfolding 3D tree structures onto 2D planes
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/327 Calibration thereof
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 : using temporal multiplexing
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 : using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/365 : using digital micromirror devices [DMD]
• CPC Class Codes: H04N 13/366 : using viewer tracking
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 13/398 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 : specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0088 : Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Improvements in the production means for dodecahedral imaging are described, to define a simple, flexible and efficient strategy for the creation of extremely wide images. Improvements in production technology include lenses and optical mounts that enable automatic alignment of film and video cameras and other imaging devices to cover multiple fields of view equal to one, two or three dodecahedral pentagons. Three double-pentagon formats, X, P and O, are described in detail, the X format producing an even hemispherical division and a full spherical view, and the P format producing simplified coverage of most of a sphere and ease of use for stereoscopic effects, and the O based on the combination of the dodecahedron and the octahedron. A triple-pentagon T format based on the use of the tetrahedron is also described, along with crisscross optics and temporal offset mounts to minimize parallax differences between the separate camera views. Standard cropping mattes for dodecahedral images are given, and compressed storage methods are suggested for a more efficient distribution of dodecahedral images in a variety of media.

Claims

1. An improved method for photographing a wide field of view up to a complete sphere, including the steps of:

centering a dodecahedron inside a concentric sphere representing a spherical field of view, and evert extension of the faces of said dodecahedron onto said sphere, thereby forming twelve pentagonal sections; and

placing within the sphere at least one movable camera having a field of view of at least one pentagonal section; and

moving the camera so as to produce a succession of images, each image being of a different section of the wide field of view from the preceding image, which images together form a composite image of the wide field of view; and

combining the images to produce a record of the appearance of the wide field of view.

2. The method claim 1, wherein at least one pentagonal section is recorded more than once from offset points of view, for a stereoscopic effect.

3. A camera apparatus for recording the appearance of a wide field of view up to a complete sphere, by making use of six equally-spaced alignment points, comprising:

at least one subcamera, each subcamera comprising a lens having a field of view sufficient to encompass two adjacent pentagonal sections of a spherical field of view divided according to the projection of a concentric pentagonal dodecahedron; and

means for centering said field of view on at least one of the six horizontal dodecahedron edges immediately above and below an equatorial plane equidistant from opposite vertices and bisecting said pentagonal dodecahedron.

4. The camera apparatus of claim 3, wherein said means for centering include a movable support member coupled to said subcamera, for the successive realignment of said field of view to produce a plurality of successive images.

5. The camera apparatus of claim 3, further including including the bisection of said spherical field of view into two halves of three dual-pentagon images each, according to a hemisphere base plane tilted approximately 54°

relative to said equatorial plane, and wherein one double-pentagon image in each of the two halves is inverted.

6. The camera apparatus of claim 3, wherein said means for centering include alignment mount surfaces in the form of modular geometric shapes, comprising a rectangle with a short side and a long side, wherein the ratio of said long side to said short side is approximately equal to the Golden Section, and a trapezoid comprising three of said short sides and one of said long sides.

7. A camera apparatus for recording the appearance of a wide field of view up to a nearly complete sphere, by making use of a set of ten equally-spaced alignment points in a single plane, comprising:

at least one subcamera, each subcamera comprising a lens having a field of view sufficient to encompass two adjacent pentagonal sections of a spherical field of view divided according to the projection of a concentric pentagonal dodecahedron; and

means for centering said field of view on at least one of the ten decahedron edge centers defined by an equatorial ring plane bisecting and equidistant from opposite faces of said pentagonal dodecahedron.

8. The apparatus of claim 7, wherein said means for centering include a movable support member for successive realignment to adjacent positions selected from among said ten dodecahedron edge centers thereby producing double-pentagon images which overlap in a manner suitable for stereoscopic effects.

9. The apparatus of claim 7, wherein said means for centering include a movable support member for successive realignment to to either the odd or the even positions selected from among said ten dodecahedron edge centers, thereby creating a set of up to five non-overlapping double-pentagon images which cover a nearly complete spherical field of view.

10. A camera apparatus for recording the appearance of a wide field of view up to a complete sphere, comprising:

at least one subcamera, each subcamera comprising a lens having a field of view sufficient to encompass two adjacent pentagonal sections of a spherical field of view divided according to the projection of a concentric pentagonal dodecahedron; and

;

means for centering said field of view on one or more of a first set of six selected dodecahedron edge centers which also represent the vertices of a concentric octahedron.

11. The camera apparatus of claim 10, wherein said means for centering include a movable support member for the successive realignment of said field of view to at least one of said first set of selected dodecahedron edge centers.

12. The camera apparatus of claim 10, with a second set of six selected dodecahedron edges being determined by the rotation of said concentric octahedron by approximately 72 degrees, according to an axis of rotation equidistant from all of the vertices of said octahedron whereby double-pentagon images taken according to the second set of six selected dodecahedron edges will at least partly overlap double-pentagon images taken according to the first set, said overlap being suitable for stereoscopic effects.

13. A camera apparatus for recording the appearance of a wide field of view up to a complete sphere, comprising:

at least one subcamera, each subcamera comprising a lens having a field of view sufficient to encompass three adjacent pentagonal sections of a spherical field of view divided according to the projection of a concentric pentagonal dodecahedron; and

means for centering said field of view on one or more of a first set of four selected dodecahedron vertices which also represent the vertices of a concentric tetrahedron.

14. The camera apparatus of claim 13, wherein a rotated position for a second triple pentagon image is created by a rotation of said centering means by approximately 90°

, according to an axis connecting the centers of opposite edges of said tetrahedron, thereby creating an overlap of portions of a first triple pentagon image and said second triple-pentagon image for stereoscopic effect.

15. An optical apparatus for the transmission of images across crossing light paths, comprising:

A plurality of divided lens element sets, each divided lens element set capable of transmitting light along an optical path, said optical path including a central optical axis and representing a lens field of view, capable of being focused on an image plane, each divided lens element set comprising a front lens element and a rear lens element,

A rigid mount array comprising a plurality of parallel opposite faces, and an opening in each of said faces containing a coupling for a coupled lens element, said coupled lens element selected from the set consisting of front lens elements and rear lens elements, so that the only light entering or leaving the array will be through said coupled len elements, and so that said front and said rear lens elements will form a divided lens element set along an optical axis that crosses the center point of said rigid mount array, with no part of any of said divided lens element sets blocking the transmission of light across said rigid mount array along any of the optical paths of said divided lens element sets,

A plurality of cameras, each of said cameras coupled to one of said rear lens elements, and each designed to record the appearance of the image presented on said image plane.

16. The apparatus of claim 15, wherein said front and rear lens elements are designed to create an optical path which includes a nodal point roughly congruent to said center point of said rigid mount array.

17. A temporal offset optical mount apparatus for minimizing parallax differences through the use of one-frame increments of distance according to a given rate of movement, comprising:

A plurality of cameras in a row, beginning with a base reference camera, said cameras with lenses arranged so that their nodal points have an approximately equal spacing apart along an alignment line,

Synchronized separate recording means for producing an image record for all of said cameras, according to a constant time scale of frame divisions,

Means for measuring the rate of movement of said cameras in a direction parallel to said alignment line,

Means for adjusting the amount of said approximately equal spacing to a distance divisible by the distance covered over a one-frame span of time by one of said nodal points,

Means for subsequently adjusting the image record of a selected camera by a number of frames representing the number of one-frame spaces between it and the base reference camera.

18. An improved camera apparatus for recording the appearance of a wide field of view up to a complete sphere, comprising:

up to six subcameras, each subcamera comprising a lens having a field of view of at least one pentagonal section of a spherical field of view divided according to the projection of a concentric pentagonal dodecahedron; and

a rigid mount coupled to the subcameras, and arranged to orient each lens to face inward along an optical axis crossing the center point of the dodecahedron.

19. Recording apparatus especially suited for the recording of nearly spherical views, comprising:

a camera array having a center point and an overall field of view of most of a sphere, the camera array comprising multiple cameras, each of which covers a segment of the overall field of view, the camera array having a blind spot including at least one segment of the field of view and extending outward from the center point, the blind spot being invisible to the camera array; and

recording equipment coupled to the camera array and disposed within the blind spot.

20. The recording apparatus of claim 19, further comprising means for transportation of said camera array and said recording equipment together in a wheeled conveyence.

21. The recording apparatus of claim 19, further comprising means for transposition and combination of image information from multiple video camera feeds to record up to three pentagonal views on one video recorder without substantial image compression.

22. A modular global video projector for the projected display of up to an entire spherical image, comprising:

at least one light source, the light source being directed into a plurality of fiber optic light guides;

a projector array comprising a plurality of projector modules, each module being coupled to at least one of said fiber optic light guides, and each module producing an image segment projected outward through a lens, the projector array producing through the projector modules a contiguously arranged composite of image segments; and

a support structure connected to said projector array.

23. The modular global video projector of claim 22, wherein each projector module uses a Digital Multimirror Device (DMD).

24. The modular global video projector of claim 22, comprising means for avoiding cast shadows in a spherical image projected on a screen surface.

25. An easily assembled modular screen for a dome theater, comprising:

a modular framework of geodesic great circle lengths, the lengths having dimensions corresponding to the radius, altitude and at least one half of an edge of a spherical dodecahedron, as projected onto a concentric sphere; and

a plurality of screen elements attached to the modular framework, each screen element having a curvature corresponding to the concentric sphere and a standardized shape, with edges corresponding to said lengths, and means for attachment to said framework.