Digital video panoramic image capture and display system

• US 5,721,585 A
• Filed: 08/08/1996
• Issued: 02/24/1998
• Est. Priority Date: 08/08/1996
• Status: Expired due to Fees

• Current Assignee: Jeffrey D. Keast, John F. Buford
• Original Assignee: Jeffrey D. Keast, John F. Buford
• Inventors: Buford, John F., Keast, Jeffrey D.
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): DIN, LUANNE PAN
• Application Number:
  US08/695,078
• Time in Patent Office: 565 Days
• Field of Search: 348/36-39, 348/42-60
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 27/642 Optical derotators, i.e. systems for compensating for image rotation, e.g. using rotating prisms, mirrors
• CPC Class Codes: G02B 5/04 Prisms
• CPC Class Codes: G02B 5/08 Mirrors vehicle mirrors involving special optical features B60R1/08
• CPC Class Codes: G09B 9/326 the image being transformed by optical means
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A system performing capture and display of both still and real-time motion picture panoramic images comprised of 360 degrees in azimuth and ±180 degrees in declination for a virtual spherical field of view wherein image capture is achieved via a horizontally scanned vertical slit image passing through a fish-eye lens, mirrors, and prisms rotated under servo motion control onto a stationary line-scan sensor to build up a continuous two dimensional image. A host computer system monitors a position encoder on the optical head, sends motion commands to the servo controller, controls sampling rate and integration time for the line-scan sensor, and does image data processing, encoding, and storage. Additional mechanisms in the optical head permit optical and solid-state image magnification, selective field of view and sub-sampling of pixel data, optical horizon shifting, and streak and slit-scan camera effects. Image display is possible either through various methods of mapping the spherical image to a conventional flat display, or using the same optical head to project the image onto the interior of a spherical screen. Synchronous image scanning and projection from a single optical head is accommodated. Variations on the optical head includes options for stereoscopic imagery and miniaturized optics. Designs permitting both adaptive and interactive control of image capture and display are presented.

Claims

1. A panoramic digital video camera for recording incident light from three-dimensional images comprising:

a scanning head comprising a rotating wide-angle lens which collects the incident light;

azimuthal means of rotating the scanning head;

a linear array sensor axis;

optical means for projecting the light from the lens onto the sensor, which includes;

a 45 degree front surface mirror;

a dove prism counter-rotated to the scanning head;

a first achromatic relay lens;

a second achromatic relay lens; and

a front-surface mirror; and

means for maintaining the light projected from the lens stationarily on the sensor during scanning,

whereby when the scanning head is rotated a vertical slice of light collected by the lens is reflected along the axis of rotation by the 45-degree front surface mirror, which then is relayed by the achromatic relay lens to the dove prism which derotates the image and passes it to the second relay lens which converges the image onto the front-surface mirror which then projects the image stationarily onto the sensor array, an electrical signal is produced from which the image can be reproduced.

2. The camera according to claim 1, wherein the front-surface mirror is replaced by a beam-splitter which passes incident light from an emitter array back through the optical system for projection onto a spherical surface, while the incoming image is imaged onto the linear sensor array.

3. The camera according to claim 2 wherein the linear array is of the type which allows images to be captured, stored and projected at rates of 30 or more images per second.

4. A high-resolution camera according to claim 3, wherein the number of array elements is equal to or greater than 2000, whereby the camera may be used in conjunction with HDTV images.

5. The camera according to claim 4, further comprising means for adjusting the front surface mirror, and in which the azimuthal span is controlled by adjusting said front surface mirror.

6. The camera according to claim 5, farther comprising means for adjusting the linear array, and in which the azimuthal resolution may be varied with varying azimuth angles by controlling said linear array.

7. The camera according to claim 6 in which by additional sensors, navigational position information can be used to trigger the automatic capture of an area, and camera position and orientation data can be recorded with each image.

8. The camera according to claim 4, wherein the optics and linear arrays permit display and recording of both optical and infra-red images.

9. The camera according to claim 6 further comprising:

means to control intensity of the image at the linear array sensor and

means to normalize the input data,

whereby horizontal intensity correction at the camera are performed by intensity control means; and

vertical intensity corrections are performed by said means to normalize the input data.

10. The camera according to claim 6, further comprising a flexible fiber optic cable located between the fish-eye lens and the means for derotation of the image, whereby the means for derotation are physically decoupled from the scanning head.

11. The camera according to claim 6 further comprising a second means of control over the scanning head, said second means of control being independent of the first means of control, whereby certain real-time specific effects or controlled distortions, such as slanting or stretching of the picture can be created thereby.

12. The camera according to claim 6 further comprising;

a system of directional microphones;

means for determining the direction of a sound from the output of the microphone system, and

means for controlling the scanning head are controlled by the sound direction, whereby audio directionality can be recreated during projection using conventional audio techniques.

13. The camera of claim 11, further comprising:

a spherical viewing module having an interior viewing surface and containing a viewing position;

means to process the output of the sensor array;

means to convert the processed output of the sensor array into a display image;

means to project the display image on the interior surface of the viewing module; and

means to rotationally control the relative position of the display image and the viewing position,

whereby when a viewer is located at the viewing position within the viewing module, a realistic reproduction of the display image may be seen by the viewer on the interior surface of the viewing module, and may be moved relative to the viewer.

14. The camera according to one of claims 1 through 12, further comprising means to store the output of the sensor array.

15. A panoramic digital video camera for recording incident light from three-dimensional images comprising:

two scanning heads, each comprising a rotating wide-angle lens which collects the incident light, said heads being fixed in location with respect to each other at a stereographic separation;

azimuthal means of rotating the scanning heads;

two linear sensor arrays, one corresponding to each scanning head;

optical means for projecting the light from the two lenses onto the sensor arrays;

means for maintaining the light projected from the lenses stationarily on the sensors during scanning;

whereby when the scanning heads are rotated a vertical slice of incident light collected by each lens is constantly projected onto the corresponding sensor array, which produces electrical signal from which the image can be stereoscopically produced and recorded.

16. The camera according to claim 15 wherein the linears array are of the type which allows images to be captured, stored and projected at rates of 30 or more images per second.

17. A high-resolution camera according to claim 16, wherein the number of array elements in each array is equal to or greater than 2000, whereby the camera may be used in conjunction with HDTV images.

18. The camera according to claim 17, wherein the optical means for projecting the light from the lens stationarily on the sensor during scanning further comprises 45 degree front surface mirrors, further comprising means for adjusting the front surface mirrors, and in which the azimuthal span is controlled by adjusting said front surface mirrors.

19. The camera according to claim 17, wherein the optics and linear arrays permit display and recording of both optical and infra-red images.

20. The camera according to one of claims 17 through 19, further comprising means to store the output of the sensor arrays.

21. The camera according to claim 18, further comprising means for adjusting the linear arrays, and in which the azimuthal resolution may be varied with varying azimuth angles by controlling said linear arrays.

22. The camera according to claim 21, in which by additional sensors, navigational position information can be used to trigger the automatic capture of an area, and camera position and orientation data can be recorded with each image.

23. The camera according to claim 21, further comprising:

means to control intensity of the image at the linear array sensors and

means to normalize the input data,

whereby horizontal intensity correction at the camera are performed by intensity control means; and

vertical intensity corrections are performed by said means to normalize the input data.

24. The camera according to claim 21, further comprising a flexible fiber optic cable located between each wide-angle lens and the means for derotation of the image, whereby the means for derotation are physically decoupled from the scanning heads.

25. The camera according to claim 21, further comprising a second means of control over the scanning heads, said second means of control being independent of the first means of control, whereby certain real-time specific effects or controlled distortions, such as slanting or stretching of the picture can be created thereby.

26. The camera according to claim 21 further comprising;

a system of directional microphones;

means for determining the direction of a sound from the output of the microphone system, and

means for controlling the scanning heads are controlled by the sound direction,

whereby audio directionality can be recreated during projection using conventional audio techniques.