Seamless multi-camera panoramic imaging with distortion correction and selectable field of view

• US 5,657,073 A
• Filed: 06/01/1995
• Issued: 08/12/1997
• Est. Priority Date: 06/01/1995
• Status: Expired due to Fees

• Current Assignee: Interactive Pictures, Inc., Pinoramic Vision Systems, Inc., Sony Corporation (Sony Group Corp.)
• Original Assignee: Panoramic Viewing Systems, Inc.
• Inventors: Henley, Stuart L.
• Primary Examiner(s): Au, Amelia
• Assistant Examiner(s): LIN, BRYAN T
• Application Number:
  US08/458,060
• Time in Patent Office: 803 Days
• Field of Search: 348/38, 348/36, 348/39, 348/37, 348/60, 354/94
• US Class Current: 348/38
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/334 using spectral multiplexing
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An imaging system for the production of panoramic or panospheric output images in still or video formats, by seamlessly merging an arbitrary number of simultaneous, overlapping input images which collectively encompass an entire field of view. The system includes a plurality of cameras which direct multiple simultaneous streams of analog or digital input into an image transformation engine, to process those streams to remove distortion and redundant information, creating a single output image in a cylindrical or spherical perspective. The output image signal is directed through an image clipping system, which acts under the control of a pan-flit-rotation-zoom controller to select a portion of the panoramic or panospheric image for display on a monitor, screen or headset.

Claims

1. An imaging system for the production of a panoramic or a panospheric output image for the simultaneous seamless merging of a plurality of overlapping input images to collectively display an entire field of panoramic view, comprising;

a plurality of cameras mounted so as to collectively capture said panoramic field of view;

a means to direct simultaneous streams of digital or analog output from each of said cameras, into an image transformation engine;

said image transformation engine arranged to process digital or analog signals so as to remove distortions generated by the image capture process, so as to seamlessly merge the images from adjacent cameras, by removing redundant pixels by digital separation and coloration to create a single output stream of digital or analog video;

a means to direct said output stream of analog or digital video to an image clipper, including a transparent color filter;

a pan-tilt-rotation-zoom controller arranged to control said image clipper to select a portion of said panoramic or panospheric image for display; and

a display device to permit the viewing, by the separate eyes of an observer, of the overlapping regions of an output image of said image clipper.

2. The imaging system as recited in claim 1, wherein at least a portion of the collective field of view is overlapping, and in which said image transformation engine provides a digital separation of redundant pixels into at least two separate output streams, each of said output streams being directed to a separate eye of an observer, so as to provide stereoscopic viewing of overlapping regions.

3. The imaging system as recited in claim 1, wherein said display device may be selected from the group comprising a monitor, television, virtual reality helmet, or projective display.

4. A process for the seamless display of a plurality of images, all of which have an overlapping field of view between adjacent images, said process comprising:

arranging a plurality of cameras so as to collectively capture a panoramic or panospheric field of view;

directing a stream of output from each of said cameras into an image transformation engine;

removing distortions introduced by the image capturing process in said image transformation engine;

removing redundant pixels to seamlessly merge said images and subsequently outputting a single stream of digital or analog video from said engine;

directing output stream of digital or analog video to an image clipper;

controlling said image clipper by a pan-tilt-rotation-zoom controller to select a portion of the merged images;

color toning each of said separate output streams so that they can be blocked by a translucent color filter arrangement prior to observation thereof;

directing each of said two separate viewable output streams from the collective overlapping field of view through said translucent color filter arrangement to separate eyes of an observer to permit stereoscopic viewing of overlapping regions; and

displaying said portion for viewing thereof.

5. The process as recited in claim 4, including the step of:

separating digitally, redundant pixels of said output stream into two separate viewable output streams.

6. The process as recited in claim 5, including the step of:

directing each of said two separate viewable output streams to the separate eyes of an observer to permit stereoscopic viewing of overlapping regions.