Method and apparatus for hemispheric imaging which emphasizes peripheral content

• US 5,508,734 A
• Filed: 07/27/1994
• Issued: 04/16/1996
• Est. Priority Date: 07/27/1994
• Status: Expired due to Term

• Current Assignee: MediaTek, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kettler, Kevin, Baker, Robert G., Freedenberg, Candace J. F., Suarez, Gustavo A., Uplinger, Kenneth A.
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/281,331
• Time in Patent Office: 629 Days
• Field of Search: 348/36, 348/37, 348/38, 348/39, 348/53, 348/580
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 3/0087 with index gradient
• CPC Class Codes: G02B 6/04 formed by bundles of fibres G02B6/24 takes precedence
• CPC Class Codes: G02B 6/06 the relative position of the fibres being the same at both ends, e.g. for transporting images
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A system for electronic imaging of a hemispheric field of view includes a camera for receiving optical images of the field of view and for producing output data corresponding to the optical images. The camera includes an optical assembly for producing images throughout a hemispheric field of view for optical conveyance to an imaging device or photographic film. The optical system assembly has lens components that selectively emphasize the peripheral content of the hemispheric field of view. An electronic imaging device within the camera or a film-to-digital date conversion system provides digitized output signals to input image memory or electronic storage devices. A transform processor selectively accesses and processes the digitized output signals from the input image memory according to user-defined criteria and stores the signals in output image memory. The signals in the output image memory can then be displayed according to the user-defined criteria.

Claims

1. A system for electronic imaging and manipulation of a hemispheric field of view, comprising:

a camera for receiving optical images of a hemispheric field of view and for producing output signals or affecting photographic film-based materials corresponding to the optical images;

an optical imaging device associated with said camera for producing optical images throughout the hemispheric field of view for optical conveyance to said camera, said optical system having a configuration adapted to capture and enhance an image of peripheral regions of the hemispheric field of view and for magnifying a portion of the field of view within a range of approximately one to forty-five degrees above the horizon of a captured image of a hemispheric field of view;

an image processing device associated with said camera and said optical imaging system for receiving optical images from said lens and for providing digitized output signals representative of the received optical images;

input image memory for receiving the digitized output signals from said imager device and for storing the digitized output signals;

image transform processing circuitry for selectively accessing and processing the digitized output signals from said input image memory according to user defined criteria;

output image memory for receiving the processed signals from the image transform processor, and

an output display or recording device connected to said output image memory for recording the signals in said output image memory.

2. The system as in claim 1, wherein said image processing device is an electronic imager.

3. The system as in claim 2, wherein said image processing device is a charge-coupled device.

4. The system as in claim 1, wherein said optical system has a configuration which images a peripheral portion of the hemispheric scene onto at least 50% of an imaging area of an imaging device.

5. The system as in claim 4, wherein said optical system includes a wide field multi-element lens positioned to direct an image to a light transmitting fiber array.

6. The system as in claim 5 wherein the fiber array is geometrically arranged to have a generally annular input end and a generally rectangular output end.

7. The system as in claim 5, wherein the fiber array is geometrically arranged to have a generally annular input end and a generally circular output end.

8. The system of claim 5 wherein the fibers of the fiber array have an imaging accuracy on the order of three microns.

9. The system as in claim 5 wherein the focal length of the wide field lens is scaled to match a desired magnification of the peripheral field of view.

21. The system as in one of claim 1 or claim 10 or claim 11, wherein said output display is worn on or attached to a viewer'"'"'s head, so that the orientation of the viewer with respect to the imaged surroundings is electronically conveyed and interpreted as user controls.

10. A system for electronic imaging and manipulation of a hemispheric field of view, comprising:

a camera for receiving optical images of a hemispheric field of view and for producing output signals or affecting photographic film-based materials corresponding to the optical images;

an optical imaging device associated with said camera for producing optical images throughout the hemispheric field of view for optical conveyance to said camera, said optical system having a configuration adapted to capture and enhance an image of peripheral regions of the hemispheric field of view, said optical imaging device including a color-aberrated multiple element wide field lens in combination with a gradient index hemispheric lens;

an image processing device associated with said camera and said optical imaging system for receiving optical images from said lens and for providing digitized output signals representative of the received optical images;

input image memory for receiving the digitized output signals from said imager device and for storing the digitized output signals;

image transform processing circuitry for selectively accessing and processing the digitized output signals from said input image memory according to user defined criteria;

output image memory for receiving the processed signals from the image transform processor, and

an output display or recording device connected to said output image memory for recording the signals in said output image memory.

11. A system for electronic imaging and manipulation of a hemispheric field of view, comprising;

a camera for receiving optical images of a hemispheric field of view and for producing output signals or affecting photographic film-based materials corresponding to the optical images;

an optical imaging device associated with said camera for producing optical images throughout the hemispheric field of view for optical conveyance to said camera, said optical system having a configuration adapted to capture and enhance an image of peripheral regions of the hemispheric field of view and for magnifying a portion of the field of view within a range of approximately one to forty-five degrees above the horizon of a captured image of a hemispheric field of view and imaging the magnified portion of the hemispheric scene onto at least 50% of an imaging area of an imaging device;

an image processing device including photographic film associated with said camera and said optical imaging system for receiving optical images from said lens and for providing digitized output signals representative of the received optical images;

input image memory for receiving the digitized output signals from said imager device and for storing the digitized output signals;

image transform processing circuitry for selectively accessing and processing the digitized output signals from said input image memory according to user defined criteria;

output image memory for receiving the processed signals from the image transform processor, and

an output display or recording device connected to said output image memory for recording the signals in said output image memory.

12. The system as in claim 11, further including a capture and conversion device that converts the optical images on said photographic film into digital output signals for input to said input image memory.

13. The system as in claim 12, wherein the capture and conversion device is an NTSC-to-digital converter.

14. The system as in claim 12, wherein data from the capture and conversion device is input to a source image frame buffer.

15. The system as in claim 14, wherein the image transform processing circuitry comprises an image re-sampling sequencer.

16. The system as in claim 15, wherein the image re-sampling sequencer controls address sequencing of pixels in the source image frame buffer.

17. The system as in claim 16, wherein the image transform processing circuitry further comprises row and column warp engines.

18. The system as in claim 17, wherein the image transform processing circuitry further comprises a warped image buffer connected to the warp engines and to which an output of the source image frame buffer is input.

19. The system as in claim 18, wherein the image transform processing circuitry further comprises a look-up table containing transformation parameters for perspective correction of hemispheric field of view images.

20. The system as in claim 18, wherein the image transform processing circuitry further comprises an interpolation coefficient buffer connected to said look-up table to update said transformation parameters, and connected to a multiply/accumulate unit.

22. A system for imaging and manipulation of a hemispheric field of view comprising:

(i) an optical system for producing an optical image of a hemispherical field of view, said optical system having a central lens axis coaxial with a central axis of the hemispherical field of view and a configuration which emphasizes through differential magnification of peripheral content of the hemispherical field of view, said optical image having a defined usable image area when projected onto a plane;

(ii) a camera optically coupled to said optical system for receiving optical images of the hemispherical field of view and for producing an output corresponding to the optical images;

(iii) input image memory coupled to said camera for receiving and storing the output from said camera;

(iv) a processor coupled to said input image memory for selectively accessing from said input image memory and processing output from said camera, said processor for transforming the output from the camera according to user defined criteria into a processor output which has a peripheral image content emphasis which differs from that of said optical system;

(v) output image memory coupled to said processor for receiving and storing the processor output; and

(vi) an output device coupled to said output image memory for rendering the stored processor output into a visual image transformed from the optical image in accordance with the user defined criteria.

23. A system according to claim 22 wherein said optical system has a configuration that images the peripheral portion of a hemispherical scene onto a portion of the optical image usable image area which is no more than about ninety percent of the optical image usable image area.

24. A system according to claim 22 wherein said optical system comprises:

a wide field multi-element lens; and

a coordinated fiber array geometrically arranged to have a generally annular input end and a generally rectangular output end.

25. A system according to claim 22 wherein said optical system comprises:

a wide field multi-element lens; and

a coordinated fiber array geometrically arranged to have a generally annular input end and a generally annular output end.

26. A system according to claim 22 wherein said optical system comprises:

a color-aberrated wide field lens; and

a gradient index hemispheric lens.

27. A system according to claim 22 wherein said output device comprises a visual display device for displaying a visual image closely adjacent the eyes of an observer.

28. A system according to claim 27 wherein said processor generates processor output which represents a selected portion of the hemispherical field of view, and said system further comprises:

sensors for detecting the orientation of the head or eyes of an observer, and

a link between said sensors to said processor for modifying the generation of processor output as a function of the detected orientation of the head or eyes of an observer.

29. A system according to claim 22 wherein said optical system has a configuration that images the peripheral portion of a hemispherical scene onto a portion of the optical image usable image area which is at least about fifty percent of the optical image usable image area.

30. A system according to claim 29 wherein said optical system has a configuration that images the peripheral portion of a hemispherical scene onto a portion of the optical image usable image area which ranges from about fifty percent to about ninety percent of the optical image usable image area.

31. A system according to claim 29 wherein said optical system has a configuration which emphasizes through differential magnification at least a portion of that content of the hemispherical field of view which lies between a base plane of the hemispherical field of view and a right cone defined about said central lens axis, said cone having an included angle of forty five degrees between said central lens axis and a line generating said cone and with said line generating said cone passing through the point of intersection of said central lens axis with said base plane.

32. A system according to claim 22 further comprising an imager device interposed between said camera and said input image memory for receiving said output from said camera and for generating a digitized output, and further wherein said input image memory, said processor, and said output image memory receive, store and process said digitized output.

33. A system according to claim 32 wherein said camera comprises a light sensitive electronic image capture element and said imager device comprises an electronic digitizer circuit.

34. A system according to claim 32 further comprising photographic film for photographically capturing said output from said camera and further wherein said imager device comprises a capture and conversion device which converts the photographically captured output from said camera into said digitized output.

35. A system according to claim 22 wherein said processor comprises image processing and warping circuitry for transforming the output from the camera according to user defined criteria into a processor output which has a peripheral content emphasis which differs from that of said optical system.

36. A system according to claim 35 wherein said image processing and warping circuit comprises an arithmetic logic unit (ALU), and further wherein said processor comprises a storage memory device coupled to said ALU for receiving and storing control programs for determining the functions served by said ALU, and a control program stored in said storage memory device and accessible to said ALU for controlling the operation of said ALU to transform the output from the camera according to user defined criteria into a processor output which has a peripheral/central content emphasis which differs from that of said optical system.

37. A system according to claim 35 wherein said processor generates processor output which represents a selected portion of the hemispherical field of view.

38. A method for electronically capturing, storing, and manipulating a hemispheric field of view, comprising the steps of:

providing an optical system having a configuration that enhances the peripheral portion of the field of view,

capturing the hemispheric field of view with the periphery-enhancing optical system and imaging the field of view onto an imager device while enhancing the peripheral field of view by selectively magnifying the visual content within an arc of between 10 and 45 degrees up from the horizon storing the captured image as a single image,

selectively accessing a portion of the stored image to user-defined criteria,

transforming the stored image so that the stored image can be displayed as a perspective-correct image, and

displaying the perspective-correct image in a user-defined format.

39. The method as in claim 38, wherein said transforming step comprises manipulating the peripheral-enhanced image into a perspective-correct image.

40. The method as in claim 38, wherein the storage step comprises storing the captured image in electronic storage.

41. A method for electronically capturing, storing, and manipulating a hemispheric field of view, comprising the steps of:

providing an optical system having a configuration that enhances the peripheral portion of the field of view,

capturing the hemispheric field of view with the periphery-enhancing optical system and imaging the field of view onto an imager device by enhancing the peripheral field of view,

storing the captured image onto photographic film as a single image;

selectively accessing a portion of the stored image according to user-defined criteria including converting the image from the photographic film into digital output format,

transforming the stored image so that the stored image can be displayed as a perspective-correct image, and

displaying the perspective-correct image in a user-defined format.

42. A method for electronically manipulating a hemispheric scene having an enhanced peripheral field of view stored as an image on photographic film, comprising the steps of:

converting the image on the photographic film into electronic output signals,

selectively accessing a portion of the output signals according to user-defined criteria,

transforming the accessed portion of the output signals by manipulating the peripheral-enhanced field of view so that the stored image can be displayed as a perspective-correct image, and

displaying the perspective-correct image in the user-defined format.

43. A method for generating image sequences over physical distances from a reduced set of enhanced-periphery hemispheric images, wherein intermediate images are interpolated from a combination of adjacent hemispheric images, the method comprising the steps of:

defining how many intermediate images are needed to satisfy the simulation of motion-video or film-based movie cameras;

interpolating needed intermediate images from a combination of adjacent hemispheric images;

identifying or electronically tagging interpolated images, then storing or recording them for later introduction to the image processing subsystem;

loading original and interpolated image data sets in appropriate sequence to the image processing subsystem at rates sufficient to simulate real-time motion video, and

displaying the perspective-correct image in the user-defined format.

44. A system for electronic manipulation of a hemispheric scene, comprising:

a camera imaging system for imaging a hemispheric field of view;

an optical system associated with said camera imaging system for producing the optical images throughout the field of view for optical conveyance to said camera imaging system;

an imager device associated with said camera for receiving the optical images from said lens and for providing digitized output signals;

input image memory for receiving the digitized output signals from said imaging device and for storing the digitized output signals;

image transform processor circuitry for selectively accessing and processing the digitized output signals from said input image memory according to user-defined criteria;

output image memory for receiving the processed signals from the image transform processor means; and

an output display device or electronic recording device connected to said output image memory for displaying or recording the signals in said output image memory according to user-defined criteria;

said optical system having a configuration that emphasizes the peripheral content of the field of view of a hemispheric scene as compared to the central content, such that said imager device receives magnified optical images of the peripheral portion of the hemispheric field of view.