Immersive imaging system

• US 20040027451A1
• Filed: 04/08/2003
• Published: 02/12/2004
• Est. Priority Date: 04/12/2002
• Status: Active Grant

• Current Assignee: Image Masters Incorporated
• Original Assignee: Image Masters Incorporated
• Inventors: Baker, Robert G.
• Granted Patent: US 7,224,382 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system, method and device for the capture and display of high-resolution stereoscopic or monoscopic wide fields-of-view (FOV) at still or video rates, presenting techniques and designs for distance measuring, for changing the working stereo capture range of a stereoscopic image capture device, and for playing pre-recorded or transmitted stereoscopic video image data through a player unit. The system supports multiple independent viewers and image analysis sub-systems of various types utilizing hardware, software and firmware. Further included is a stereoscopic videoconferencing embodiment that captures and transmits an entire room'"'"'s view and automatically directs focus of images.

Claims

1. An imaging system for producing wide-angle stereoscopic images, the system comprising:

at least two optical sub-systems mounted so as to collectively capture a wide-angle field-of-view; and

a stereoscopic display processing sub-system for processing overlapping images simultaneously from each optical sub-system;

wherein each optical sub-system includes;

a field-of-view which overlaps with a field-of-view of each adjacent optical sub-system;

a first optical path with a first lens, a first lens-mounting structure and a first reflective element with a first reflective surface for directing an image in a first field-of-view to project on a first image sensor in the first optical path, wherein the first image sensor is mounted to a bottom side of a top capture board and the first reflective element has a first rear surface;

a second optical path with a second lens, a second lens-mounting structure and a second reflective element with a second reflective surface for directing an image in a second field-of-view to project on a second image sensor in the second optical path, wherein the second image sensor is mounted on the top side of a bottom capture board and the second reflective element has a second rear surface;

wherein the first rear surface and the second rear surface are positioned so that first rear surface and the second rear surface are facing each other in order for the first reflective surface of the first reflective element and the second reflective surface of the second reflective element to capture overlapping images derived from an image in the first field-of-view and an image in the second field-of-view on a common physical optical plane.

2. The imaging system of claim 1, wherein the wide-angle field-of-view is a portion of one of a panoramic field-of-view and spherical field-of-view.

3. The imaging system of claim 1, wherein the stereoscopic display processing sub-system further includes at least one display driver for providing at least one of multiple independent images simultaneously from at least one optical sub-system.

4. The imaging system of claim 3, wherein the stereoscopic display processing sub-system includes a point-of-view determinator to enable a user to select a direction for viewing the captured visual environment, the image data for which is taken from the imaging sub-systems.

5. The imaging system of claim 4, wherein the overlapping images which are selectable by the user include a field-of-view captured between the first optical path and the second optical path at an average interpupillary separation distance.

6. The imaging system of claim 4, wherein the overlapping images which are selectable by the user include a field-of-view captured between the first optical path and the second optical path at extrapupillary separation distance, and wherein the stereoscopic display processing sub-system further includes an interpolation processor for normalizing the overlapping images to an average interpupillary separation distance.

7. The imaging system of claim 1, wherein the overlapping images represent separate superimposed right-eye and left-eye image datasets which are organized as individual continuous image datasets in memory from which subsets representing visual areas of interest are selected.

8. The imaging system of claim 4, wherein the point-of-view determinator is driven by at least one audio beam-steering circuit.

9. The imaging system of claim 1, wherein the stereoscopic display processing sub-system includes a distance determination process which calculates a distance of at least one object in the first field-of-view and the second field-of-view, the distance calculated by using triangulation of a first ray received from the object through the first optical path and a second ray received from the object in the second optical path.

10. The imaging system of claim 1, wherein the stereoscopic display processing sub-system includes processing images in at least one of a still image rate and a video image rate.

11. The imaging system of claim 1, further comprising:

a drive mechanism coupled to each imaging sub-system, so as to move each imaging sub-system along a radial line from a center point about which all the imaging sub-systems are positioned so as to selectively adjust an effective stereo viewing range.

12. An imaging system for producing wide-angle stereoscopic images, the system comprising:

at least two optical sub-systems mounted so as to collectively capture a wide-angle field-of-view; and

a display processing sub-system for processing an image from each imaging sub-system so as to produce at least one stereoscopic image from overlapping fields-of-view captured from at least two imaging sub-systems;

wherein each imaging sub-system includes;

a field-of-view, which overlaps with a field-of-view of each adjacent optical sub-system;

an optical path with a lens, a lens-mounting structure, a reflective element; and

a mechanical sub-system for mechanically moving the reflective element in order to direct an image in a field-of-view to project on an image sensor in the optical path.

13. The imaging system of claim 12, wherein the wide-angle field-of-view is one of a panoramic field-of-view and spherical field-of-view.

14. The imaging system of claim 12, wherein the display processing sub-system further includes at least one display driver for providing at least one of multiple independent images simultaneously from at least one imaging sub-system.

15. The imaging system of claim 14, wherein the display processing sub-system includes a point-of-view determinator to enable a user to select a direction for viewing the captured visual environment, the image data for which is taken from at least one of the imaging sub-systems.

16. The imaging system of claim 12, wherein the display processing sub-system produces a stereoscopic image captured from a first image sensor in a first optical sub-system and a second image sensor in a second optical sub-system positioned at average interpupillary separation distance.

17. The imaging system of claim 12, wherein the display processing sub-system produces a stereoscopic image captured from a first image sensor in a first optical sub-system and a second image sensor in a second optical sub-system positioned at an extrapupillary separation distance, and wherein the display processing sub-system further includes an interpolation processor for normalizing the stereoscopic image to an average interpupillary separation distance.

18. The imaging system of claim 12, wherein the display processing sub-system includes a distance determination processor which calculates a distance of at least one object in the field-of-view using triangulation of a first ray received from the object through a first optical path and a second ray received from the object in a second optical path.

19. The imaging system of claim 12 wherein the display processing sub-system includes processing images in at least one of a still image rate and a video image rate.

20. The imaging system of claim 12, wherein the mechanical sub-system further comprises:

a drive mechanism coupled to each imaging sub-system, so as to move each imaging sub-system along a radial line from a center point about which all the imaging sub-systems are positioned so as to selectively adjust an effective stereo viewing range.

21. An imaging system for producing higher resolution composite images comprising:

a first image sensor of a given pixel density that produces a first image with a first set of pixel locations and a first set of inter-pixel spaces; and

a second image sensor of a given pixel density that produces a second image with a second set of pixel locations and a second set of inter-pixel spaces;

wherein the first image sensor is aligned relative to the second image sensor such that at least one of;

the first set of pixel locations aligns with the second set of inter-pixel spaces; and

the first set of inter-pixel spaces aligns with the second set of pixel locations;

so as to produce a higher-resolution composite image from the first image sensor and the second image sensor.

22. The imaging system of claim 21, further comprising:

an interpolator for producing higher apparent resolutions of the composite image.

23. The imaging system of claim 21, wherein the composite image is produced in at least one of a still image rate and a video image rate.

24. The imaging system of claim 21, wherein the first set of pixel locations aligns with the second set of inter-pixel spaces so as to avoid overlap with the second set of pixel locations.

25. A playback system comprising:

at least one point-of-view determinator for selecting a point-of-view from a wide-angle field-of-view;

at least one user interface for receiving user input to adjust the point-of-view determinator; and

at least one display driver for rendering images at one of a still image rate and a video image rate from the datasets selected.

26. The playback system of claim 25, further comprising:

at least one track synchronizer for reading at least two image streams wherein each image stream includes a plurality of datasets corresponding to the wide-angle field of view and wherein the datasets in each data stream corresponding to the point-of-view are selected.

27. The playback system of claim 25, further comprising:

a plurality of display drivers for rendering multiple independent images to separate viewers simultaneously where each independent image has a point-of-view which is selectable by a user.

28. The playback system of claim 25, further comprising:

at least one recordable medium for providing the image streams therefrom.

29. The playback system of claim 28, wherein the datasets are at least one of a monoscopic dataset and stereoscopic dataset.

30. The playback system of claim 26, wherein the user interface provides a selection of a point-of-view from at least one of a predetermined order and a user-selectable order.

31. A recorder for recording two or more video streams comprising:

an input for receiving at least two streaming video datasets, wherein each of the datasets represents a point-of-view from a wide-angle field-of-view; and

a recording device for recording the datasets from each streaming video on one or more recordable media whereby each dataset is uniquely identified based upon the point-of-view being represented so that each dataset necessary to re-display the point-of-view is identified for assembly in coherent fashion.

32. The recorder of claim 31, wherein the recording device further comprises recording at least one of a still image rate and a video image rate.

33. The recorder of claim 31, whereby each dataset is uniquely identified based upon the point-of-view being represented by at least one of a time-based identifier and a sequence-based identifier.