Stereoscopic wide field of view imaging system

• US 8,004,558 B2
• Filed: 04/07/2006
• Issued: 08/23/2011
• Est. Priority Date: 04/07/2005
• Status: Active Grant

• Current Assignee: Axis Engineering Technologies, Inc., Massachusetts Institute of Technology
• Original Assignee: Axis Engineering Technologies, Inc., Massachusetts Institute of Technology
• Inventors: Sedwick, Raymond J., Jonas, Eric M., Prechtl, Eric F.
• Primary Examiner(s): Villecco; John M
• Application Number:
  US11/400,029
• Publication Number:
  US 20070126863A1
• Time in Patent Office: 1,964 Days
• Field of Search: 348/47, 348/48, 348/39, 348/51, 348/52, 348/53, 348/211.2, 348/218.1
• US Class Current: 348/48
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A stereoscopic imaging system incorporates a plurality of imaging devices or cameras to generate a high resolution, wide field of view image database from which images can be combined in real time to provide wide field of view or panoramic or omni-directional still or video images.

Claims

1. A stereoscopic imaging system comprising:

a plurality of cameras arranged in camera pairs mounted to a fixed hub, each camera in a camera pair oriented such that their optical axes face similar directions to receive image data unobstructed by optical elements external to the camera to create a parallax effect, each camera in the camera pair disposed on opposite sides of the hub and separated by a distance perpendicular to their optical axes, the distance extending through the hub, one camera from each camera pair associated with a right eye and the other camera from each camera pair associated with a left eye, each camera in the camera pair being separately operable from the other camera in the camera pair, and each camera operative to provide a two-dimensional image; and

at least one processor for receiving image data separately and independently from each of the plurality of cameras, the processor operative to transmit image data from the cameras associated with the right eye to a right eye display of an image display device and to transmit image data from the cameras associated with the left eye to a left eye display of the image display device to provide stereoscopic imaging.

2. The stereoscopic imaging system of claim 1, wherein the processor is operative to blend multiple images from the cameras associated with the right eye into a single image and to blend multiple images from the cameras associated with the left eye into a single image.

3. The stereoscopic imaging system of claim 1, wherein the processor is operative to blend at least two images to form a wide field of view image.

4. The stereoscopic imaging system of claim 1, wherein the processor is operative to blend multiple images to form a panoramic image.

5. The stereoscopic imaging system of claim 1, wherein the processor is operative to blend multiple images to form a 360°

image.

6. The stereoscopic imaging system of claim 1, wherein the processor comprises a daughter board associated with each camera and operative to receive data signals from the associated camera, and a main processor board operative to receive data signals from each camera daughter board.

7. The stereoscopic imaging system of claim 1, wherein the processor comprises a plurality of devices in communication with a frame buffer operative to process image frames sequentially from a camera input device to an image transmission device.

8. The stereoscopic imaging system of claim 7, wherein the processor is operative to process image frames serially.

9. The stereoscopic imaging system of claim 7, wherein the processor is operative to process image frames in parallel.

10. The stereoscopic imaging system of claim 7, wherein the devices include a frame rotation device.

11. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to perform pixel corrections.

12. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to manipulate individual pixels or neighborhoods of pixels by identifying common pixel coordinates and forming a coordinate transformation matrix to map neighboring images into a common coordinate reference frame.

13. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to adjust brightness, color, contrast, integration time, shutter speed, white balance, signal gain, saturation level, or gamma correction.

14. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to perform an edge detection operation, linear filtering, or background enhancement.

15. The stereoscopic imaging system of claim 7, wherein the devices include a field integration device.

16. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to blend multiple images into a single image by identifying common pixel coordinates and forming a coordinate transformation matrix to map neighboring images into a common coordinate reference frame.

17. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to perform cropping, panning, or zooming of an image.

18. The stereoscopic imaging system of claim 7, wherein the devices include a device operative to perform whole-image affine, quadratic, Euclidean, or rotational transformation operations.

19. The stereoscopic imaging system of claim 7, wherein the devices include a data compression device.

20. The stereoscopic imaging system of claim 7, wherein the devices include an image data encoding device.

21. The stereoscopic imaging system of claim 7, wherein the frame buffer comprises a multiplexed random access memory in synchronous communication with a memory element.

22. The stereoscopic imaging system of claim 7, wherein the devices include field programmable devices.

23. The stereoscopic imaging system of claim 7, wherein the devices include field programmable gate arrays.

24. The stereoscopic imaging system of claim 7, wherein the devices include application specific integrated circuits.

25. The stereoscopic imaging system of claim 7, wherein the devices include a digital signal processor.

26. The stereoscopic imaging system of claim 7, wherein the devices include an electronic processor.

27. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data to a head mounted display device.

28. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data to the image display device in a page flipping mode.

29. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data to the image display device separately to a right eye display and to a left eye display.

30. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data to a video monitor or a video projector.

31. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data via a wireless connection.

32. The stereoscopic imaging system of claim 1, wherein the processor is operative to transmit the image data via a wired connection.

33. The stereoscopic imaging system of claim 1, further comprising at least one microphone in communication with the processor to provide an audio input.

34. The stereoscopic imaging system of claim 1, wherein a first subset of the camera pairs is arranged in a plane to provide a wide field of view, and a second subset of the camera pairs is arranged out of the plane to provide a different wide field of view, all camera pairs arranged to provide a common parallax effect, and the processor is operative to blend the images to assemble a multi-directional stereoscopic image.

35. A method of providing a stereoscopic image comprising:

arranging a plurality of cameras in camera pairs mounted to a fixed hub, each camera in a camera pair oriented such that their optical axes face similar directions to receive image data unobstructed by optical elements external to the camera, each camera in the camera pair disposed on opposite sides of the hub and separated by a distance perpendicular to their optical axes, the distance extending through the hub, one camera from each camera pair associated with a right eye and the other camera from each camera pair associated with a left eye, each camera in the camera pair being separately operable from the other camera in the camera pair, and each camera operative to provide a two-dimensional image;

obtaining image data from the plurality of cameras;

transmitting image data from the cameras associated with the right eye to a right eye display of an image display device and separately and independently transmitting image data from the cameras associated with the left eye to a left eye display of the image display device to provide stereoscopic imaging.

36. The method of claim 35, further comprising blending the image data from the plurality of cameras associated with one of the right eye or the left eye into a single wide field of view image.

37. The method of claim 35, further comprising blending the image data from the plurality of cameras to form a 360°

panoramic image.

38. The method of claim 35, further comprising processing frames serially from the plurality of cameras to output an image to the display device.

39. The method of claim 35, wherein a first subset of the camera pairs is arranged in a plane to provide a wide field of view, and a second subset of the camera pairs is arranged out of the plane to provide a different wide field of view, all camera pairs arranged to provide a common parallax effect, and the processor is operative to blend the images to assemble a multi-directional stereoscopic image.