Stereoscopic wide field of view imaging system

• US 20070014347A1
• Filed: 04/07/2006
• Published: 01/18/2007
• Est. Priority Date: 04/07/2005
• Status: Active Grant

• Current Assignee: Axis Engineering Technologies, Inc., Massachusetts Institute of Technology
• Original Assignee: Axis Engineering Technologies, Inc.
• Inventors: Jonas, Eric, Prechtl, Eric, Sedwick, Raymond
• Granted Patent: US 7,982,777 B2
• Time in Patent Office: Days
• US Class Current: 375/240.10
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A stereoscopic imaging system incorporates a plurality of imaging devices or cameras to generate a high resolution, wide field of view image database from which images can be combined in real time to provide wide field of view or panoramic or omni-directional still or video images.

Claims

1. A panoramic imaging system comprising:

a plurality of cameras arranged about a hub having a center, a first portion of the cameras facing in a clockwise direction relative to the hub center, a second portion of the cameras facing in a counterclockwise direction relative to the hub center; and

at least one processor operative to receive image data from the plurality of cameras, the processor operative to blend images from the cameras into a single panoramic image.

2. The panoramic imaging system of claim 1, wherein the processor is operative to blend images from the cameras into a 360°

panoramic image.

3. The panoramic imaging system of claim 1, wherein the cameras are arranged such that their optical axes lie substantially in a plane.

4. The panoramic imaging system of claim 1, wherein the cameras are arranged such that their optical axes are substantially tangential to a circle about the hub center.

5. The panoramic imaging system of claim 1, wherein the cameras are arranged with a first subset of the first portion angled upwardly and a second subset of the first portion angled downwardly, and a first subset of the second portion angled upwardly and a second subset of the second portion angled downwardly.

6. The panoramic imaging system of claim 1, wherein a number of cameras are arranged to provide an omni-directional image when blended by the processor.

7. The panoramic imaging system of claim 1, wherein the plurality of cameras are arranged in camera pairs, each camera in a camera pair oriented such that their optical axes face similar directions, each camera in the camera pair separated by an equal distance, one camera from each camera pair associated with a right eye and the other camera from each camera pair associated with a left eye.

8. The panoramic imaging system of claim 1, wherein the processor is further operative to transmit image data from the cameras associated with the right eye to a right eye display of an image display device and to transmit image data from the cameras associated with the left eye to a left eye display of the image display device to provide stereoscopic imaging.

9. The panoramic imaging system of claim 1, wherein the cameras are arranged such that a portion of one camera is visible by an adjacent camera, and

the processor is operative to provided a blended image from the one camera and the adjacent camera in which the obstructed view portion from the image of the adjacent camera is removed and replaced with an overlapping view portion from the one camera.

10. The panoramic imaging system of claim 1, wherein the processor is operative to provide a blended image from multiple cameras in which an obstructed view portion from the image of any one of the multiple cameras is removed and replaced with an overlapping view portion from an adjacent camera.

11. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data via a wireless connection.

12. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data via a wired connection.

13. The panoramic imaging system of claim 1 further comprising at least one microphone in communication with the processor to provide an audio input.

14. The panoramic imaging system of claim 1, wherein the processor comprises a daughter board associated with each camera and operative to receive data signals from the associated camera, and a main processor board operative to receive data signals from each camera daughter board.

15. The panoramic imaging system of claim 1, wherein the processor comprises a plurality of devices in communication with a frame buffer operative to process image frames sequentially from a camera input device to an image transmission device.

16. The panoramic imaging system of claim 15, wherein the processor is operative to process image frames serially.

17. The panoramic imaging system of claim 15, wherein the processor is operative to process image frames in parallel.

18. The panoramic imaging system of claim 15, wherein the devices include a frame rotation device.

19. The panoramic imaging system of claim 15, wherein the devices include a device operative to perform pixel corrections.

20. The panoramic imaging system of claim 15, wherein the devices include a device operative to manipulate individual pixels or neighborhoods of pixels.

21. The panoramic imaging system of claim 15, wherein the devices include a device operative to adjust brightness, color, contrast, integration time, shutter speed, white balance, signal gain, saturation level, or gamma correction.

22. The panoramic imaging system of claim 15, wherein the devices include a device operative to perform an edge detection operation, linear filtering, or background enhancement.

23. The panoramic imaging system of claim 15, wherein the devices include a field integration device.

24. The panoramic imaging system of claim 15, wherein the devices include a device operative to blend multiple images into a single image.

25. The panoramic imaging system of claim 15, wherein the devices include a device operative to perform cropping, panning, or zooming of an image.

26. The panoramic imaging system of claim 15, wherein the device include a device operative to perform whole-image affine, quadratic, Euclidean, or rotational transformation operations.

27. The panoramic imaging system of claim 15, wherein the devices include a data compression device.

28. The panoramic imaging system of claim 15, wherein the devices include an image data encoding device.

29. The panoramic imaging system of claim 15, wherein the frame buffer comprises a multiplexed random access memory in synchronous communication with a memory element.

30. The panoramic imaging system of claim 15, wherein the devices include field programmable devices.

31. The panoramic imaging system of claim 15, wherein the devices include field programmable gate arrays.

32. The panoramic imaging system of claim 15, wherein the devices include application specific integrated circuits.

33. The panoramic imaging system of claim 15, wherein the devices include a digital signal processor.

34. The panoramic imaging system of claim 15, wherein the devices include an electronic processor.

35. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data to a head mounted display device.

36. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data to the image display device in a page flipping mode.

37. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data to the image display device separately to a right eye display and to a left eye display.

38. The panoramic imaging system of claim 1, wherein the processor is operative to transmit the image data to a video monitor or a video projector.

39. A method of providing a panoramic image comprising:

arranging a plurality of cameras about a hub having a center, a first portion of the cameras facing in a clockwise direction relative to the hub center, a second portion of the cameras facing in a counterclockwise direction relative to the hub center;

providing at least one processor having a memory and a plurality of processing devices arranged to process frames serially from an input to an output to a display device;

inputting image data from the plurality of cameras as frames to the processor;

blending a plurality of frames to provide a single panoramic image; and

transmitting the panoramic image from the processor to the display device.

40. The method of claim 39, further comprising adjusting brightness, color, contrast, integration time, shutter speed, white balance, signal gain, saturation level, or gamma correction.

41. The method of claim 39, further comprising rotating a frame to a different orientation

42. The method of claim 39, further comprising cropping, panning, or zooming the image.

43. The method of claim 39, further comprising compressing the image data prior to transmission to the display device.

44. The method of claim 39, further comprising encoding the image data prior to transmission to the display device.

45. The method of claim 39, further comprising transmitting the panoramic image to a plurality of display devices.

46. The method of claim 39, further comprising receiving the transmitted image at the display device, and providing cropping, panning, and zooming of the image at the display device.

47. A panoramic imaging system comprising:

a plurality of cameras arranged about a hub having a center, the cameras arranged such that their optical axes are substantially tangential to a circle about the center of the hub; and

at least one processor operative to receive image data from the plurality of cameras, the processor operative to blend images from the cameras into a single panoramic image.

48. The panoramic imaging system of claim 47, wherein the processor is operative to blend images from the cameras into a 360°

panoramic image.

49. The panoramic imaging system of claim 47, wherein the processor is operative to blend images from the cameras into a 4π

steradian omni-directional image.

50. The panoramic imaging system of claim 47, wherein the cameras are arranged such that a portion of one camera is visible by an adjacent camera, and

the processor is operative to provide a blended image from the one camera and the adjacent camera in which the obstructed view portion from the image of the adjacent camera is removed and replaced with an overlapping view portion from the one camera.

51. The panoramic imaging system of claim 47, wherein the processor comprises a plurality of devices in communication with a frame buffer operative to process image frames sequentially from a camera input device to an image transmission device.