Distributed video sensor panoramic imaging system

• US 10,375,355 B2
• Filed: 12/01/2011
• Issued: 08/06/2019
• Est. Priority Date: 11/16/2006
• Status: Active Grant

• Current Assignee: Immersive Licensing, Inc.
• Original Assignee: Immersive Licensing, Inc.
• Inventors: Park, Michael C., White, Patrick W., Sandven, Steven C.
• Primary Examiner(s): Nguyen, Minh Chau
• Application Number:
  US13/309,414
• Publication Number:
  US 20120075414A1
• Time in Patent Office: 2,805 Days
• Field of Search: 348 36, 348 48
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A panoramic imaging system includes a plurality of separated video cameras that may be distributed around an object. A series of images captured by at least one of the separated video cameras is stored in a predetermined file format. The panoramic system further includes a viewer module that may render the series of images using the first file format. Moreover, the panoramic system includes a calibration module capable of modifying information associated with at least one of the series of images, where the modification results in calibration data stored in a configuration file and/or in the predetermined file format. The viewer module may also be capable of rendering the series of images in accordance with the predetermined file format.

Claims

1. A panoramic imaging system, comprising:

a plurality of video cameras, each of the plurality of video cameras captures images, wherein the plurality of video cameras is separately and unevenly distributed on an object to have an unobstructed view of a region exterior to the object; and

a capture module;

converts the captured images from each of the plurality of video cameras into one or more image frames, each converted image frame comprising;

image data comprising at least portions of the captured images from each of the plurality of video cameras; and

calibration data based at least in part on the uneven distribution of the video cameras on the object, and corresponding to the image data, the calibration data specifying how to display a panoramic image based on the image data comprising at least portions of the captured images from each of the plurality of video cameras;

transmits the one or more converted image frames to a viewer module, the viewer module configured to display the panoramic image based on the calibration data and the image data comprising at least portions of the captured images from each of the plurality of video cameras;

wherein the plurality of video cameras is unevenly distributed on the object to exclude an obstruction of the object from the field of view of the plurality of video cameras.

2. The panoramic imaging system of claim 1, wherein the plurality of video cameras comprises a first set of cameras containing at least one of a first type of camera and a second set of cameras containing at least one of a second type of camera.

3. The panoramic imaging system of claim 2, wherein the first type of camera captures a different portion of the electromagnetic spectrum than the second type of camera.

4. The panoramic imaging system of claim 2, wherein a region viewed by the first set of cameras overlaps at least in part with a region viewed by the second set of cameras.

5. The panoramic imaging system of claim 1, further comprising, a calibration module modifies the calibration data contained in the image frames.

6. The panoramic imaging system of claim 5, wherein the calibration module is configured to modify the calibration data to generate second calibration data in response to a command provided by a user.

7. The panoramic imaging system of claim 1, wherein the one or more image frames further comprise data indicative of a location where the image data was captured.

8. The panoramic imaging system of claim 1, wherein the capture module is operatively connected to the plurality of video cameras via at least a wireless connection.

9. The panoramic imaging system of claim 1, wherein the capture module transmits the one or more image frames to the viewer module via at least a wireless connection.

10. The panoramic imaging system of claim 1, further comprising a data store stores the one or more image frames on a non-transitory storage medium, the data store operatively connected to the capture module.

11. The panoramic imaging system of claim 1, wherein, the calibration data comprise information associated with the image data, the information may be one or more of image alignment data, color balance data, location data, time data, identification data, and user metadata.

12. The panoramic imaging system of claim 1, wherein the calibration data comprise one or more of a height of one or more of the captured images, a field of view of one or more of the captured images, a horizontal axis of one or more of the captured images, a vertical axis of one or more of the captured images, and an optical parameter of one or more of the captured images.

13. The panoramic imaging system of claim 1, wherein the object is a vehicle.

14. A method of capturing panoramic images, comprising:

capturing images with each of a plurality of video cameras, wherein the plurality of video cameras is separately and unevenly distributed on an object to have an unobstructed view of a region exterior to the object;

at a capture module, receiving the captured images;

at the capture module, generating an image frame comprising;

image data comprising at least portions of the captured images from each of the plurality of video cameras; and

calibration data based at least in part on the uneven distribution of the video cameras on the object, and corresponding to the image data, the calibration data specifying how to display a panoramic image based on the image data comprising at least portions of the captured images from each of the plurality of video cameras; and

transmitting the image frame to a viewer module, the viewer module configured to display the panoramic image based on the calibration data and the image data comprising at least portions of the captured images from each of the plurality of video cameras;

wherein the plurality of video cameras is unevenly distributed on the object to exclude an obstruction of the object from the field of view of the plurality of video cameras.

15. The method of claim 14, wherein capturing images with a plurality of video cameras comprises capturing at least one image with at least one of a first type of camera and capturing at least one image with at least one of a second type of camera.

16. The method of claim 15, wherein the first type of camera captures a different portion of the electromagnetic spectrum than the second type of camera.

17. The method of claim 14, wherein capturing images with the plurality of cameras comprises capturing one image at a higher resolution than at least one other image.

18. The method of claim 17, wherein generating the image frame comprises integrating the higher resolution image into the image frame.

19. The method of claim 14, further comprising:

at the capture module, receiving location data associated with at least one of the captured images,

wherein the image frame further comprises the received location data.

20. The method of claim 14, further comprising:

responsive to generating the image frame, transmitting the image frame to a data store for storing image frames; and

storing the image frame in the non-transitory storage medium.

21. The method of claim 14, wherein transmitting the image frame to the viewer module comprises wirelessly transmitting the image frame to a remote location.

22. A method of visually rendering a panoramic image, comprising:

receiving one or more image frames, each image frame comprising;

image data representing at least portions of a plurality of images, each image from among the plurality of images captured by a corresponding one of a plurality of video cameras, the plurality of video cameras separately and unevenly distributed on an object to have an unobstructed view of a region exterior to the object, and

calibration data based at least in part on the uneven distribution of the video cameras on the object, and corresponding to the image data, the calibration data specifying how to display a panoramic image based on the image data representing at least portions of the plurality of images captured by the corresponding plurality of video cameras;

displaying the panoramic image based on the calibration data and the image data representing at least portions of the plurality of images captured by the corresponding plurality of video cameras;

wherein the plurality of video cameras is unevenly distributed on the object to exclude an obstruction of the object from the field of view of the plurality of video cameras.

23. The method of claim 22, wherein the image frame comprises image data representing a first captured image and a second captured image, the second image having a higher resolution than the first image and depicting part of a region covered by the first image.

24. The method of claim 23, wherein visually rendering the image data comprises rendering the second image at a position within the first image corresponding to the region depicted by the second image.

25. The method of claim 22, wherein the at least one image frame is received from a data store.

26. The method of claim 22, wherein the at least one image frame is received from a module configured to generate the at least one image frame.

27. The method of claim 22, wherein displaying the image data is performed in real-time with capturing of the one or more images by the plurality of video cameras.

28. A non-transitory computer-readable medium having instructions stored therein, the instructions when executable by a processor cause the processor to:

receive one or more image frames, each image frame comprising;

image data representing at least portions of a plurality of images, each image from among the plurality of images captured by a corresponding one of a plurality of video cameras, the plurality of video cameras separately and unevenly distributed on an object to have an unobstructed view of a region exterior to the object; and

calibration data based at least in part on the uneven distribution of the video cameras on the object, and corresponding to the image data, the calibration data specifying how to display a panoramic image based on the image data representing at least portions of the plurality of images captured by the corresponding plurality of video cameras; and

display the panoramic image based on the calibration data and the image data representing at least portions of the plurality of images captured by the corresponding plurality of video cameras;

wherein the plurality of video cameras is unevenly distributed on the object to exclude an obstruction of the object from the field of view of the plurality of video cameras.

29. The medium of claim 28, wherein the image frame comprises image data representing a first captured image and a second captured image, the second image having a higher resolution than the first image and depicting part of a region covered by the first image.

30. The medium of claim 29, wherein visually rendering the image data comprises rendering the second image at a position within the first image corresponding to the region depicted by the second image.

31. The medium of claim 28, wherein visually rendering the image data is performed in real-time with capturing of the one or more images by the plurality of cameras.

32. A non-transitory computer-readable medium having instructions stored therein, the instructions when executable by a processor cause the processor to:

receive images captured by each of a plurality of cameras, wherein the plurality of cameras is separately and unevenly distributed on an object to have an unobstructed view of a region exterior to the object;

generate an image frame comprising;

image data comprising at least portions of the captured images from each of the plurality of video cameras; and

calibration data based at least in part on the uneven distribution of the video cameras on the object, and corresponding to the image data, the calibration data specifying how to display a panoramic image based on the image data comprising at least portions of the captured images from each of the plurality of video cameras; and

transmit the image frame to a viewer module, the viewer module configured to display the panoramic image based on the calibration data and the image data comprising at least portions of the captured images from each of the plurality of video cameras;

wherein the plurality of video cameras is unevenly distributed on the object to exclude an obstruction of the object from the field of view of the plurality of video cameras.

33. The medium of claim 32, further comprising instructions causing the processor to receive location data associated with at least one of the captured images, wherein the image frame further comprises the received location data.

34. The medium of claim 32, wherein transmitting the image frame to the viewer module comprises transmitting the image frame to a remote location.