Method and apparatus for providing virtual processing effects for wide-angle video images

• US 6,778,211 B1
• Filed: 04/10/2000
• Issued: 08/17/2004
• Est. Priority Date: 04/08/1999
• Status: Expired due to Term

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: IPIX Corporation
• Inventors: Zimmermann, Steven Dwain, Gourley, Christopher Shannon
• Primary Examiner(s): Christensen, Andrew
• Assistant Examiner(s): TRAN, NHAN T
• Application Number:
  US09/546,659
• Time in Patent Office: 1,590 Days
• Field of Search: 348/218.1, 348/36-39, 348/246, 348/143, 348/207.99, 358/450, 382/294, 382/300, 345/629-634, 345/427, 345/8
• US Class Current: 348/218.1
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method and apparatus is disclosed for providing special effects for wide-angle video images. The special effects include replacing a selected portion of a wide-angle video image with a predetermined image or an associated portion of a video image captured in a subsequent frame. Other special effects include image intensity scaling for wide-angle video images.

Claims

1. A method of creating video images, comprising:

a) capturing an immersive video of an environment with a video image capture system, the image capture system comprising one or more video cameras and a structure supporting the one or more cameras, the immersive video having a spherical field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video whereby the image of at least a portion of the image capture system is replaced with other image data to represent an unobstructed view of the environment.

2. The method of claim 1, wherein the image capture system comprises back-to-back fisheye lenses.

3. The method of claim 1, wherein the other image data is captured with the image capture system.

4. The method of claim 3, wherein the other image data originates from one or more other frames in the immersive video.

5. The method of claim 1, wherein the image of at least a portion of the image capture system is an image of the supporting structure.

6. The method of claim 1, wherein the image capture system is moving relative to the environment.

7. The method of claim 6, wherein said structure is a vehicle.

8. The method of claim 1, wherein the image of at least a portion of the image capture system has a shape, wherein said other image data has substantially the same shape as the image of at least a portion of the image capture system.

9. The method of claim 1, wherein the step of modifying includes modifying the other image data.

10. The method of claim 9, wherein the image of at least a portion of the image capture system has a curvature, and wherein the step of modifying includes warping the image represented by said other image data to substantially the same curvature as said image of at least a portion of the image capture system.

11. The method of claim 10, wherein said other image data represents a planar image prior to warping.

12. The method of claim 9, wherein the step of modifying the other image data includes preprocessing the other image data.

13. The method of claim 1, further comprising the step of recording data representing the positioning of the image capture system while capturing the immersive video.

14. A system operable to carry out the steps comprising the method of claim 1.

15. A computer readable medium comprising instructions to perform the method of claim 1.

16. A method of creating video images, comprising:

a) capturing an immersive video of an environment with a video image capture system, the image capture system comprising one or more video cameras and a structure supporting one or more cameras, the immersive video having a spherical field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video whereby the image of at least a portion of the image capture system is replaced with other image data to represent an unobstructed view of the environment, wherein the other image data is captured with the image capture system and originates from one or more other frames in the immersive video.

17. A method of creating video images, comprising:

a) capturing an immersive video of an environment with a video image capture system, wherein the video image capture system is moving relative to the environment, the image capture system comprising one or more video cameras and a structure supporting the one or more cameras, the immersive video having a spherical field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video whereby the image of at least a portion of the image capture system is replaced with other image data to represent an unobstructed view of the environment.