Method and apparatus for providing virtual processing effects for wide-angle video images

• US 7,312,820 B2
• Filed: 07/26/2004
• Issued: 12/25/2007
• Est. Priority Date: 04/08/1999
• Status: Expired due to Term

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: IPIX Corporation
• Inventors: Zimmermann, Steven Dwain, Gourley, Christopher Shannon
• Primary Examiner(s): Ometz; David
• Assistant Examiner(s): TRAN, NHAN T
• Application Number:
  US10/898,865
• Publication Number:
  US 20050007483A1
• Time in Patent Office: 1,247 Days
• Field of Search: 348/39, 348/218.1, 348143-153, 345629-634
• US Class Current: 348/218.1
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method and apparatus is disclosed for providing special effects for wide-angle video images. The special effects include replacing a selected portion of a wide-angle video image with a predetermined image or an associated portion of a video image captured in a subsequent frame. Other special effects include image intensity scaling for wide-angle video images.

Claims

1. A method of processing video images, comprising:

a) receiving first image data representing an immersive video of an environment, the immersive video having been captured with a video image capture system, the immersive video having a wide-angle field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs a portion of the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video, the step of modifying comprising the step of replacing data representing the image of at least a portion of the image capture system with other image data representing an unobstructed view of at least said obstructed portion to produce a processed immersive video having an unobstructed immersive image.

2. The method of claim 1, wherein the first image data is received via wireless transmission.

3. The method of claim 1, wherein the first image data is received via transmission from said image capture system.

4. The method of claim 3, wherein the transmission of the first image data and the capture of the immersive video are substantially contemporaneous.

5. The method of claim 3, wherein the step of receiving the first image data via transmission from said image capture system includes downloading said first image data from a memory device in the image capture system.

6. The method of claim 1, wherein the first image data is received stored in a tangible computer readable medium.

7. The method of claim 1, wherein said wide-angle field of view is a spherical field of view.

8. The method of claim 1, the other image data having been captured with the image capture system.

9. The method of claim 8, wherein the other image data originates from one or more frames in the immersive video.

10. The method of claim 1, wherein the step of modifying includes modifying the other image data.

11. The method of claim 10, wherein the image of at least a portion of the image capture system has a curvature, and wherein the step of modifying includes warping the image represented by said other image data to substantially the same curvature as said image of at least a portion of the image capture system.

12. The method of claim 11, wherein said other image data represents a planar image prior to warping.

13. The method of claim 1, further comprising the step of:

c) modifying at least one of at least a portion of the first image data and at least a portion of the other image data to balance image intensity about the unobstructed image in the processed immersive video.

14. The method of claim 1, the step of modifying including removing the image of at least a portion of the image capture system from at least one of said frames to leave a remainder image, wherein said other image data is derived from said remainder image.

15. A system operable to carry out the steps comprising the method of claim 1.

16. A computer readable medium comprising instructions executable by a computer to perform the method of claim 1.

17. A method of processing video images, comprising:

a) receiving first image data representing an immersive video of an environment, the immersive video having been captured with a video image capture system, the immersive video having a wide-angle field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video, the step of modifying comprising the step of superimposing an image represented by other image data over the image of at least a portion of the image capture system to represent an unobstructed view of the environment.

18. A method of processing video images, comprising:

a) acquiring a plurality of video images of an environment, said video images having been captured by a video capture system, wherein at least one of said video images includes an image of at least a portion of said video capture system obstructing at least a portion of said environment, wherein said video images comprise a plurality of frames, at least two of said frames representing substantially contemporaneous images captured at substantially the same moment in time;

b) seaming together at least two of said substantially contemporaneous images to create a seamed immersive image of said environment at said moment in time, said seamed immersive image including an image of at least a portion of said video capture system obstructing at least a portion of said environment; and

c) modifying said seamed immersive image to represent an unobstructed immersive view of said environment.

19. A method of processing video images, comprising:

a) acquiring a plurality of video images of an environment, said video images having been captured by a video capture system, wherein at least one of said video images includes an obstructed image, said obstructed image comprising an image of at least a portion of said video capture system obstructing at least a portion of said environment, wherein said video images comprise a plurality of frames, at least two of said frames representing substantially contemporaneous images captured at substantially the same moment in time;

b) modifying at least one of said obstructed images to represent an unobstructed view of said environment; and

c) seaming together said substantially contemporaneous images to create a seamed immersive image of said environment at said moment in time.

20. A method of processing video images, comprising:

a) acquiring video image data representing an immersive view of an environment, said image data having been captured by an image capture system, wherein the image capture system comprises a support member, wherein said immersive view includes an image of at least a portion of said support member obstructing at least a portion of said environment;

b) assembling said video image data into an immersive video; and

c) modifying image data representing said image of at least a portion of said support member to represent an unobstructed immersive view of said environment.