Method and apparatus for providing virtual processing effects for wide-angle video images

• US 20050007483A1
• Filed: 07/26/2004
• Published: 01/13/2005
• Est. Priority Date: 04/08/1999
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Zimmermann, Steven Dwain, Gourley, Christopher Shannon
• Granted Patent: US 7,312,820 B2
• Time in Patent Office: Days
• US Class Current: 348/335
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method and apparatus is disclosed for providing special effects for wide-angle video images. The special effects include replacing a selected portion of a wide-angle video image with a predetermined image or an associated portion of a video image captured in a subsequent frame. Other special effects include image intensity scaling for wide-angle video images.

Claims

1-28. -28. (canceled)

29. A method of processing video images, comprising:

a) receiving first image data representing an immersive video of an environment, the immersive video having been captured with a video image capture system, the immersive video having a wide-angle field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs an obstructed portion of the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video, the step of modifying comprising the step of replacing data representing the image of at least a portion of the image capture system with other image data representing an unobstructed view of at least said obstructed portion to produce a processed immersive video having an unobstructed immersive image.

30. The method of claim 29, wherein the first image data is received via wireless transmission.

31. The method of claim 29, wherein the first image data is received via transmission from said image capture system.

32. The method of claim 31, wherein the transmission of the first image data and the capture of the immersive video are substantially contemporaneous.

33. The method of claim 31, wherein the step of receiving the first image data via transmission from said image capture system includes downloading said first image data from a memory device in the image capture system.

34. The method of claim 29, wherein the first image data is received stored in a tangible computer readable medium.

35. The method of claim 29, wherein said wide-angle field of view is a spherical field of view.

36. The method of claim 29, the other image data having been captured with the image capture system.

37. The method of claim 36, wherein the other image data originates from one or more frames in the immersive video.

38. The method of claim 29, wherein the step of modifying includes modifying the other image data.

39. The method of claim 38, wherein the image of at least a portion of the image capture system has a curvature, and wherein the step of modifying includes warping the image represented by said other image data to substantially the same curvature as said image of at least a portion of the image capture system.

40. The method of claim 39, wherein said other image data represents a planar image prior to warping.

41. The method of claim 29, further comprising the step of:

c) modifying at least one of at least a portion of the first image data and at least a portion of the other image data to balance image intensity about the unobstructed image in the processed immersive video.

42. The method of claim 29, the step of modifying including removing the image of at least a portion of the image capture system from at least one of said frames to leave a remainder image, wherein said other image data is derived from said remainder image.

43. A system operable to carry out the steps comprising the method of claim 29.

44. A computer readable medium comprising instructions to perform the method of claim 29.

45. A method of processing video images, comprising:

a) receiving first image data representing an immersive video of an environment, the immersive video having been captured with a video image capture system, the immersive video having a wide-angle field of view and a plurality of frames, wherein at least a portion of the image capture system obstructs the view of the environment and one or more frames includes an image of the obstructing portion of the image capture system; and

b) modifying one or more frames of the immersive video, the step of modifying comprising the step of superimposing an image represented by other image data over the image of at least a portion of the image capture system to represent an unobstructed view of the environment.

46. A method of processing video images, comprising:

a) acquiring a plurality of video images of an environment, said video images having been captured by a video capture system, wherein at least one of said video images includes an image of at least a portion of said video capture system obstructing at least a portion of said environment, wherein said video images comprise a plurality of frames, at least two of said frames representing substantially contemporaneous images captured at substantially the same moment in time;

b) seaming together at least two of said substantially contemporaneous images to create a seamed immersive image of said environment at said moment in time, said seamed immersive image including an image of at least a portion of said video capture system obstructing at least a portion of said environment; and

c) modifying said seamed immersive image to represent an unobstructed immersive view of said environment.

47. A method of processing video images, comprising:

a) acquiring a plurality of video images of an environment, said video images having been captured by a video capture system, wherein at least one of said video images includes an obstructed image, said obstructed image comprising an image of at least a portion of said video capture system obstructing at least a portion of said environment, wherein said video images comprise a plurality of frames, at least two of said frames representing substantially contemporaneous images captured at substantially the same moment in time;

b) modifying at least one of said obstructed images to represent an unobstructed view of said environment; and

c) seaming together said substantially contemporaneous images to create a seamed immersive image of said environment at said moment in time.

48. A method of processing video images, comprising:

a) acquiring video image data representing an immersive view of an environment, said image data having been captured by a video capture system, wherein said immersive view includes an image of at least a portion of said image capture system obstructing at least a portion of said environment;

b) assembling said video image data into an immersive video; and

c) modifying image data representing said image of at least a portion of said image capture system to represent an unobstructed immersive view of said environment.