Method and apparatus for real time insertion of images into video
First Claim
1. A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising:
- receiving a stream of original images taken by a camera of a real, three-dimensional site;
for each original image in the image stream,identifying a background area within the original image from at least one target area in which a target image will be inserted;
rendering, for each at least one target area, a target image based on at least a productions three-dimensional model of the at least one target area within the site and the camera'"'"'s position and pointing direction, wherein the target area is comprised of a virtual surface that does nor exist within the site;
identifying occlusions within the at least one target area of the original image; and
combining the background of the original image, the at least one target image, and the image of the occlusions into an output image.
5 Assignments
0 Petitions
Accused Products
Abstract
The present invention comprises a real time image insertion system for video utilizes a three-dimensional model of at least one target area within a site for identifying the target area within an original video image of the site, and for rendering a target image and inserting it into the target area of the video image. The target area may be a real, existing surface of the site or an imaginary surface defined only in the model. The model is rendered from the position and angle of a camera that generated the video image. By using a three dimensional model of the site to generate the target image, the resulting image with the inserted target material appears more realistic.
-
Citations
29 Claims
-
1. A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising:
-
receiving a stream of original images taken by a camera of a real, three-dimensional site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a productions three-dimensional model of the at least one target area within the site and the camera'"'"'s position and pointing direction, wherein the target area is comprised of a virtual surface that does nor exist within the site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising:
-
a model renderer for generating, a synthetic image based on a predefined three-dimensional reference model of a target area with a site from a known position of a camera, the target area including a virtual surface that does not exist within the site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image. - View Dependent Claims (13)
-
-
14. A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising:
-
receiving a stream of original images taken by a camera of a real site; for each original image in the image stream; identifying a background area within the original image from at least one target area in which a target image will be inserted by identifying at least one target area within the image based on a predefined model of selected surfaces of the site and the camera'"'"'s position and pointing direction, the target area including a virtual surface that does not exist within the site, wherein identifying at least one target area within the image includes; receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera; rendering the predefined model from the camera'"'"'s position, based on the camera parameters; and generating from the rendering a mask for defining the target area within the image; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera'"'"'s position and pointing direction, the camera'"'"'s position being predefined within the model, wherein rendering includes receiving camera parameters, the camera parameters including parameters indicating the pointing direction of the camera, and rendering the predefined model based at least in part on the camera parameters; identifying occlusions within the at least one target area of the original image; combining the background of the original image, the at least one target image, and the image of the occlusions into an output image. - View Dependent Claims (15, 16)
-
-
17. A method for inserting a target image into an image stream of a real site taken by a camera, the method comprising:
-
receiving a stream of original images taken by a camera of a real site; for each original image in the image stream, identifying a background area within the original image from at least one target area in which a target image will be inserted; rendering, for each at least one target area, a target image based on at least a predefined three-dimensional model of the at least one target area within the site and the camera'"'"'s position and pointing direction, the three-dimensional model being of less than the entire site; identifying occlusions within the at least one target area of the original image; and combining the background of the original image, the at least one target image, and the image of the occlusions into an output image. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. An apparatus for inserting a target image into video of a real site taken by a camera, the apparatus comprising:
-
a model renderer for generating a synthetic image based on a predefined three-dimensional reference model including a target area within a site from a known position of a camera, the three-dimensional model being of less than the entire site, the synthetic image having a target image inserted in the target area; an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image; and an image combiner for combining the masked background image with the target image. - View Dependent Claims (29)
-
Specification