Method and device for modifying a zone in successive images
First Claim
1. Device for modifying the content of images of a sequence of images having a high degree of correlation between successive images, comprising picture forming means delivering each image as of a matrix of pixels, the radiometric values (luminance and possibly chrominance) of each pixel being in digital form, characterized in that it further comprises:
- means for storing reference marks for locating a target area in the image and representations of the target area at a plurality of scales and for storing representations, at the same scales, of a pattern which can be superimposed on the target area,means for recognizing the reference marks in the current image and deriving therefrom the position and dimensions of the target area, andmeans for automatically determining, for each image, by spatial correlation calculation based on the reference marks, a geometrical transformation to which the pattern should be subjected to correspond to the variations of appearance of the target area, in all successive images of a same shot and for substituting, in the image, the pixels representive of the pattern for the pixels representative of the target area.
6 Assignments
0 Petitions
Accused Products
Abstract
The process makes it possible to modify the content of a sequence of images which can be represented as a pixel matrix. For that a non-deformable target zone to be replaced in the images is identified and is characterized, both in position and in dimensions, by reference to marks located close to each other and having an unchanging geometrical relation with the target zone in each image. The position, with respect to the marks, to be assigned to a model or pattern which can be superimposed on the target zone, stored in digital form, is determined. For each particular image of a same shot the geometrical transformation to be carried out on the model so that it corresponds to variations in the appearance of the target zone in all the successive images in the same shot is determined automatically by a correlation based on the marks. Last the pixels representative of the model are substituted for the pixels representative of the target zone in the image.
202 Citations
43 Claims
-
1. Device for modifying the content of images of a sequence of images having a high degree of correlation between successive images, comprising picture forming means delivering each image as of a matrix of pixels, the radiometric values (luminance and possibly chrominance) of each pixel being in digital form, characterized in that it further comprises:
-
means for storing reference marks for locating a target area in the image and representations of the target area at a plurality of scales and for storing representations, at the same scales, of a pattern which can be superimposed on the target area, means for recognizing the reference marks in the current image and deriving therefrom the position and dimensions of the target area, and means for automatically determining, for each image, by spatial correlation calculation based on the reference marks, a geometrical transformation to which the pattern should be subjected to correspond to the variations of appearance of the target area, in all successive images of a same shot and for substituting, in the image, the pixels representive of the pattern for the pixels representative of the target area.
-
-
2. Method for modifying the content of a sequence of images of the same shot, representing an evolutionary scene and having a high degree of correlation between successive images of said same shot, the images being represented as a material arrangement of pixels, comprising the steps of:
-
(a) identifying a non-deformable target zone to be replaced in the scene by a set of reference marks distributed within said target zone; (b) in each current one of said images, searching for said current image and recognizing said set of reference marks in said current image by pattern recognition process; (c) identifying the position and dimensions of said target zone in said current image by referring to said reference marks; (d) determining a position, with respect to said reference marks, to give to a pattern superimposable on the target zone, stored in digital form as pixels; (e) for each said current image of said sequence, automatically computing a geometrical transformation to be undergone by the pattern to make it match the variable appearance of the target zone by a correlation based on the reference marks, in all successive images of the same shot; and (f) substituting the pixels representative of the pattern in the image for the pixels representative of the target zone. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9)
-
-
10. Method for modifying the content of a sequence of images of the same shot, representing an evolutionary scene and having a high degree of correlation between successive images of said same shot, by substituting a representation of variable appearance of a non-deformable target zone in each said image with a representation of a predetermined pattern, comprising the preliminary steps of:
-
storing a pattern apt to match said target zone; defining said target zone by a set of characterizing data distributed within said target zone and having an unchanging mutual relation to said target zone; storing said set of characterizing data; and
, in each of said successive images of said shot, the steps of;searching said image for and recognizing said set of characterizing data; carrying out a correlation calculation based on said characterizing data to find a position of said pattern and a geometrical transformation to impress to said pattern to obtain a representation matching the variable appearance of the target zone; geometrically transforming said pattern using results of the correlation calculation; and substituting the representation of the target zone in each said successive image with the respective geometrically transformed representation of said pattern.
-
-
11. Method for modifying the content of a sequence of images of a same shot, consisting of a matrix arrangement of pixels, representing an evolutionary scene and having a high degree of correlation between successive images, by substituting a representation of a non-deformable target zone in each said image with a predetermined pattern, said method comprising the steps of:
-
(a) storing, as pixels, a pattern apt to match said representation in digital form; (b) defining said representation of said target zone as a set of characterizing data distributed within said target zone and having an unchanging mutual relation in said target zone; (c) storing a full representation of said target zone and said set of characterizing points; (d) searching each said successive image of said shot for and recognizing said set of characterizing data; (e) for each image of said sequence where said set of characterizing data is recognized, carrying out a correlation calculation based on said characterizing data to find a position to give to said pattern and a geometrical transformation to impress to said pattern to make it match the variable appearance of the target zone; (f) geometrically transforming said pattern using results of the calculation; and (g) substituting the pixels representing the target zone in said image with pixels representing the respective geometrically transformed pattern.
-
-
12. Method for modifying the content of a sequence of images of a same shot, representing an evolutionary scene and having a high degree of correlation between successive images of said same shot, by substituting a non-deformable target in each said image with a predetermined pattern, said method comprising the steps of:
-
(a) storing a plurality of representations, at different scale, of a pattern apt to match said target; (b) storing a plurality of representation of said target at the same scales; (c) defining each said representation of said target as a set of corresponding characterizing points distributed within said target and having an unchanging mutual relation in said target; (d) storing said sets of characterizing points; and
, in each successive image of said shot,(e) searching for said sets of characterizing points in each said image; (f) carrying out a correlation calculation based on said reference marks to find an optimum one among said representations of said target zone, and a geometrical transformation to impress to the corresponding optimum representation of said pattern to make it match the variable appearance of the target zone; (g) geometrically transforming said optimum representation of said pattern using results of the correlation calculation; and (h) substituting the representation of the target zone with the respective geometrically transformed pattern representation.
-
-
13. Device for modifying the content of video images of a sequence of video images having a high degree of correlation between successive images, each said image being represented by a matrix of pixels, said device comprising:
-
first means for storing a set of reference marks identifying a non-deformable target zone present in some at least of the video images of said sequence and for storing a plurality of representations of said target zone at a plurality of scales, said reference marks having a fixed relation with and being close to said target zone; second means for storing a plurality of representations, at the same scales as said representations of the target zone, of a pattern having a shape enabling to superimpose it on the target zone; third means for searching for and recognizing said set of reference marks in a current image of said sequence and deriving therefrom the position and dimensions of the target zone, using said representations of said target zone at different scales; fourth means for automatically computing, for each image, by spatial correlation calculation based on the reference marks, a geometrical transformation to which one respective of said representations of the pattern should be subjected to match the target zone in said current image; and fifth means for subjecting said respective representation to said geometrical transformation and for inserting, in the current image, pixels representative of the geometrically transformed pattern for the pixels representative of the target zone. - View Dependent Claims (14)
-
-
15. Device for replacing a representation of a nondeformable target with a matching representation of a predetermined pattern in successive video images of a sequence of images having a high degree of correlation between successive images, said device comprising:
-
means for storing reference marks identifying said target, said reference marks being distributed at least within said target; means for storing at least a representation of said pattern; means for detecting said target in each of said images of said sequence and deriving the position, size and orientation of the representation of the target, based on a search for a best possible match between said stored reference marks and respective reference marks extracted from the images; means for geometrically transforming said representation of said pattern to match the respective representation of the target; and means for substituting, in each said image of said sequence where the target is found, the geometrically transformed representation of said pattern for the representation of the target. - View Dependent Claims (16, 17)
-
-
18. Device for modifying the content of video images of a sequence of images, originating from a TV camera having a variable focal distance and orientation, without sudden time or space disruption, comprising:
-
first means for storing a plurality of representations of a predetermined target present in video images of said sequence at a plurality of scales and for storing a plurality of reference marks identifying said target, said reference marks being distributed within said target; second means for storing a plurality of representations, at the same scales as said target, of a pattern having a shape fit to be superimposed on the target; means for recognizing at least some of said reference marks in a current image of said sequence, deriving therefrom the position and dimensions of edges of the target zone by comparing them to said plurality of representations of said target at different scales and selecting that stored representation of said target which best matches the target found in the image; means for computing, for each image, based on said reference marks, a mathematical model which best represents a geometrical transformation to which the stored representation of the pattern at the same scale as the matching representation of the target should be subjected to match the respective representation of the target, in said current image of said sequence; and means for subjecting the latter cited representation of the target to said geometrical transformation and for substituting, in the current image, the geometrically transformed pattern representation for the target. - View Dependent Claims (19, 20, 21, 22, 23)
-
-
24. Apparatus for replacing a target in a sequence of successive video images of a scene, delivered by a video camera, with a stored pattern, comprising:
-
means for storing a pattern and for storing representation of said target as a set of a plurality of characterizing data distributed in said target; means for automatically recognizing said set of characterizing data in each current image in a sequence of video images of a scene wherein the images have a high degree of correlation between successive images without human intervention and deriving an estimated location, size and perspective of said target in said current image from the relative location of the characterizing points in said current image; means for geometrically transforming said stored pattern responsive to said estimated size and perspective to make said pattern match said target in said current image; and means for inserting said geometrically transformed pattern into said current image in substitution of said target. - View Dependent Claims (25, 26, 27, 28)
-
-
29. Method for replacing a target in a sequence of successive video images of a scene, delivered by a video camera, with a stored pattern, comprising the steps of:
-
(a) storing a representation of said target as a plurality of characterizing points distributed in said image; (b) automatically recognizing, by pattern recognition processes, at least a subset of said set of characterizing points in each current image in the sequence and deriving an estimated location, size and perspective of said target in said current image from the relative location of the characterizing points in said current image; (c) geometrically transforming said stored pattern responsive to said estimated size and perspective to make said transformed pattern match said target in said current image; and (d) inserting said geometrically transformed pattern into said current image in substitution of said target. - View Dependent Claims (30, 31, 32, 33, 34, 35, 36, 37, 38)
-
-
39. Method for replacing a target in a sequence of successive video images of a scene, delivered by a video camera, with a stored pattern, comprising the steps of:
-
(a) storing a representation of said target as a plurality of distributed characterizing points image and a full representation of said target; (b) automatically recognizing at least a subset of said set of characterizing points in each current image in the sequence and deriving an estimated location, size and perspective of said target in said current image from the relative locations of the characterizing points in said current image; (c) geometrically transforming said representation of said target responsive to said estimated size and perspective; (d) determining, by pixel-by-pixel comparison, which part of the geometrically transformed representation of the target is identical to a part of the target as it appears in said current image; (e) geometrically transforming said stored pattern responsive to said estimated size and perspective to make said transformed pattern match said target in said current and said geometrically transformed pattern into said current image in substitution of said target; (f) substituting only a part of the target, for which, during step (d), identity was found between the current image and the transformed representation of the target, with a matching part of said geometrically transformed pattern. - View Dependent Claims (40, 41, 42, 43)
-
Specification