Method and device for modifying a zone in successive images

• US 5,515,485 A
• Filed: 09/20/1994
• Issued: 05/07/1996
• Est. Priority Date: 04/11/1990
• Status: Expired due to Term

• Current Assignee: Evs International (Swiss) Sàrl
• Original Assignee: Symah Vision, SA
• Inventors: Rebuffet, Michel, Luquet, Andre
• Primary Examiner(s): Herndon, Heather R.
• Assistant Examiner(s): Roberts, Paul A.
• Application Number:
  US08/308,912
• Time in Patent Office: 595 Days
• Field of Search: 395/135, 395/127, 395/138, 395/139, 348/578, 348/580
• US Class Current: 345/632
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence
• CPC Class Codes: Y10S 345/96 Iterative display of preconfigured images

Abstract

A process modification of the content of a sequence of images which can be represented as a matrix of pixels. For that purpose a non-deformable target zone to be replaced in the scene is identified and is characterized, both in position and in dimensions, by reference to marks located close to each other and connected to the target zone in each image. The position with respect to the marks to be assigned to a model which can be superimposed on the target zone, stored in digital form, is determined. For each particular image, the geometrical transformation to be carried out on the model so that it corresponds to variations in the appearance of the target zone in all the successive images in the same shot is determined automatically using a correlation calculation based on the marks. Data suitable for enabling replacement of each current representation of the target are derived from the transformation and output. Apparatus for carrying out the method is also described.

Claims

1. Method for enabling replacement of a current representation of a target in each image of a sequence of successive video images of a scene delivered by a video camera, with a stored pattern, independent of the size of the target representation, said method comprising the steps of:

(a) storing data identifying said target as set of a plurality of distributed characterizing points and a full representation of said target;

(b) automatically recognizing at least a subset of said set of characterizing points in each current image in turn in the sequence and deriving an estimated location, size and perspective of the current representation of said target in said current image from the relative locations of the characterizing points in said current image;

(c) geometrically transforming said full representation of said target responsive to said estimated size and perspective;

(d) determining, by pixel-by-pixel comparison, which part of the geometrically transformed representation of the target is identical to a part of the current representation of said target as it appears in said current image; and

(e) generating data necessary for enabling replacement of each said current representation of the target by a matching representation of said stored pattern as a result of the transformation and comparison.

2. Method according to claim 1, wherein step (a) comprises storing characterizing points distributed out of said target and within said target.

3. Method according to claim 1, wherein said characterizing points are stored as neighborhoods of pixels.

4. Method for modifying the content of a sequence of images of a same shot, representing an evolutionary scene and having a high degree of correlation between successive images of said same shot, the images being represented as a matrix arrangement of pixels, said method comprising the steps of:

(a) identifying a non-deformable target zone to be replaced in the scene by a set of reference marks distributed within said target zone;

(b) in each current one of said images, searching for said current image and recognizing said set of reference marks in said current image by a pattern recognition process;

(c) identifying the position and dimensions of said target zone in said current image by referring to said reference marks;

(d) determining a position, with respect to said reference marks, to give to a pattern superimposable on the target zone, stored in digital form as pixels;

(e) for each said current image of said sequence, automatically computing a geometrical transformation to be undergone by the pattern to make it match the variable appearance of the target zone by a correlation based on the reference marks, in all successive images of the same shot; and

(f) generating results of steps (d) and (e) as data enabling the substitution of the pixels representative of the pattern in the image for the pixels representative of the target zone.