Stabilizing estimate of location of target region inferred from tracked multiple landmark regions of a video image

• US 5,488,675 A
• Filed: 03/31/1994
• Issued: 01/30/1996
• Est. Priority Date: 03/31/1994
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: David Sarnoff Research Center, Inc.
• Inventors: Hanna, Keith J.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): ANDERSON, DAVID
• Application Number:
  US08/222,207
• Time in Patent Office: 670 Days
• Field of Search: 382/1, 382/22, 382/57, 382/41, 382/103, 382/284, 382/291, 382/298, 348/578, 348/580, 348/584, 348/589, 348/590, 348/593, 395/118, 395/133, 395/127, 395/135
• US Class Current: 382/284
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Any of multiple landmark regions of a scene being viewed by an image sensor are used for interring the location of a target region of the scene in which a pattern is to be inserted in order to avoid the problem of the view of a region being occluded. Because of changes, such as the pointing angle of the sensor, there is a slight difference in size and position in the location of a target region inferred from one landmark region and from another landmark region. this results in a jitter when a first landmark region being used for inferring the location of a target region is replaced by a second landmark region in response to the first landmark region being occluded. This jitter is eliminated by deriving an error-correcting component that compensates for the aforesaid slight difference in size and position in the location of a target region inferred from different landmark regions.

Claims

1. In an image processing method for inserting a given pattern at a target region having a particular location with respect to a scene being viewed by an image sensor which produces a series of successive image frames, wherein said scene includes at least two landmark regions displaced in position from one another;

and wherein said method comprises the computation steps of (a) inferring the size and position of said particular location within each image frame of said scene from the size and position of a first one of said landmark regions represented within each image frame of said scene and (b) independently inferring the size and position of said particular location within each image frame of said scene from the size and position of a second one of said landmark regions represented within each image frame of said scene, whereby there may be a difference in the size and position of said particular location within each image frame of said scene inferred from the size and position of said second one of said landmark regions with respect to the size and position of said particular location within each image frame of said scene inferred from the size and position of said first one of said landmark regions;

the improvement comprising the step of;

c) modifying the size and position of the particular location within one of said image frames of said scene inferred from the size and position of said second one of said landmark regions within that one of said image frames so that it is substantially the same as the size and position of the particular location within that one of said image frames of said scene inferred from the size and position of said first one of said landmark regions.

2. The method of claim 1, wherein:

step (c) comprises the steps of;

d) employing a least squares solution of a given error function of zoom, horizontal position translation, and vertical position translation to compute said difference in the, size and position of said particular location in said one of said image frames of said scene; and

e) adding said computed difference in the size and position of said particular location in said one of said image frames of said scene to the particular location within that one of said image frames of said scene inferred from the size and position of said second one of said landmark regions thereby to modify the size and position of the particular location within that one of said image frames of said scene inferred from the size and position of said second one of said landmark regions so that it is substantially the same as the size and position of the particular location within that one of said image frames of said scene inferred from the size and position of said first one of said landmark regions

3. The method of claim 2, comprising the steps of:

e) initially inserting said pattern in a predetermined location of each of a set of one or more successive image frames inferred by step (a);

f) during the occurrence an image frame of said first set, employing step (b) to infer the predetermined location and then employing step (d) to compute said difference;

g) storing said difference; and

h) thereafter, during one image frame occurring subsequent to said first set, inserting said pattern in a modified predetermined location of said one image frame computed by adding said stored difference to the predetermined location inferred by step (b).

4. The method of claim 2, comprising the steps of.

e) recording a plurality of images of said scene each at a different zoom and/or translation position to derive a "world map" of said scene;

f) for each one of said plurality of said recorded images, employing step (a), (b) and (d) to individually derive said difference for that one of said plurality of said recorded images;

g) for each one of said plurality of said recorded images, add said difference for that recorded image to said predetermined location inferred by step (b) to derive a corrected predetermined location for that one of said plurality of said recorded images;

h) store the corrected predetermined location for each one of said plurality of said recorded images in association therewith;

i) compare the current image frame with said recorded images to determine which recorded image most nearly corresponds to the current image frame; and

j) employ the stored corrected predetermined location of said recorded image that most nearly corresponds to the current image frame to insert said pattern at a location of the current image frame that corresponds to said stored corrected predetermined location.

5. The method of claim 4, wherein:

said first one of said landmark regions in each of said plurality of said recorded images is located substantially at said predetermined location of said target region.