Video object tracking by estimating and subtracting background
First Claim
1. A method for tracking an object among a plurality of image frames, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the method comprising the steps of:
- defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being within observable portions of the background for the given image frame;
deriving a background model from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames;
classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame;
defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and
identifying the object as being the pixels within the foreground mask for the current image frame.
1 Assignment
0 Petitions
Accused Products
Abstract
An object is tracked among a plurality of image frames. In an initial frame an operator selects an object. The object is distinguished from the remaining background portion of the image to yield a background and a foreground. A model of the background is used and updated in subsequent frames. A model of the foreground is used and updated in the subsequent frames. Pixels in subsequent frames are classified as belonging to the background or the foreground. In subsequent frames, decisions are made, including: which pixels do not belong to the background; which pixels in the foreground are to be updated; which pixels in the background were observed incorrectly in the current frame; and which background pixels are being observed for the first time. In addition, mask filtering is performed to correct errors, eliminate small islands and maintain spatial and temporal coherency of a foreground mask.
84 Citations
27 Claims
-
1. A method for tracking an object among a plurality of image frames, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the method comprising the steps of:
-
defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being within observable portions of the background for the given image frame;
deriving a background model from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames;
classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame;
defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and
identifying the object as being the pixels within the foreground mask for the current image frame. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the apparatus comprising:
-
a background model derived from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames;
a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on a current state of the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and
a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A method for tracking an object among a plurality of image frames, the method comprising the steps of:
-
defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame;
maintaining a background model of background pixels which have been observable in at least three consecutive image frames by predicting, for each one pixel of a current image frame, a background value for said one pixel based upon a predicted background value of said one pixel from the prior image frame, a pixel value of said one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame;
classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame;
defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and
identifying the object as being the pixels within the foreground mask for the current image frame. - View Dependent Claims (16, 17, 18)
-
-
19. A method for tracking an object among a plurality of image frames, the method comprising the steps of:
-
defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame;
maintaining a background model of background pixels which have been observable in at least three consecutive image frames;
classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame;
defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and
identifying the object as being the pixels within the foreground mask for the current image frame;
wherein the step of classifying comprises identifying an event from the group of events comprising;
prospectively include said one pixel in the background mask for the current image frame;
motion is detected in the immediate past for said one pixel;
motion is detected in the immediate future for said one pixel;
prospectively include said one pixel in the foreground pixel for the current image frame; and
said one pixel is not to be updated in the background model. - View Dependent Claims (20, 21)
-
-
22. A method for tracking an object among a plurality of image frames, the method comprising the steps of:
-
defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame;
maintaining a background model of background pixels which have been observable in at least three consecutive image frames;
classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame;
defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and
identifying the object as being the pixels within the foreground mask for the current image frame; and
wherein the step of classifying comprises identifying an event from the group of events comprising;
prospectively include said one pixel in the background mask for the current image frame;
a first degree of motion is detected in the immediate past for said one pixel;
the first degree of motion is detected in the immediate future for said one pixel;
a second degree of motion is detected in the immediate past for said one pixel;
the second degree of motion is detected in the immediate future for said one pixel;
prospectively include said one pixel in the foreground pixel for the current image frame; and
said one pixel is not to be updated in the background model. - View Dependent Claims (23, 24)
-
-
25. An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising:
-
a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames;
a first processor which predicts a background value for said each one pixel based upon a predicted background value of said each one pixel from the prior image frame, a pixel value of said each one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame;
a second processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and
a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame.
-
-
26. An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising:
-
a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames;
a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and
a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame;
wherein the processor identifies an event from the group of events comprising;
prospectively include said one pixel in the background mask for the current image frame;
motion is detected in the immediate past for said one pixel;
motion is detected in the immediate future for said one pixel;
prospectively include said one pixel in the foreground pixel for the current image frame; and
said one pixel is not to be updated in the background model.
-
-
27. An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising:
-
a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames;
a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and
a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame;
wherein the processor identifies an event from the group of events comprising;
prospectively include said one pixel in the background mask for the current image frame;
a first degree of motion is detected in the immediate past for said one pixel;
the first degree of motion is detected in the immediate future for said one pixel;
a second degree of motion is detected in the immediate past for said one pixel;
the second degree of motion is detected in the immediate future for said one pixel;
prospectively include said one pixel in the foreground pixel for the current image frame; and
said one pixel is not to be updated in the background model.
-
Specification