Motion segmentation system with multi-frame hypothesis tracking
First Claim
1. A method comprising:
- identifying a plurality of motion classes for a frame of video data, each motion class having at least one region classified thereto;
determining a region to be a poorly classified region forming a set of class hypotheses for the poorly classified region, the set of class hypotheses containing at least one possible motion class;
determining a similarity measure for the poorly classified region with respect to a set of past and future video data, the similarity measure indicating a degree of similarity or consistency between a hypothesis for the motion of the poorly classified region and the motion of the corresponding regions in past and future frames; and
reclassifying the poorly classified region to one of the at least one possible motion classes according to the similarity measure.
1 Assignment
0 Petitions
Accused Products
Abstract
In one embodiment, a video encoder encodes digital video using motion segmentation. According to one embodiment, motion segmentation is performed using multiple frames. The motion segmentation may operate by determining multiple classification hypotheses and by re-classifying poorly classified regions according to the multi-frame hypothesis tracking algorithm. This involves determining a similarity measure for each hypothesis class, and then assigning a classification to the region with the hypothesis class that is most similar or consistent with past or future data. This hypothesis tracking algorithm for re-classification can handle inherent problems like occlusion and ambiguous motion classification, that plague standard methods. In another embodiment, the motion segmentation algorithm may be integrated into a larger video encoding system. The system may incorporate motion estimation, motion classification, identification and selection of poor blocks, and re-classification of poor blocks using the multi-frame hypothesis tracking algorithm.
64 Citations
21 Claims
-
1. A method comprising:
-
identifying a plurality of motion classes for a frame of video data, each motion class having at least one region classified thereto;
determining a region to be a poorly classified region forming a set of class hypotheses for the poorly classified region, the set of class hypotheses containing at least one possible motion class;
determining a similarity measure for the poorly classified region with respect to a set of past and future video data, the similarity measure indicating a degree of similarity or consistency between a hypothesis for the motion of the poorly classified region and the motion of the corresponding regions in past and future frames; and
reclassifying the poorly classified region to one of the at least one possible motion classes according to the similarity measure. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computer-readable medium comprising computer program instructions which, when executed by a processor, cause the processor to perform they hypothesis algorithm comprising:
-
identifying a plurality of motion classes for a frame of video data, each motion class having at least one region classified thereto;
determining a region to be a poorly classified region forming a set of class hypotheses for the poorly classified region, the set of class hypotheses containing at least one possible motion class;
determining a similarity measure for the poorly classified region with respect to a set of past and future video data, the similarity measure indicating a degree of similarity or consistency between a hypothesis for the motion of the poorly classified region and the motion of the corresponding regions in past and future frames; and
reclassifying the poorly classified region to one of the at least one possible motion classes according to the similarity measure. - View Dependent Claims (10, 11, 12)
-
-
13. A video device comprising:
-
means for forming a set of class hypotheses for a region of a frame of video data;
means for determining a similarity measure for the region with respect to a set of past and future video data; and
means for reclassifying the region according to the similarity measure. - View Dependent Claims (14, 15, 16)
-
-
17. A video device comprising:
a motion compensation component configured to form a set of class hypotheses for a region of a frame of video data, to determine a similarity measure for the region with respect to a set of past and future data, and to reclassify the region according to the similarity measure. - View Dependent Claims (18, 19)
-
20. A method comprising:
-
performing motion estimation for a frame containing a plurality of motion classes, each motion class having at least one region classified thereto;
identifying at least one poorly classified region;
selecting a new motion class for each poorly classified region;
reclassifying the poorly classified region to the new motion class; and
re-estimating the poorly classified region based on a result of reclassifying the poorly classified region. - View Dependent Claims (21)
-
Specification