Segmentation of image features using hierarchical analysis of multi-valued image data and weighted averaging of segmentation results
First Claim
1. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels and a second spatial image characteristic different from the first image characteristic, comprising:
- obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame;
segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features;
segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatially segmented video image features; and
forming jointly segmented first and second video image features as a weighted sum of the first and second motion segmented video image features and the first and second spatial segmented video image features;
wherein the first and second motion segmented image features are computed independently from the first and second spatial segmented video image features.
2 Assignments
0 Petitions
Accused Products
Abstract
Homogeneous moving objects of arbitrary shapes are segmented and tracked with respect to the motion of the objects. In an intraframe mode of operation, a segmentation method includes obtaining a motion representation of corresponding pixels in the selected video image frame and a preceding video image frame to form motion-segmented video image features. Video image features are also segmented according to their spatial image characteristics (e.g., color) to form spatially-segmented video image features. Finally, the video image features are jointly segmented as a weighted combination of the motion-segmented video image features and the spatially-segmented video image features. The joint motion and spatial segmentation of image features provides enhanced accuracy in representing moving image features. This enhanced accuracy is particularly beneficial because the motion of image features is a significant display characteristic for human observers.
247 Citations
51 Claims
-
1. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels and a second spatial image characteristic different from the first image characteristic, comprising:
-
obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame; segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features; segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatially segmented video image features; and forming jointly segmented first and second video image features as a weighted sum of the first and second motion segmented video image features and the first and second spatial segmented video image features; wherein the first and second motion segmented image features are computed independently from the first and second spatial segmented video image features. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having motion relative to the second image feature between the selected video image frame and a preceding video image frame, comprising:
-
obtaining a motion vector representation of corresponding pixels in the selected video image frame and the preceding video image frame; and segmenting the first and second video image features according to the motion vector representation, including determining a residue portion of the motion vector representation simplifying the residue portion to create a simplified motion vector representation, including filtering motion vectors of the residue portion to reduce variation in the motion vectors; distinguishing a region or regions of homogenous motion in the simplified motion vector representation; and repeating the determining, simplifying and distinguishing steps, so as to form first and second motion segmented video image features. - View Dependent Claims (14, 15, 16, 17, 18)
-
-
19. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature, comprising:
-
obtaining a representation of the multidimensional characteristic for pixel locations in the selected video image frame; determining a residue portion of the multidimensional characteristic representation; simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics; distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation; and repeating the determining, simplifying and distinguishing steps to form a first segmented video image feature. - View Dependent Claims (20, 21, 22, 23, 24)
-
-
25. A computer-readable medium storing computer-executable programming for segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels of a second spatial image characteristic different from the first image characteristic, the medium comprising:
-
programming for obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame; programming for segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features; programming for segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatial segmented video image features; and programming for forming jointly segmented first and second video image features as a weighted sum of the first and second motion segmented video image features and the first and second spatial segmented video image features. - View Dependent Claims (26, 27)
-
-
28. A method for segmenting an arbitrary first image feature in a selected video image frame from a second image feature in a second video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature between the first and second video image frames, comprising:
-
obtaining a representation of the multidimensional characteristic for pixel locations in the first video image frame determining a residue portion of the multidimensional characteristic representation; simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics; distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation; and
repeating the determining, simplifying and distinguishing steps to form first and second segmented video image features. - View Dependent Claims (29, 30)
-
-
31. A computer-readable medium storing computer-executable programming for segmenting arbitrary image features in selected video image frames, the arbitrary image features having differing multidimensional characteristics that vary between the video image frames, the medium comprising:
-
a) programming code for obtaining representations of the multidimensional characteristic for pixel locations in the video image frames; b) programming code for determining a residue portion of the multidimensional characteristic representation for a video image frame; c) programming code for simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics; d) programming code for distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation; and e) programming code for iteratively executing the programming code listed in b), c), and d) above to form at least a first segmented video image feature. - View Dependent Claims (32, 33, 34, 35)
-
-
36. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels and a second spatial image characteristic different from the first image characteristic, comprising:
-
obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame; segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features; segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatially segmented video image features; and forming jointly segmented first and second video image features according to the first and second motion segmented video image features and the first and second spatial segmented video image features; wherein segmenting the first and second video image features according to the motion representation includes iterative application of a hierarchical motion segmentation engine; and wherein the hierarchical motion segmentation engine applies nonlinear motion simplification to the motion representation on each iteration according to a nonlinear filter that functions to maintain distinctions between the first and second video image features. - View Dependent Claims (37)
-
-
38. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having motion relative to the second image feature between the selected video image frame and a preceding video image frame, comprising:
-
obtaining a motion vector representation of corresponding pixels in the selected video image frame and the preceding video image frame; and segmenting the first and second video image features according to the motion vector representation to form first and second motion segmented video image features, including iteratively applying a hierarchical motion segmentation engine; wherein the hierarchical motion segmentation engine applies nonlinear motion simplification to the motion vectors on each iteration according to a nonlinear filter that functions to maintain distinctions between the first and second video image features. - View Dependent Claims (39)
-
-
40. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature between the selected video image frame and a preceding video image frame, comprising:
-
obtaining a representation of the multidimensional characteristic for corresponding pixels in the selected video image frame and the preceding video image frame; and iteratively applying a hierarchical characteristic segmentation engine for segmenting the first and second video image features according to the multidimensional characteristic representation to form first and second motion segmented video image features, wherein the hierarchical motion segmentation engine simplifies the multidimensional characteristic representation on each iteration according to a nonlinear filter that functions to maintain distinctions between the first and second video image features. - View Dependent Claims (41)
-
-
42. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature, comprising;
-
obtaining a representation of the multidimensional characteristic for pixel locations in the selected video image frame; modeling the multidimensional characteristics at pixel locations in the selected frame to compute a modeled image and determining a residue portion of the multidimensional characteristic representation, where the residue portion comprises a portion of the selected frame identified by computing a difference between the selected image frame and the modeled image; simplifying the residue portion to create a simplified multidimensional characteristic representation; distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation; and
repeating the determining, simplifying and distinguishing steps to form a first segmented video image feature. - View Dependent Claims (43)
-
-
44. A computer-readable medium storing computer-executable instructions for segmenting arbitrary image features in a first video image frame, an arbitrary first image feature having motion relative to an adjacent arbitrary second image feature between the first video image frame and a preceding second video image frame, the computer-executable instructions for:
-
obtaining a segmentation of the second video image frame, wherein the segmentation incorporates motion segmented video image features based upon comparison of the second video image frame with a preceding third video image frame; obtaining a motion vector representation of corresponding pixels in the first video image frame and the second video image frame; identifying one or more seed regions in the segmentation of the second video image frame; and segmenting the first and second video image features in the first video image frame by classifying each of plural motion vectors of the motion vector representation based upon similarity to the identified one or more seed regions.
-
-
45. A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the first image feature having motion relative to the second image feature between the selected video image frame and a preceding video image frame, the method comprising:
-
obtaining a motion vector representation of corresponding pixels in a selected video image frame and a preceding video image frame; obtaining a segmentation for the preceding video image frame; segmenting a first video image feature and a second video image feature in the selected video image frame classifying plural motion vectors of the motion vector representation based upon the segmentation for the preceding video image frame. - View Dependent Claims (46, 47, 48, 49)
-
-
50. An apparatus for segmenting image features of a selected video image frame, a first image feature having motion relative to an adjacent second image feature between the selected video image frame and a preceding video image frame, the apparatus comprising:
-
means for obtaining a motion vector representation of corresponding pixels in a selected video image frame and a preceding video image frame; means for obtaining a video feature segmentation for the preceding video image frame; and means for segmenting a first video image feature in the selected video image frame from an adjacent second video image feature in the selected video image frame, said means for segmenting including a means for classifying each of plural motion vectors of the motion vector representation based upon the video image feature segmentation for the preceding video image frame. - View Dependent Claims (51)
-
Specification