Apparatus and method for processing video data

• US 20060067585A1
• Filed: 09/20/2005
• Published: 03/30/2006
• Est. Priority Date: 09/21/2004
• Status: Active Grant

• Current Assignee: Euclid Discoveries LLC
• Original Assignee: Euclid Discoveries LLC
• Inventors: Pace, Charles Paul
• Granted Patent: US 7,426,285 B2
• Time in Patent Office: Days
• US Class Current: 382/243
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/553 Motion estimation dealing with occlusions
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

An apparatus and methods for processing video data are described. The invention provides a representation of video data that can be used to assess agreement between the data and a fitting model for a particular parameterization of the data. This allows the comparison of different parameterization techniques and the selection of the optimum one for continued video processing of the particular data. The representation can be utilized in intermediate form as part of a larger process or as a feedback mechanism for processing video data. When utilized in its intermediate form, the invention can be used in processes for storage, enhancement, refinement, feature extraction, compression, coding, and transmission of video data. The invention serves to extract salient information in a robust and efficient manner while addressing the problems typically associated with video data sources

Claims

1. A computer apparatus for the purpose of generating an encoded form of video signal data from a plurality of video frames, comprising:

a means of identifying corresponding elements of said object between two or more frames;

a means of modeling such correspondences to generate modeled correspondences;

a means of resampling pel data in said video frames associated with said object, said resampling means utilizing said modeled correspondences;

a means of restoring the spatial positions of the re-sampled pel data, said restoring process utilizing the modeled correspondences;

said object being one or more objects, and

said re-sampled data being the intermediate form of the data.

2. The method of claim 1 wherein the object is tracked by a tracking method, comprising:

a means of detecting an object in the video frame sequence;

a means of tracking said object through two or more frames of the video frame sequence;

said object detection and tracking means comprising a Viola/Jones face detection algorithm.

3. The method of claim 1 wherein the object is segmented from video frame using a segmentation method, comprising:

a means of segmenting said pel data associated with said object from other pel data in said video frame sequence;

a means of composing said restored pels together with associated segmentation data to create original video frame;

said segmentation means comprising temporal integration.

4. The method of claim 1 wherein the correspondence models are factored into global models, comprising:

a means of integrating correspondence measurements into a model of global motion;

said correspondence modeling means comprising a robust sampling consensus for the solution of a two dimensional affine motion model, and

said correspondence modeling means comprising a sampling population based on finite differences generated from block-based motion estimation between two or more video frames in said sequence.

5. The method of claim 1 wherein the intermediate data is further encoded, comprising:

a means of decomposing said normalized object pel data into an encoded representation;

a means of recomposing said normalized object pel data from encoded representation;

said decomposing means comprising Principal Component Analysis, and

said recomposing means comprising Principal Component Analysis.

6. The method of claims 2, 3, and 4, 5 wherein the methods are combined, comprising:

a means of tracking a multitude of objects as in claim 2 in order to identify corresponding elements in claim 1;

a means of segmenting the pels as in claim 3 in order to resample pel data in claim 1;

a means of integrating motion parameters as in claim 4 in order to facilitate a compact parameterization of the motion parameters in claim 1;

a means of decomposing and recomposing said normalized and segmented pels as in claim 5.

7. The method of claim 5 wherein the non-object pels of the frame are modeled as the object pels are in a method, comprising:

said object being the residual, non-object, of the frame when other objects are removed.

8. The method of claim 5 wherein the segmented and resampled pels are combined with a conventional video compression/decompression process, comprising:

a means of supplying the said re-sampled pels as standard video data to conventional video compression process;

a means of storing and transmitting model correspondence data along with corresponding encoded video data;

whereby said compression/decompression method can enable said conventional video compression method to increase compression efficiency.