Video compression repository and model reuse

• US 9,532,069 B2
• Filed: 10/29/2014
• Issued: 12/27/2016
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: Euclid Discoveries LLC
• Original Assignee: Euclid Discoveries LLC
• Inventors: Pace, Charles P., DeForest, Darin, Lee, Nigel, Pizzorni, Renato, Wingard, Richard
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/527,477
• Publication Number:
  US 20150124874A1
• Time in Patent Office: 790 Days
• Field of Search: 382/232, 382/243, 382/305, 37524001-24029
• US Class Current: 1/1
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/54 using feature points or meshes

Abstract

Systems and methods of improving video encoding/decoding efficiency may be provided. A feature-based processing stream is applied to video data having a series of video frames. Computer-vision-based feature and object detection algorithms identify regions of interest throughout the video datacube. The detected features and objects are modeled with a compact set of parameters, and similar feature/object instances are associated across frames. Associated features/objects are formed into tracks, and each track is given a representative, characteristic feature. Similar characteristic features are clustered and then stored in a model library, for reuse in the compression of other videos. A model-based compression framework makes use of the preserved model data by detecting features in a new video to be encoded, relating those features to specific blocks of data, and accessing similar model information from the model library. The formation of model libraries can be specialized to include personal, “smart” model libraries, differential libraries, and predictive libraries. Predictive model libraries can be modified to handle a variety of demand scenarios.

Claims

1. A method of providing video data, comprising:

encoding a subject video stream by a feature-based compression process that utilizes feature models from a global feature model library, said encoding implicitly using the feature models to indicate macroblocks in the subject video to encode and explicitly using pixels of the feature models to improve prediction of said macroblocks, resulting in an encoded video data, where the use of feature models to indicate macroblocks in the subject video avoids the need to segment features from non-features during the feature-based compression process of the subject video, and wherein the feature-based compression process applies feature-based prediction across multiple different video sources based on the feature models, the multiple different video sources being at least the subject video stream and one or more input videos used to generate the global feature model library; and

transmitting the encoded video data to a requesting device upon command, said feature models from the global feature model library being made accessible to the requesting device and enabling decoding of the encoded video data at the requesting device;

wherein the global feature model library is formed by;

receiving the one or more input videos, each input video being different from the subject video stream; and

for each of the input videos, generating feature information and a respective feature model.

2. The method as claimed in claim 1 wherein the global feature model library is further formed by storing in a data store or on cloud storage the feature models generated from the input videos, the data store or cloud storage providing pertinent feature models to the feature-based compression process and the requesting device.

3. The method as claimed in claim 1 wherein the global feature model library is further formed by:

for each input video, identifying and indexing features in the input video, the indexed features forming the respective feature model, said indexing including, for each identified feature, indicating location in the input video of the identified feature; and

wherein the decoding of the encoded video data at the requesting device uses the indexed features and corresponding feature locations in the input videos to obtain said feature models and decode the encoded video data.

4. The method as claimed in claim 3 wherein the indexing of features is a hash-based index.

5. The method as claimed in claim 1 wherein the feature models from the global feature model library form a working model subset library that is specialized per requesting device or per subject video stream.

6. The method as claimed in claim 1 wherein the feature models from the global feature model library form a working model subset library that is a differential library with respect to state of any libraries in the requesting device.

7. The method as claimed in claim 1 wherein the feature models from the global feature model library form a working model subset library that is a predictive model library storing feature models as a function of profile of an end user of the requesting device.

8. The method as claimed in claim 7 wherein the predictive model library has modifiable (settable) parameters enabling application to a variety of demand scenarios.

9. A video compression system comprising:

a model library storing video data and serving as a source of streaming video; and

a codec operatively coupled to the model library, and in response to a request for a subject video, the codec being executed by a processor to (i) encode stored video data in the model library corresponding to the requested certain subject video and to (ii) stream the encoded video data from the model library, wherein the codec applies feature-based prediction and compression using feature models from a global feature model library, wherein the global feature model library is formed by;

receiving one or more input videos, each input video being different from the stored video data in the model library corresponding to the requested certain video; and

for each of the input videos, generating feature information and a-respective feature models;

such that the codec applies feature-based compression, which includes implicitly using feature models to indicate which macroblocks in the subject video to encode and explicitly using the pixels of the feature models to improve prediction of said macroblocks, where association of the feature models from the global feature model library with macroblocks in the subject video avoids the need to segment features from non-features during the feature-based compression process of the subject video.

10. The video compression system as claimed in claim 9 wherein the codec:

encodes the stored video data in the model library by a feature-based compression that applies feature-based prediction based on the feature models, and

transmits the encoded video data to a requesting device, the encoded video data being the streamed video data from the model library.

11. The video compression system as claimed in claim 10 wherein the feature models from the global feature model library are made accessible to the requesting device and enable decoding of the encoded video data at the requesting device.

12. The video compression system as claimed in claim 10 wherein the global feature model library is further formed by:

for each input video, identifying and indexing features in the input video, the indexed features forming the respective feature model, said indexing including, for each identified feature, indicating location in the input video of the identified feature; and

wherein decoding of the encoded video data at the requesting device uses the indexed features and corresponding feature locations in the input videos to obtain said feature models and decode the encoded video data.

13. The video compression system as claimed in claim 12 wherein the indexing of features is a hash-based index.

14. The video compression system as claimed in claim 9 wherein the feature models are stored in the model library.

15. The video compression system as claimed in claim 9 wherein the feature models from the global feature model library form a working model subset library that is specialized per requesting device or per streamed encoded video data.

16. The video compression system as claimed in claim 9 wherein the feature models from the global feature model library form a working model subset library that is a differential library with respect to state of any libraries in a requesting device.

17. The video compression system as claimed in claim 9 wherein the feature models from the global feature model library form a working model subset library that is a predictive model library storing feature models as a function of profile of end-user.

18. The video compression system as claimed in claim 17 wherein the predictive model library has modifiable parameters enabling application to a variety of demand scenarios.