Content adaptive video encoder

• US 6,909,745 B1
• Filed: 06/05/2001
• Issued: 06/21/2005
• Est. Priority Date: 06/05/2001
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Puri, Atul, Civanlar, Mehmet Reha
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/874,873
• Time in Patent Office: 1,477 Days
• Field of Search: 375/240.01, 375/240.12, 375/240.08, 375/240.24, 375/240.02, 348/384.1, 348/397.1, 348/399.1, 348/404.1, 382/236, 382/239, 382/243
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

A system for content adaptive encoding and decoding video is disclosed. The system comprises modules for segmenting video content into segments based on predefined classifications or models. Examples of such classifications comprise action scenes, slow scenes, low or high detail scenes, and brightness of the scenes. Based on the segment classifications, each segment is encoded with a different encoder chosen from a plurality of encoders. Each encoder is associated with a model. The chosen encoder is particularly suited to encoding the unique subject matter of the segment. The coded bit-stream for each segment includes information regarding which encoder was used to encode that segment. A matching decoder of a plurality of decoders is chosen using the information in the coded bitstream to decode each segment using a decoder suited for the classification or model of the segment. If scenes exist which do not fall in a predefined classification, or where classification is more difficult based on the scene content, these scenes are segmented, coded and decoded using a generic coder and decoder.

Claims

1. A system for content adaptive encoding of data comprising:

(1) an extractor that divides video content into temporal portions;

(2) a locator that associates descriptors to each portion based on portion content, the locator further locating at least one of subsegments and regions of interest;

(3) a mapper that maps each portion of the video content to a model from a plurality of models based on the portion descriptors, the mapper further comprising;

(a) a plurality of content model units, each content model unit of the plurality of content model units being associated with a model of the plurality of models;

(b) a plurality of comparators, each comparator of the plurality of comparators connected to a content model unit and an output from the extractor or an output from the locator; and

(c) a plurality of selectors, wherein each selector of the plurality of selectors is connected to two of the comparators; and

(4) a plurality of encoders, each encoder of the plurality of encoders configured to encode portions according to the model associated with the portion.

2. The system of claim 1, wherein the plurality of content model units further comprises a generic content model unit.

3. The system of claim 2, wherein the mapper outputs from the selectors a signal that controls a switch to route portions to one of the plurality of encoders.

4. A system for content adaptive encoding of data comprising:

(1) means for dividing video content into temporal portions;

(2) means for associating descriptors to each portion based on portion content, the means for associating descriptors further locating at least one of subsegments and regions of interest;

(3) means for mapping each portion of the video content to a model from a plurality of models based on the portion descriptors, the means for mapping further comprising;

(a) a plurality of content model units, each content model unit of the plurality of content model units being associated with a model of the plurality of models;

(b) a plurality of comparators, each comparator of the plurality of comparators connected to a content model unit and an output from the extractor or an output from the locator; and

(c) a plurality of selectors, wherein each selector of the plurality of selectors is connected to two of the comparators; and

(4) a plurality of encoders, each encoder of the plurality of encoders configured to encode portions according to the model associated with the portion.

5. The system of claim 4, wherein the plurality of content model units further comprises a generic content model unit.

6. The system of claim 5, wherein the means for mapping outputs from the selectors a signal that controls a switch to route portions to one of the plurality of encoders.

7. A method for content adaptive encoding of data, the method comprising:

(1) dividing video content into temporal portions;

(2) associating descriptors to each portion based on portion content, the means for associating descriptors further locating at least one of subsegments and regions of interest;

(3) mapping each portion of the video content to a model from a plurality of models based on the portion descriptors, the step of mapping each portion utilizing;

(a) a plurality of content model units, each content model unit of the plurality of content model units being associated with a model of the plurality of models;

(b) a plurality of comparators, each comparator of the plurality of comparators connected to a content model unit and an output from the extractor or an output from the locator; and

(c) a plurality of selectors, wherein each selector of the plurality of selectors is connected to two of the comparators; and

(4) encoding each portion of the video content using one of a plurality of encoders, each encoder of the plurality of encoders configured to encode portions according to the model associated with the portion.

8. The method of claim 7, wherein the plurality of content model units further comprises a generic content model unit.

9. The method of claim 8, wherein the step of mapping each portion of the video content further comprises outputting from the selectors a signal that controls a switch to route portions to one of the plurality of encoders.

10. A computer-readable medium storing instructions for controlling a computing device to adaptively encode data based on content, the instructions comprising:

(1) dividing video content into temporal portions;

(2) associating descriptors to each portion based on portion content, the means for associating descriptors further locating at least one of subsegments and regions of interest;

(3) mapping each portion of the video content to a model from a plurality of models based on the portion descriptors, the step of mapping each portion utilizing;

(a) a plurality of content model units, each content model unit of the plurality of content model units being associated with a model of the plurality of models;

(b) a plurality of comparators, each comparator of the plurality of comparators connected to a content model unit and an output from the extractor or an output from the locator; and

(c) a plurality of selectors, wherein each selector of the plurality of selectors is connected to two of the comparators; and

(4) encoding each portion of the video content using one of a plurality of encoders, each encoder of the plurality of encoders configured to encode portions according to the model associated with the portion.

11. The computer-readable medium of claim 10, wherein the plurality of content model units further comprises a generic content model unit.

12. The computer-readable medium of claim 11, wherein the step of mapping each portion of the video content further comprises outputting from the selectors a signal that controls a switch to route portions to one of the plurality of encoders.