SYSTEM AND METHOD FOR OBJECT BASED PARAMETRIC VIDEO CODING

• US 20140348231A1
• Filed: 08/06/2014
• Published: 11/27/2014
• Est. Priority Date: 09/04/2009
• Status: Active Grant

• Current Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Inventors: Chaudhury, Santanu, Mathur, Mona, Khandelia, Aditya, Tripathi, Subarna, Lall, Brejesh, Gorecha, Saurabh, Dutta Roy, Sumantra
• Granted Patent: US 9,813,731 B2
• Time in Patent Office: Days
• US Class Current: 375/240.08
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A video compression framework based on parametric object and background compression is proposed. At the encoder, an embodiment detects objects and segments frames into regions corresponding to the foreground object and the background. The object and the background are individually encoded using separate parametric coding techniques. While the object is encoded using the projection of coefficients to the orthonormal basis of the learnt subspace (used for appearance based object tracking), the background is characterized using an auto-regressive (AR) process model. An advantage of the proposed schemes is that the decoder structure allows for simultaneous reconstruction of object and background, thus making it amenable to the new multi-thread/multi-processor architectures.

Claims

1. A digital video encoder, comprising:

one or more memories; and

digital signal processing circuitry, which, in operation;

segments frames in a digital video sequence into object regions and background regions;

generates object motion and appearance parameters based on segmented object regions; and

generates an auto regressive background model based on segmented background regions.

2. The digital video encoder of claim 1 wherein the digital signal processing circuitry comprises:

a segmentation module which, in operation, segments frames in the digital sequence into object regions and background regions;

an object parameter generator which, in operation, generates object motion and appearance parameters; and

an auto-regressive model generator, which, in operation, generates the auto regressive background model based on segmented background regions.

3. The encoder of claim 2 wherein the digital signal processing circuitry comprises an encoding module which, in operation, compresses background I frames, and wherein the segmentation module, in operation, provides background I frames to the encoding module.

4. The encoder of claim 3 wherein the digital signal processing circuitry comprises a block categorizer which, in operation, categorizes blocks in background P frames regions into edge blocks and non-edge blocks, wherein the encoding module, in operation, compresses the edge blocks and the auto regressive model generator, in operation, generates the auto regressive background model based on non-edge blocks of the segmented background regions.

5. The encoder of claim 4, wherein the digital signal processing circuitry comprises a motion threader, which, in operation, generates motion thread information based on non-edge blocks of segmented background regions, wherein the auto regressive model generator, in operation, generates the auto regressive background model based on the motion thread information.

6. The encoder of claim 5, wherein the digital signal processing circuitry comprises a block removal module, which, in operation, removes background blocks based on an error criteria.

7. The encoder of claim 4 wherein the block categorizer comprises a gradient-based edge detector.

8. The encoder of claim 2 wherein the object parameter generator comprises an object tracking module, a residual calculator and an H.264 I slice encoder and the object tracking module, in operation, generates object motion and appearance parameters based on received object region information and decoded objects received from the residual calculator.

9. The encoder of claim 3 wherein the encoding module comprises an H.264/AVG encoder.

10. A method comprising:

segmenting frames in a digital video sequence into object regions and background regions;

generating object motion and appearance parameters based on segmented object regions; and

generating auto regressive model parameters based on segmented background regions.

11. The method of claim 10, comprising:

compressing background I frames using transform-based encoding.

12. The method of claim 10, comprising:

compressing background I frames using H.264/AVC encoding.

13. The method of claim 10, comprising categorizing blocks in background P frames into edge blocks and non-edge blocks, wherein the generating the auto regressive background model is based on non-edge blocks of the background P frames.

14. The method of claim 13, comprising:

encoding the edge blocks using H.264/AVC encoding.

15. The method of claim 13, comprising:

generating motion thread information based on non-edge blocks of the background P frames.

16. The method of claim 13 wherein the categorizing blocks comprises gradient-based edge detection.

17. The method of claim 10 wherein the generating object motion and appearance parameters comprises calculating and compressing object residuals.

18. A non-transitory, computer-readable memory medium whose contents cause a digital signal processor to perform a method of encoding a video image sequence, the method comprising:

segmenting frames in the digital video sequence into object regions and background regions;

generating object motion and appearance parameters based on segmented object regions; and

generating auto regressive model parameters based on segmented background regions.

20. The medium of claim 18 wherein the method comprises categorizing blocks in background P frames into edge blocks and non-edge blocks, and the generating the auto regressive background model is based on non-edge blocks of the background P frames.

21. The medium of claim 20 wherein the method comprises:

generating motion thread information based on non-edge blocks of the background P frames.

22. The medium of claim 20 wherein the categorizing blocks comprises gradient-based edge detection.

19. The medium of claim medium 18 wherein the contents are instructions, which, when executed by the digital signal processor, cause the digital signal processor to perform the method.