Video Stream Modifier

• US 20080304561A1
• Filed: 12/19/2005
• Published: 12/11/2008
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: MaxLinear Communications, LLC (f/k/a Entropic Communications, LLC) (Maxlinear Incorporated)
• Original Assignee: NXP B.V. (NXP Semiconductors NV)
• Inventors: Tichelaar, Hans, Vanderheijden, Geert
• Granted Patent: US 8,363,714 B2
• Time in Patent Office: Days
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16

Abstract

A compressed data stream modifier 270 is disclosed. The stream modifier modifies an input data stream that may comprise audio and video data streams conforming to any one of the well-known video compression standards, for example, MPEG or AVC. The input stream is first de-multiplexed to obtain a single video elementary stream. The video elementary stream is then decoded by means of a variable-length decoder and provided to the data stream modifier. A decoding complexity of each frame is estimated by an estimator 210 and provided to a controller 250. The controller selects a number of regions from each frame based on some psycho-visual criteria, such that a method of modification of these regions can bring the decoding complexity within an available capacity at a decoder. An efficient means of adapting a decoding complexity to an available capacity is thus obtained.

Claims

1. A device for modifying a compressed data stream to be decoded at a decoder, the compressed data stream being obtained by encoding a sequence of pictures comprising regions of pixels, the device comprising:

estimating means for estimating a decoding complexity required for decoding a picture from the compressed data stream; and

modifying means for modifying the compressed data stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that the modifying means are arranged to modify a method of decoding indicated in the compressed data stream.

2. A device as claimed in claim 1, wherein the decoding complexity is based on a computational load or a memory access bandwidth for decoding the picture.

3. A device as claimed in claim 1, wherein the modifying means are arranged to select a region on the basis of a heuristic psycho-visual selection criterion and modify a method of decoding for said region.

4. A device as claimed in claim 3, wherein the selection criterion is a proximity of the region to boundaries of the picture.

5. A device as claimed in claim 3, wherein the selection criterion is a speed of movement of the region.

6. A device as claimed in claim 3, wherein the selection criterion is a measure of texture of the region.

7. A device as claimed in claim 1, wherein the modifying means are arranged to modify a method of motion-compensated prediction of a region of a picture from bi-directional prediction to unidirectional prediction.

8. A device as claimed in claim 1, wherein the modifying means are arranged to modify a quantized transform coefficient present in the compressed data stream.

9. A device as claimed in claim 3, wherein the modifying means are arranged to select a region of rectangular shape.

10. A device as claimed in claim 3, wherein the modifying means are arranged to select a region of a shape that adapts to an object in the picture.

11. A video-processing system for compression and decompression of a sequence of pictures, the system comprising:

an encoder for encoding the sequence of pictures and output of a compressed data stream;

a device for modifying the compressed data stream according to claim 1; and

a decoder for decoding the modified compressed data stream to obtain a sequence of pictures.

12. A video-processing apparatus comprising

receiving means for receiving a signal comprising a compressed data stream corresponding to a sequence of pictures;

and

a device for modifying the compressed data stream according to claim 1.

13. A method of modifying a compressed data stream to be decoded at a decoder, the compressed data stream being obtained by encoding a sequence of pictures comprising regions of pixels, the method comprising:

estimating a decoding complexity required for decoding a picture from the compressed data stream; and

modifying the stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that

modifying the compressed data stream comprises modifying a method of decoding indicated in the stream.

14. A computer program product comprising a code enabling a processor to execute a method of modifying a compressed data stream according to claim 13, the computer program product comprising:

a code for estimating a decoding complexity required for decoding a picture from the compressed data stream; and

a code for modifying the stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that

the code for modifying the compressed data stream comprises a code for modifying a method of decoding indicated in the stream.