Video stream modifier

• US 8,363,714 B2
• Filed: 12/19/2005
• Issued: 01/29/2013
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: MaxLinear Communications, LLC (f/k/a Entropic Communications, LLC) (Maxlinear Incorporated)
• Original Assignee: Entropic Communications Incorporated (Maxlinear Incorporated)
• Inventors: Vanderheijden, Geert, Tichelaar, Hans
• Primary Examiner(s): Sam, Phirin
• Application Number:
  US11/722,557
• Publication Number:
  US 20080304561A1
• Time in Patent Office: 2,598 Days
• Field of Search: None
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16

Abstract

A compressed data stream modifier 270 is disclosed. The stream modifier modifies an input data stream that may comprise audio and video data streams conforming to any one of the well-known video compression standards, for example, MPEG or AVC. The input stream is first de-multiplexed to obtain a single video elementary stream. The video elementary stream is then decoded by means of a variable-length decoder and provided to the data stream modifier. A decoding complexity of each frame is estimated by an estimator 210 and provided to a controller 250. The controller selects a number of regions from each frame based on some psycho-visual criteria, such that a method of modification of these regions can bring the decoding complexity within an available capacity at a decoder. An efficient means of adapting a decoding complexity to an available capacity is thus obtained.

Claims

1. A device for modifying a compressed data stream to be decoded at a decoder, the compressed data stream being obtained by encoding a sequence of pictures comprising regions of pixels, the device comprising:

estimating means for estimating a decoding complexity required for decoding a picture from the compressed data stream; and

modifying means for modifying the compressed data stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that the modifying means are arranged to modify a method of decoding indicated in the compressed data stream;

wherein the modifying means are arranged to select a region on the basis of a heuristic psycho-visual selection criterion and modify a method of decoding for said region.

2. A device as claimed in claim 1, wherein the decoding complexity is based on a computational load or a memory access bandwidth for decoding the picture.

3. A device as claimed in claim 1, wherein the selection criterion is a proximity of the region to boundaries of the picture.

4. A device as claimed in claim 1, wherein the selection criterion is a speed of movement of the region.

5. A device as claimed in claim 1, wherein the selection criterion is a measure of texture of the region.

6. A device as claimed in claim 1, wherein the modifying means are arranged to modify a method of motion-compensated prediction of a region of a picture from bi-directional prediction to unidirectional prediction.

7. A device as claimed in claim 1, wherein the modifying means are arranged to modify a quantized transform coefficient present in the compressed data stream.

8. A device as claimed in claim 1, wherein the modifying means are arranged to select a region of rectangular shape.

9. A device as claimed in claim 1, wherein the modifying means are arranged to select a region of a shape that adapts to an object in the picture.

10. A video-processing system for compression and decompression of a sequence of pictures, the system comprising:

an encoder for encoding the sequence of pictures and output of a compressed data stream;

a device for modifying the compressed data stream according to claim 1; and

a decoder for decoding the modified compressed data stream to obtain a sequence of pictures.

11. A video-processing apparatus comprising

receiving means for receiving a signal comprising a compressed data stream corresponding to a sequence of pictures;

and

a device for modifying the compressed data stream according to claim 1.

12. A method of modifying a compressed data stream to be decoded at a decoder, the compressed data stream being obtained by encoding a sequence of pictures comprising regions of pixels, the method comprising:

estimating, via means for estimating, a decoding complexity required for decoding a picture from the compressed data stream; and

modifying, via means for modifying the compressed data stream, the stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that modifying the compressed data stream, via the means for modifying, comprises modifying a method of decoding indicated in the stream;

wherein the modifying means are arranged to select a region on the basis of a heuristic psycho-visual selection criterion and modify a method of decoding for said region.

13. A method as claimed in claim 12, wherein the decoding complexity is based on a computational load or a memory access bandwidth for decoding the picture.

14. A method as claimed in claim 12, wherein the selection criterion is a proximity of the region to boundaries of the picture, a speed of movement of the region, or a measure of texture of the region.

15. A method as claimed in claim 12, wherein the modifying means are arranged to modify a method of motion-compensated prediction of a region of a picture from bi-directional prediction to unidirectional prediction.

16. A method as claimed in claim 12, wherein the modifying means are arranged to modify a quantized transform coefficient present in the compressed data stream or arranged to select a region of a shape that adapts to an object in the picture.

17. A device for modifying a compressed data stream to be decoded at a decoder, the compressed data stream being obtained by encoding a sequence of pictures comprising regions of pixels, the device comprising:

an estimating component that estimates a decoding complexity required for decoding a picture from the compressed data stream; and

a modifying component that modifies the compressed data stream when the decoding complexity of the picture approaches a capacity of the decoder, characterized in that the modifying component is arranged to modify a method of decoding indicated in the compressed data stream;

wherein the modifying component is arranged to select a region on the basis of a heuristic psycho-visual selection criterion and modify a method of decoding for said region.

18. A device as claimed in claim 17, wherein the decoding complexity is based on a computational load or a memory access bandwidth for decoding the picture.

19. A device as claimed in claim 17, wherein the selection criterion includes a proximity of the region to boundaries of the picture, a speed of movement of the region, or a measure of texture of the region.

20. A device as claimed in claim 17, wherein the modifying component is arranged to modify a quantized transform coefficient present in the compressed data stream or arranged to select a region of a shape that adapts to an object in the picture.