Calibration system for adaptive media encoder

• US 10,080,022 B2
• Filed: 12/12/2014
• Issued: 09/18/2018
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Original Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Inventors: Annamraju, Venu, Rajamani, Harish, Kamarthi, Mallikarjuna
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US14/568,893
• Publication Number:
  US 20150172727A1
• Time in Patent Office: 1,376 Days
• Field of Search: 37524025
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

A data processing system for calibrating a media codec comprising a sequence of time-stamped frames and comprising: an encoder subsystem configured to perform encoding in accordance with one or more encode parameters; a decoder subsystem; and a calibration system comprising: a data store for storing an encoded media stream; and a calibration monitor configured to, on the media codec entering a calibration mode, cause: the decoder subsystem to decode the encoded media stream so as to generate a decoded media stream; the encoder subsystem to re-encode said decoded media stream; and the re-encoded media stream to pass back into the decoder subsystem; the calibration monitor being configured to, through variation of the encode parameters of the encoder subsystem, identify maximal encode parameters corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.

Claims

1. A data processing system for calibrating a media codec having an encoder configured to perform encoding in accordance with one or more encode parameters and operable to encode a media stream comprising a sequence of time-stamped frames, the data processing system comprising:

a decoder subsystem; and

a calibration monitor operable in a calibration mode of the media codec to form a closed loop between the decoder subsystem and the encoder of said media codec by;

causing the decoder subsystem to decode an encoded media stream so as to generate a decoded media stream and apply said decoded media stream as an input to the encoder of said media codec;

causing the encoder of said media codec to re-encode said decoded media stream from said decoder subsystem; and

causing the re-encoded media stream to loop back into the decoder subsystem for decoding the re-encoded media stream;

the calibration monitor being configured to, in the calibration mode, identify maximal encode parameters for the encoder of said media codec corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.

2. A data processing system as claimed in claim 1, the calibration monitor being configured to vary the encode parameters of the encoder of said media codec and the decoder subsystem in accordance with a predetermined algorithm so as to identify the maximal encode parameters.

3. A data processing system as claimed in claim 1, wherein the media codec is adaptive and the calibration monitor is configured to allow the encode parameters of the encoder of said media codec to adapt according to an adaptation algorithm of the encoder of said media codec.

4. A data processing system as claimed in claim 1, the decoder subsystem being configured to perform decoding in accordance with one or more decode parameters, and the calibration monitor being further configured to, through variation of the decode parameters of the decoder subsystem, identify maximal decode parameters corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.

5. A data processing system as claimed in claim 1, further comprising a network simulation unit for connection between the encoder of said media codec and the decoder subsystem and configured to introduce network characteristics calculated according to a predefined model into an encoded media stream passing through it, the calibration monitor being configured to cause the re-encoded media stream to pass through the network simulation unit before passing back into the decoder subsystem.

6. A data processing system as claimed in claim 5, wherein the network characteristics include one or more of data loss, data errors and jitter.

7. A data processing system as claimed in claim 5, wherein the predefined model is configured so as to cause network characteristics to be introduced on a pseudo-random basis.

8. A data processing system as claimed in claim 1, the encoder further comprising a pre-processor configured to rescale at least some of the sequence of time-stamped frames received from the decoder subsystem.

9. A data processing system as claimed in claim 8, wherein the pre-processor is configured to rescale at least some of the sequence of time-stamped frames by reducing the size of those frames.

10. A data processing system as claimed in claim 1, wherein the calibration monitor is configured to cause the encoder to sleep for a determined period of time following each successfully encoded frame, said period of time being determined so as to cause the encoder to become overloaded by the encoded media stream.

11. A method of calibrating a media codec for encoding a media stream comprising a sequence of time-stamped frames, the media codec comprising an encoder configured to perform encoding in accordance with one or more encode parameters, the method comprising:

playing out an encoded media stream to a decoder subsystem so as to cause the decoder subsystem to decode the encoded media stream and generate a decoded media stream;

applying the decoded media stream as an input to the encoder of said media codec;

causing the encoder of the media codec to re-encode said decoded media stream from the decoder subsystem;

forming a closed loop by looping the re-encoded media stream back into the decoder subsystem for decoding the re-encoded media stream; and

identifying maximal encode parameters for the encoder of the media codec corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.

12. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system for generating a representation of a digital circuit from definitions of circuit elements and data defining rules for combining those circuit elements, cause at least one processor of the computer system to generate a data processing system for calibrating a media codec having an encoder configured to perform encoding in accordance with one or more encode parameters and operable to encode a media stream comprising a sequence of time-stamped frames, the data processing system comprising:

a decoder subsystem; and

a calibration monitor operable in a calibration mode of the media codec to form a closed loop between the decoder subsystem and the encoder of the media codec by;

causing the decoder subsystem to decode an encoded media stream so as to generate a decoded media stream and apply said decoded media stream as an input to the encoder of said media codec;

causing the encoder of the media codec to re-encode said decoded media stream from said decoder subsystem; and

causing the re-encoded media stream to loop back into the decoder subsystem for decoding the re-encoded media stream;

the calibration monitor being configured to, in the calibration mode, identify maximal encode parameters for the encoder of said media codec corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.

13. A non-transitory computer readable storage medium having stored thereon computer executable instructions that when executed cause at least one processor to:

play out an encoded media stream to a decoder subsystem of a media codec for encoding a media stream comprising a sequence of time-stamped frames, the media codec comprising an encoder configured to perform encoding in accordance with one or more encode parameters, so as to cause the decoder subsystem to decode the encoded media stream and generate a decoded media stream;

apply said decoded media stream an input to the encoder of said media codec;

cause the encoder of said media codec to re-encode said decoded media stream from said decoder subsystem;

form a closed loop by looping the re-encoded media stream back into the decoder subsystem for decoding the re-encoded media stream; and

identify maximal encode parameters for the encoder of said media codec corresponding to the greatest steady-state demand on the media codec that permits decoding of the sequence of time-stamped frames at a rate in accordance with their associated timestamps.