Encoder pre-analyser

• US 10,602,163 B2
• Filed: 12/28/2017
• Issued: 03/24/2020
• Est. Priority Date: 05/06/2016
• Status: Active Grant

• Current Assignee: Magic Pony Technology Limited (X Holdings Corp.)
• Original Assignee: Magic Pony Technology Limited (X Holdings Corp.)
• Inventors: Van Leuven, Sebastiaan, Wang, Zehan, Bishop, Robert David
• Primary Examiner(s): Vazquez Colon, Maria E
• Application Number:
  US15/856,906
• Publication Number:
  US 20180124415A1
• Time in Patent Office: 817 Days
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/1883 the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence

Abstract

The present disclosure relates to analysing input data, prior to encoding, using one or more hierarchical algorithms. According to a first aspect, there is provided a method for producing output data using one or more input data and one or more hierarchical algorithms, comprising the steps of applying the hierarchical algorithm to the one or more input data; and producing output data to be used by an encoder; wherein one of the one or more input data is uncompressed; and wherein the output data is used to modify a decision making process associated with the encoder.

Claims

1. A method for producing output data using one or more input data and one or more hierarchical algorithms, comprising:

applying the hierarchical algorithm to the one or more input data to estimate visual artefacts in the one or more input data caused by compression, the estimation of visual artefacts indicating one or more regions for the encoder to allocate an increase in processing power to modify perceptual reconstruction of the one or more regions;

producing by the hierarchical algorithm output data that includes a saliency map to be used by an encoder; and

causing the encoder to produce encoded output from the one or more input data based on the saliency map;

wherein one of the one or more input data is uncompressed; and

wherein the output data is used to modify a decision making process associated with the encoder.

2. The method of claim 1, wherein the decision making process may be any of:

a mode decision, or a rate distortion optimisation, or a rate distortion optimised quantisation.

3. The method of claim 1, wherein the output data is any of:

an output bit stream syntax data;

binary data;

text data;

metadata;

bit allocation description;

or an intermediate format.

4. The method of claim 3, wherein the intermediate format describes the outcome of any of the decisions of the encoder and/or sub-processes executed in the encoder.

5. The method of claim 4, wherein the intermediate format describes any of a Group of Pictures structure, frame level quantisation, frame level delta quantisation, frame level bit budget, block structure of a current picture, coding unit sizes, prediction sizes, transform sizes, mode decision information, transform mode, palette mode, inter-layer prediction, Adaptive Colour Transform, Adaptive Motion Vector Resolution, Intra-Block Copy, motion vector information, delta motion vector information, inter-view prediction mode, inter-view prediction motion vector, quantisation parameters, delta quantisation parameters, and/or quantisation matrices.

6. The method of claim 1, wherein the estimation of visual artefacts results in any of a probability density function, visual distortion function, psycho-visual distortion function, bit allocation function, bit density function, energy retention function, detail retention function, and/or bit probability function.

7. The method of claim 1, wherein the hierarchical algorithm indicates any of the Group of Pictures structure, and/or delta quantisation, and/or quantisation of each frame.

8. The method of claim 1, wherein an input to the one or more hierarchical algorithms is a saliency map.

9. The method of claim 1, wherein an intermediate step of the one or more hierarchical algorithms comprises producing a saliency map.

10. The method of claim 1, wherein the one or more input data is any of pixel accurate, sub-block accurate, block-accurate, group of blocks accurate, slice accurate, tile accurate, field accurate, or frame accurate.

11. The method of claim 1, wherein the output data is any of pixel accurate, sub-block accurate, block-accurate, group of blocks accurate, slice accurate, tile accurate, field accurate, or frame accurate.

12. The method of claim 1, further comprising the step of training the one or more hierarchical algorithms.

13. The method of claim 12, wherein the step of training comprises using the uncompressed input data and a reconstructed decompressed data from the encoder.

14. The method of claim 12, wherein the step of training comprises minimising a predetermined cost function.

15. An apparatus comprising:

at least one processor; and

at least one memory including computer program code which, when executed by the at least one processor, causes the apparatus to;

apply a hierarchical algorithm to one or more input data to estimate visual artefacts in the one or more input data caused by compression, the estimation of visual artefacts indicating one or more regions for the encoder to allocate an increase in processing power to modify perceptual reconstruction of the one or more regions;

produce, by the hierarchical algorithm, output data that includes a saliency map to be used by an encoder; and

cause the encoder to produce encoded output from the one or more input data based on the saliency map;

wherein one of the one or more input data is uncompressed; and

wherein the output data is used to modify a decision making process associated with the encoder.

16. The apparatus of claim 15, wherein the estimation of visual artefacts results in any of a probability density function, visual distortion function, psycho-visual distortion function, bit allocation function, bit density function, energy retention function, detail retention function, and/or bit probability function.

17. The apparatus of claim 15, wherein an output of the hierarchical algorithm is a saliency map, and wherein the encoder interprets the saliency map.