Region-of-Interest Aware Video Coding

• US 20150016510A1
• Filed: 09/26/2013
• Published: 01/15/2015
• Est. Priority Date: 07/10/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Carlsson, Pontus, Hemmendorff, Magnus, Hofbauer, Konrad, Nikiforov, Sergei, Zhao, David
• Granted Patent: US 9,167,255 B2
• Time in Patent Office: Days
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value

Abstract

An encoder comprising an encoding module and an adaptation module. The encoding module is configured to encode video in at least one region-of-interest and outside the region-of-interest. The encoding comprises quantization, and the encoding module is operable to apply a difference in quantization granularity between the encoding inside and outside the region-of-interest. The adaptation module is configured to determine at least one metric representative of a difference in benefit of the quantization inside and outside the region-of-interest, and to adapt the difference in quantization granularity in dependence on this metric.

Claims

1. An encoder comprising:

an encoding module configured to encode video in at least one region-of-interest and outside the region-of-interest, the encoding comprising quantization and the encoding module being operable to apply a difference in quantization granularity between the encoding inside and outside the region-of-interest; and

an adaptation module configured to determine at least one metric representative of a difference in benefit of the quantization inside and outside the region-of-interest, and to adapt the difference in quantization granularity in dependence on said metric.

2. The encoder of claim 1, wherein the metric comprises an indication of bitrate incurred by encoding inside the region-of-interest relative to bitrate incurred by encoding outside, and the adaptation module is configured to perform said adaptation based on said indication of relative bitrate.

3. The encoder of claim 2, wherein the determination of said indication comprises determining whether the bitrate incurred in the region-of-interest is above a target amount greater than the bitrate incurred outside the region-of-interest.

4. The encoder of claim 3, wherein the adaptation module is configured to decrease the difference in quantization granularity on condition of detecting that the bitrate inside the region-of-interest is above the target, and/or to increase the difference in quantization granularity on condition of detecting that the bitrate inside the region-of-interest is below the target.

5. The encoder of claim 4, wherein the adaptation module is configured to test the threshold at a series of discrete times, and to decrement the difference in quantization granularity by a predetermined step each time the bitrate inside the region-of-interest is detected as being above the target, and/or to increment the difference in quantization granularity by a predetermined step each time the bitrate inside the region-of-interest is detected as being below the target.

6. The encoder of claim 5, wherein the discrete times are one per frame.

7. The encoder of claim 3, wherein the amount by which the target is greater than the bitrate outside region-of-interest is a function of the bitrate outside the region-of-interest.

8. The encoder of claim 7, wherein the target is K times the bitrate outside the region-of-interest where K is a constant.

9. The encoder of claim 8, where K is substantially equal to 4.

10. The encoder of claim 1, wherein the adaptation module is configured to adapt the quantization granularity by adjusting a quantisation offset parameter and the offset parameter is negative such that increasing the offset parameter decreases the difference in granularity.

11. The encoder of claim 1, wherein the adaptation module is configured to cap the difference in quantization granularity to a maximum difference.

12. The encoder of claim 1, wherein the adaptation module is configured to:

determine a measure of the bitrate incurred by encoding in the region-of-interest, and determine a measure of the bitrate incurred by encoding outside the region-of-interest, and wherein the determination of said indication comprises comparing the measures to determine the indication

13. The encoder of claim 12, wherein the measure of bitrate is an average number of bits per pixel.

14. The encoder of claim 12, wherein the measure is determined over the whole region-of-interest and/or the whole region outside the region-of-interest.

15. The encoder of claim 1, wherein the encoding module is configured to encode in multiple regions-of-interest, and the metric is representative of a difference in benefit of the quantization inside the regions-of-interest relative to outside.

16. The encoder of claim 1, comprising a bitrate controller configured to set an overall quantization granularity, and the adaptation module is configured to adapt the difference in quantization granularity and a quantization granularity inside or outside the region-of-interest to obtain an overall output bitrate substantially corresponding to the overall quantization granularity target.

17. The encoder of claim 1, wherein the encoder is arranged to output the encoded video in a live video stream.

18. The encoder of claim 1, wherein the encoder is arranged to output the encoded video over a network.

19. A computer program product comprising code embodied on a computer-readable storage medium and configured so as when executed on a processing apparatus to perform operations of:

encoding video in at least one region-of-interest and outside the region-of-interest, the encoding comprising quantization and the encoding module being operable to apply a difference in quantization granularity between the encoding inside and outside the region-of-interest;

determining at least one metric representative of a difference in benefit of the quantization inside and outside the region-of-interest; and

adapting the difference in quantization granularity in dependence on said metric.

20. A method of providing an encoder comprising an encoding module for encoding video in at least one region-of-interest and outside the region-of-interest, wherein the encoding comprises quantization and the encoding module is operable to apply a difference in quantization granularity between the encoding inside and outside the region-of-interest;

the method comprising;

providing an adaptation module configured to determine at least one dynamically evaluated metric representative of a difference in benefit of the quantization inside and outside the region-of-interest, and adapt the difference in quantization granularity in dependence on said dynamically evaluated metric;

arranging said dynamically evaluated metric and said adaptation such that the behaviour of the encoder approximates that which would be achieved by optimizing a rate-distortion function based on an overall distortion metric D_p=D_ROI+w·

D_non-ROI, where D_ROI is a measure of distortion inside the region-of-interest, D_non-ROI is a measure of distortion outside the region-of-interest, and w is a weighting factor.