Bit-rate control for video coding using object-of-interest data

• US 10,230,950 B2
• Filed: 05/30/2013
• Issued: 03/12/2019
• Est. Priority Date: 05/30/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Peng, Ya-Ti, Chiu, Yi-Jen, Jiang, Hong
• Primary Examiner(s): Philippe, Gims S
• Assistant Examiner(s): Hasan, Mainul
• Application Number:
  US13/906,031
• Publication Number:
  US 20140355671A1
• Time in Patent Office: 2,112 Days
• Field of Search: 37524008
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock

Abstract

Systems, articles, and methods include bit-rate control for video coding using object-of-interest data that adjusts the quality of part of an image depending on the distance of the position of a macroblock from an object in the image.

Claims

1. A computer-implemented method for video coding, comprising:

receiving image data indicating a position of at least one content object in a scene provided on an image to be displayed;

pre-setting importance levels available to form a macroblock level map of an image and that indicate a desired image quality of each level, the levels comprising at least one object level to indicate a location of a macroblock of image data within one of the objects and at least two non-object importance levels comprising one non-object importance level to be adjacent macroblocks forming one of the objects and the other non-object importance level to be spaced from the macroblocks forming one of the objects, wherein multiple macroblocks can be assigned to the same pre-set non-object level, and wherein each non-object level covers a different range of distances from the object;

generating the macroblock level map comprising;

determining a position of individual macroblocks of a plurality of macroblocks forming at least part of the image and relative to the position of the object, and

assigning one of the pre-set levels to multiple macroblocks depending on whether the position of the macroblock corresponds to the range of distances of a level; and

adjusting, by a video encoder, the quality of a part of the image provided by at least one of the macroblocks depending, at least in part, on the position of the macroblock relative to the position of the object on the image and comprising;

adjusting a filter strength of a filter by using the macroblock level map and by an adjustment depending on the level of the macroblock, wherein a same adjustment value of the filter strength is applied to all image data within the same macroblock and varies from macroblocks in one level to macroblocks in another level,

filtering the number of bits associated with a macroblock and by using the adjusted filter strength to reduce the number of bits associated with the macroblock after an encoder partitions the image into macroblocks but before providing the macroblock to transform and quantize data of the macroblock, and

setting a scaling factor to be used to calculate a quantization parameter (QP) assigned to a macroblock by using the macroblock level map used to adjust the filter strength.

2. The method of claim 1 comprising determining which of the macroblocks are non-object macroblocks that do not form a part of the object, and wherein adjusting comprises adjusting the image quality for the non-object macroblocks.

3. The method of claim 1 wherein adjusting comprises adjusting the quality of the part of the image for a macroblock depending, at least in part, on a distance from the macroblock to a point defined relative to the object.

4. The method of claim 1 wherein adjusting comprises adjusting the quality of the part of the image depending, at least in part, on a distance from a macroblock to at least one of:

a center point and a centroid of the object.

5. The method of claim 1 wherein each level being associated with at least one different distance from a point defined relative to the object.

6. The method of claim 1 comprising associating each of a plurality of the macroblocks with at least one of a plurality of the levels, each level being associated with at least one different distance from a point defined relative to the object, and wherein at least two levels are provided for labeling non-object macroblocks.

7. The method of claim 1 comprising associating each of a plurality of the macroblocks with at least one of a plurality of the levels, each level being associated with at least one different distance from a point defined relative to the object, and wherein at least one level is disposed adjacent to macroblocks having an outer boundary of the object, and at least one other level is spaced from macroblocks that display the object.

8. The method of claim 1 wherein adjusting a filter strength to reduce the number of bits associated with a macroblock comprises obtaining the level of a macroblock;

and adjusting the filter strength to be stronger as the macroblock is farther from the object as indicated by the level.

9. The method of claim 1, comprising determining which of the macroblocks are non-object macroblocks that do not form a part of the object, and wherein adjusting comprises adjusting the image quality for the non-object macroblocks,

wherein adjusting comprises adjusting the quality of the part of the image for a macroblock depending, at least in part, on a distance from the macroblock to a point defined relative to the object,

wherein adjusting comprises adjusting the quality of the part of the image depending, at least in part, on a distance from a macroblock to at least one of:

a center point and a centroid of the object,

wherein each level being associated with at least one different distance from a point defined relative to the object,

wherein at least one level is disposed adjacent to macroblocks having an outer boundary of the object, and at least one other level is spaced from macroblocks that display the object,

wherein adjusting a filter strength to reduce the number of bits associated with a macroblock comprises obtaining the level of a macroblock; and

adjusting the filter strength to be stronger as the macroblock is farther from the object as indicated by the level;

wherein adjusting the quality depends, at least in part, on a distance from at least one macroblock to a center point of a group of objects on the image, and

wherein the object is a human face.

22. The method of claim 1 wherein the filter is a low pass filter that eliminates high frequency data of a macroblock depending on the strength of the filter as modified depending on the position of the macroblock.

10. A system for video coding on a computer:

a display;

at least one processor communicatively coupled to the display;

at least one memory communicatively coupled to the processor;

a macroblock level map generator communicatively coupled to the processor and configured to;

receive image data indicating a position of at least one content object in a scene provided on an image to be displayed,

pre-set importance levels available to form a macroblock level map of an image and that indicate a desired image quality of each level, the levels comprising at least one object level to indicate a location of a macroblock of image data within one of the objects and at least two non-object importance levels comprising one non-object importance level to be adjacent macroblocks forming one of the objects and the other non-object importance level to be spaced from the macroblocks forming one of the objects, wherein multiple macroblocks can be assigned to the same pre-set non-object level, and wherein each non-object level covers a different range of distances from the object, and

generate a macroblock level map comprising;

determining a position of individual macroblocks of a plurality of macroblocks forming at least part of the image, and

assigning one of the pre-set levels to multiple macroblocks depending on whether the position of the macroblock corresponds to the range of distances of a level; and

a video rate controller communicatively coupled to the processor and configured to adjust the quality of a part of the image provided by at least one of the macroblocks depending, at least in part, on the position of the macroblock relative to the position of the object on the image and comprises;

adjusting a filter strength of a filter by using the macroblock level map and by an adjustment depending on the level of the macroblock, wherein a same adjustment value of the filter strength is applied to all image data within the same macroblock and varies from macroblocks in one level to macroblocks in another level,

filtering the number of bits associated with a macroblock and by using the adjusted filter strength to reduce the number of bits associated with the macroblock after an encoder partitions the image into macroblocks but before providing the macroblock to transform and quantize data of the macroblock, and

setting a scaling factor to be used to calculate a quantization parameter (QP) assigned to a macroblock by using the macroblock level map used to adjust the filter strength.

11. The system of claim 10 wherein the macroblock level map generator is configured to determine which of the macroblocks are non-object macroblocks that do not form a part of the object, and wherein the video rate controller is configured to adjust the image quality for the non-object macroblocks.

12. The system of claim 10 wherein the video rate controller is configured to adjust the quality of the part of the image for a macroblock depending, at least in part, on a distance from the macroblock to a point defined relative to the object.

13. The system of claim 10 wherein the video rate controller is configured to adjust the quality of the part of the image depending, at least in part, on a distance from a macroblock to at least one of:

a center point and a centroid of the object.

14. The system of claim 10 wherein each level being associated with at least one different distance from a point defined relative to the object.

15. The system of claim 10 wherein each level being associated with at least one different distance from a point defined relative to the object, and wherein at least two levels are provided for labeling non-object macroblocks.

16. The system of claim 10 wherein each level being associated with at least one different distance from a point defined relative to the object, and wherein at least one level is disposed adjacent to macroblocks having an outer boundary of the object, and at least one other level is spaced from macroblocks that display the object.

17. The system of claim 10 wherein the video rate controller operates by adjusting the filter strength to be stronger as the macroblock is farther from the object as indicated by the level.

18. The system of claim 10 wherein the video rate controller is configured to adjust the quality of the image depending, at least in part, on a distance from at least one macroblock to a center point of a group of objects on the image.

19. The system of claim 10 wherein the macroblock level map generator determines which of the macroblocks are non-object macroblocks that do not form a part of the object,

wherein the video rate controller is configured to adjust the image quality for the non-object macroblocks,

wherein the video rate controller is configured to adjust the quality of the part of the image for a macroblock depending, at least in part, on a distance from the macroblock to a point defined relative to the object,

wherein the video rate controller is configured to adjust the quality of the part of the image depending, at least in part, on a distance from a macroblock to at least one of:

a center point and a centroid of the object,

wherein each of the levels being associated with at least one different distance from a point defined relative to the object,

wherein at least one of the levels is disposed adjacent to macroblocks having an outer boundary of the object, and at least one other level is spaced from macroblocks that display the object,

wherein the bit rate controller operates by adjusting the filter strength to be stronger as the macroblock is farther from the object as indicated by the level,

and

wherein the video rate controller is configured to adjust the quality of the image depending, at least in part, on a distance from at least one macroblock to a center point of a group of objects on the image.

23. The system of claim 10 wherein the filter is a low pass filter that eliminates high frequency data of a macroblock depending on the strength of the filter as modified depending on the position of the macroblock.

20. At least one non-transitory article for video coding having stored therein a computer program having instructions that when executed, cause the computer to:

access image data indicating a position of at least one content object in a scene provided on an image to be displayed;

pre-set importance levels available to form a macroblock level map of an image and that indicate a desired image quality of each level the levels comprising at least one object level to indicate a location of a macroblock of image data within one of the objects and at least two non-object importance levels comprising one non-object importance level to be adjacent macroblocks forming one of the objects and the other non-object importance level to be spaced from the macroblocks forming one of the objects, wherein multiple macroblocks can be assigned to the same pre-set non-object level and wherein each non-object level covers a different range of distances from the object;

generate a macroblock level map comprising determining a position of individual macroblocks of a plurality of macroblocks forming at least part of the image comprising assigning a level to individual macroblocks that indicate a position of the macroblock relative to the position of the object, wherein each level includes a range of distances from the object, and wherein each macroblock has image data of a plurality of pixels; and

adjust, by a video encoder, the quality of a part of the image provided by at least one of the macroblocks depending, at least in part, on the position of the macroblock relative to the position of the object on the image and comprising;

adjust a filter strength of a filter by using the macroblock level map and by an adjustment depending on the level of the macroblock, wherein a same adjustment value of the filter strength is applied to all image data within the same macroblock and varies from macroblocks in one level to macroblocks in another level,

provide the option to perform either;

(1) filter the number of bits associated with a macroblock and by using the adjusted filter strength to reduce the number of bits associated with the macroblock after an encoder partitioned the image into macroblocks but before providing the macroblock to transform and quantize data of the macroblock, or

(2) set a scaling factor to be used to calculate a quantization parameter (QP) assigned to a macroblock by using the macroblock level map used to adjust the filter strength, or

(3) both (1) and (2);

wherein the computing device provides the option to alternatively perform options (1), (2), and (3).

21. The article of claim 20 comprising instructions that when executed, cause the computer to:

determine which of the macroblocks are non-object macroblocks that do not form a part of the object, and wherein adjust comprises adjust the image quality for the non-object macroblocks,

wherein adjust the quality comprises adjusting the quality of the part of the image for a macroblock depending, at least in part, on a distance from the macroblock to a point defined relative to the object,

wherein adjust the quality comprises adjusting the quality of the part of the image depending, at least in part, on a distance from a macroblock to at least one of;

a center point and a centroid of the object,

associate individual macroblocks of a plurality of the macroblocks with at least one of a plurality of the levels, each level being associated with at least one different distance from a point defined relative to the object,

wherein at least one level is disposed adjacent to macroblocks having an outer boundary of the object, and at least one other level is spaced from macroblocks that display the object,

wherein adjusting a filter strength to reduce the number of bits associated with a macroblock comprises adjusting the filter strength to be stronger as the macroblock is farther from the object as indicated by the level, wherein the filter is a low pass filter that eliminates high frequency data of a macroblock depending on the strength of the filter as modified depending on the position of the macroblock,

wherein adjust the quality depends, at least in part, on a distance from at least one macroblock to a center point of a group of objects on the image, and

wherein the object is a human face.