Video encoding

• US 10,542,277 B2
• Filed: 10/24/2017
• Issued: 01/21/2020
• Est. Priority Date: 10/24/2017
• Status: Active Grant

• Current Assignee: Arm Ltd. (United Kingdom) (SoftBank Group Corp.)
• Original Assignee: Arm Ltd. (United Kingdom) (SoftBank Group Corp.)
• Inventors: Stolt, Fredrik Peter, Bjorklund, John Nils Andreas
• Primary Examiner(s): Beck, Leron
• Application Number:
  US15/792,231
• Publication Number:
  US 20190124347A1
• Time in Patent Office: 819 Days
• Field of Search: 37524001-24029
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques

Abstract

A method of video encoding is provided. The method includes downscaling a block of a frame of a video to generate a downscaled block. The method includes downscaling a reference block of a reference frame of the video to generate a downscaled reference block. The method includes processing the downscaled block to calculate first compression efficacy data indicative of a first efficacy of encoding the block using intra-frame prediction. The method includes processing the downscaled block and the downscaled reference block to calculate second compression efficacy data indicative of a second efficacy of encoding the block using inter-frame prediction. The method includes based on at least one of the first compression efficacy data or the second compression efficacy data, determining that the block is to be encoded using inter-frame prediction. Other methods of video encoding and a video encoder system are also provided.

Claims

1. A method of video encoding comprising:

downscaling a block of a frame of a video to generate a downscaled block;

processing the downscaled block to generate compression efficacy data indicative of an efficacy of encoding the block using inter-frame prediction;

based on the compression efficacy data, determining that the block is to be encoded using inter-frame prediction;

encoding the block using inter-frame prediction; and

downscaling a reference block of a reference frame of the video to generate a downscaled reference block, wherein;

the processing the downscaled block comprises processing the downscaled block with the downscaled reference block to calculate motion data representative of an estimated motion between the downscaled block and the downscaled reference block; and

the compression efficacy data is representative of a motion comparison between the estimated motion and a predicted motion.

2. The method according to claim 1, comprising, after the determining that the block is to be encoded using inter-frame prediction, encoding the block using inter-frame prediction without generating an intra-frame predicted block based on the block.

3. The method according to claim 1, comprising, after the determining that the block is to be encoded using inter-frame prediction:

generating, using inter-frame prediction, a plurality of inter-frame predicted blocks, each using different motion data representative of different estimated motion between the inter-frame predicted block and a reference block of the reference frame;

selecting motion data corresponding to an inter-frame predicted block of the plurality of inter-frame predicted blocks; and

encoding the block relative to the reference block using the selected motion data.

4. The method according to claim 1 wherein the predicted motion is between a further downscaled block of the frame and a corresponding further downscaled reference block of the reference frame.

5. A method of video encoding comprising:

downscaling a block of a frame of a video to generate a downscaled block;

processing the downscaled block to generate compression efficacy data indicative of an efficacy of encoding the block using inter-frame prediction;

based on the compression efficacy data, determining that the block is to be encoded using inter-frame prediction;

encoding the block using inter-frame prediction;

downscaling a reference block of a reference frame of the video to generate a downscaled reference block, wherein;

the processing the downscaled block comprises;

generating, using inter-frame prediction, an inter-frame predicted downscaled block based on the downscaled block and the downscaled reference block; and

the compression efficacy data is representative of an inter-frame prediction comparison between the inter-frame predicted downscaled block and the downscaled block;

the inter-frame prediction comparison is representative of an inter-frame prediction difference between the inter-frame predicted downscaled block and the downscaled block; and

the determining that the block is to be encoded relative to the reference block comprises determining that the inter-frame prediction difference is equal to or smaller than a threshold inter-frame prediction difference.

6. The method according to claim 5, wherein:

the processing the downscaled block comprises;

generating, using intra-frame prediction, an intra-frame predicted downscaled block based on the downscaled block; and

wherein the compression efficacy data is based on a comparison calculated using the intra-frame predicted downscaled block and the inter-frame predicted downscaled block.

7. The method according to claim 6, comprising calculating the comparison by comparing:

a first difference between the intra-frame predicted downscaled block and the downscaled block; and

a second difference between the inter-frame predicted downscaled block and the downscaled block.

8. The method according to claim 7, wherein the determining that the block is to be encoded using inter-frame prediction comprises determining that a ratio of the second difference to the first difference is equal to or smaller than a threshold ratio.

9. The method according to claim 6, wherein the generating the intra-frame predicted downscaled block comprises calculating a value of the intra-frame predicted downscaled block based on a respective value of at least one other previously encoded and reconstructed downscaled pixel of the frame.

10. The method according to claim 6, wherein the generating the inter-frame predicted downscaled block comprises transforming the reference block using motion data representative of an estimated motion between the downscaled block and the reference block.

11. The method according to claim 5,

comprising, after the determining that the block is to be encoded using inter-frame prediction, encoding the block using inter-frame prediction without generating an intra-frame predicted block based on the block.

12. The method according to claim 5, wherein:

the processing the downscaled block comprises processing the downscaled block with the downscaled reference block to calculate motion data representative of an estimated motion between the downscaled block and the downscaled reference block; and

the compression efficacy data is representative of a motion comparison between the estimated motion and a predicted motion.

13. The method according to claim 12, wherein the predicted motion is between a further downscaled block of the frame and a corresponding further downscaled reference block of the reference frame.

14. The method according to claim 5, comprising, after the determining that the block is to be encoded using inter-frame prediction:

generating, using inter-frame prediction, a plurality of inter-frame predicted blocks, each using different motion data representative of different estimated motion between the inter-frame predicted block and a reference block of the reference frame;

selecting motion data corresponding to an inter-frame predicted block of the plurality of inter-frame predicted blocks; and

encoding the block relative to the reference block using the selected motion data.

15. The method according to claim 14, comprising performing the generating, the selecting and the encoding without using intra-frame prediction.

16. A video encoder system comprising:

storage for storing a frame of a video;

at least one processor communicatively coupled to the storage; and

an encoder module configured to;

downscale a block of the frame to generate a downscaled block;

process the downscaled block to generate compression efficacy data indicative of an efficacy of encoding the block using inter-frame prediction;

based on the compression efficacy data, determine that the block is to be encoded using inter-frame prediction;

encode the block using inter-frame prediction; and

downscale a reference block of a reference frame of the video to generate a downscaled reference block, wherein;

the processing the downscaled block comprises;

generating, using intra-frame prediction, an intra-frame predicted downscaled block based on the downscaled block; and

generating, using inter-frame prediction, an inter-frame predicted downscaled block based on the downscaled block and the downscaled reference block, and wherein;

the compression efficacy data is based on a comparison calculated using the intra-frame predicted downscaled block and the inter-frame predicted downscaled block, wherein;

the comparison is calculated by comparing;

a first difference between the intra-frame predicted downscaled block and the downscaled block; and

a second difference between the inter-frame predicted downscaled block and the downscaled block, and wherein,

the determining that the block is to be encoded using inter-frame prediction comprises determining that a ratio of the second difference to the first difference is equal to or smaller than a threshold ratio.

17. The video encoder system according to claim 16, wherein the encoder module comprises a motion estimation module configured to perform the downscaling, the processing and the determining.

18. The video encoder system according to claim 17, wherein the motion estimation module is further configured to:

process the downscaled block with the downscaled reference block to calculate motion data representative of an estimated motion between the downscaled block and the downscaled reference block.

19. The video encoder system according to claim 16, wherein:

the generating, using the intra-frame prediction, the intra-frame predicted downscaled block comprises calculating a value of the intra-frame predicted downscaled block based on a respective value of at least one other previously encoded and reconstructed downscaled pixel of the frame.

20. The video encoder system according to claim 16, wherein the generating the inter-frame predicted downscaled block comprises transforming the reference block using motion data representative of an estimated motion between the downscaled block and the reference block.