SEGMENTATION-BASED PARAMETERIZED MOTION MODELS

• US 20180270497A1
• Filed: 12/12/2017
• Published: 09/20/2018
• Est. Priority Date: 03/15/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Mukherjee, Debargha, Liu, Yuxin, Parker, Sarah
• Granted Patent: US 10,506,249 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/517 by encoding
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 19/547 Motion estimation performed in a transform domain
• CPC Class Codes: H04N 19/557 Motion estimation characterised by stopping computation or iteration based on certain criteria, e.g. error magnitude being too large or early exit
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

Encoding and decoding using parametrized motion models are disclosed. A method includes segmenting the video frame with respect to a reference frame resulting in a segmentation, determining a first motion vector for the current block based on the segmentation, determining a second motion vector for the current block using translational motion compensation, and encoding, for the current block, the one of the first motion vector and the second motion vector corresponding to a smaller error. The segmentation includes a segment containing the current block and a parameterized motion model for the segment. Another method includes identifying a parameterized motion model corresponding to a motion model type, associating the parameterized motion model with a segment of a reference frame, and, in response to determining that the current block is encoded using the parameterized motion model, decoding the current block using the parameterized motion model.

Claims

1. A method of encoding a current block of a video frame, comprising:

segmenting the video frame with respect to a reference frame resulting in a segmentation, the segmentation comprising a segment containing the current block and a parameterized motion model for the segment;

determining a first motion vector for the current block based on the segmentation;

determining a second motion vector for the current block using translational motion compensation; and

encoding, for the current block, the one of the first motion vector and the second motion vector corresponding to a smaller error.

2. The method of claim 1, wherein the segmenting the video frame with respect to a reference frame resulting in a segmentation comprises:

determining, using a motion model type, parameters of the parameterized motion model.

3. The method of claim 2, wherein the motion model type is selected from a set comprising a homographic motion model type, an affine motion model type, a similarity motion model type, and translational motion model type.

4. The method of claim 2, wherein the encoding the one of the first motion vector and the second motion vector comprises:

encoding the parameters of the parameterized motion model.

5. The method of claim 1, wherein the encoding the one of the first motion vector and the second motion vector comprises:

encoding a motion model type corresponding to the parameterized motion model.

6. The method of claim 1, wherein the segmenting the video frame with respect to a reference frame resulting in a segmentation comprises:

segmenting the video frame with respect to the reference frame based on one or more of a plurality of motion model types to provide a respective segmentation, the plurality of motion model types comprising a translational motion model type, a similarity motion model type, an affine motion model type, and a homographic motion model type, each of the model types having a respective complexity;

calculating a global motion error for each segmentation; and

selecting the segmentation corresponding to a least complex motion model type corresponding to global motion error below a threshold.

7. An apparatus configured to encode a current block of a video frame, the apparatus comprising:

a processor configured to execute instructions stored in a non-transitory storage medium to;

segment the video frame with respect to a reference frame resulting in a segmentation, the segmentation comprising a segment containing the current block and a parameterized motion model for the segment;

determine a first motion vector for the current block based on the segmentation;

determine a second motion vector for the current block using translational motion compensation; and

encode, for the current block, the one of the first motion vector and the second motion vector corresponding to a smaller error.

8. The apparatus of claim 7, wherein the instructions to segment the video frame with respect to a reference frame resulting in a segmentation include instructions executable by the processor to:

determine, using a motion model type, parameters of the parameterized motion model.

9. The apparatus of claim 8, wherein the motion model type is selected from a set comprising a homographic motion model type, an affine motion model type, a similarity motion model type, and translational motion model type.

10. The apparatus of claim 8, wherein the instructions to encode the one of the first motion vector and the second motion vector include instructions executable by the processor to:

encode the parameters of the parameterized motion model.

11. The apparatus of claim 7, wherein the instructions to segment the video frame with respect to a reference frame resulting in a segmentation include instructions executable by the processor to:

segment the video frame with respect to the reference frame based on one or more of a plurality of motion model types to provide a respective segmentation, the plurality of motion model types comprising a translational motion model type, a similarity motion model type, an affine motion model type, and a homographic motion model type, each of the model types having a respective complexity;

calculate a global motion error for each segmentation; and

select the segmentation corresponding to a least complex motion model type corresponding to global motion error below a threshold.

12. A method of decoding a current block of a video frame, comprising:

identifying a parameterized motion model;

associating the parameterized motion model with a segment of a reference frame; and

in response to determining that the current block is encoded using the parameterized motion model, decoding the current block using the parameterized motion model.

13. The method of claim 12, wherein the determining that the current block is encoded using the parameterized motion model comprises:

decoding, from a compressed bitstream, an indicator identifying that the current block is encoded using the parameterized motion model.

14. The method of claim 13, wherein the indicator comprises a global motion model type.

15. The method of claim 14, wherein the global motion model type indicates global motion or zero global motion.

16. The method of claim 13, further comprising:

determining parameters of the parameterized motion model based on the global motion model type.

17. The method of claim 12, wherein the determining that the current block is encoded using the parameterized motion model comprises:

decoding, from a compressed bitstream, parameters of the parameterized motion model.

18. The method of claim 12, further comprising:

in response to determining that the current block is not encoded using the parameterized motion model, decoding the current block using translational motion compensation.

19. The method of claim 18, wherein the determining that the current block is not encoded using the parameterized motion model comprises:

decoding, from a compressed bitstream, an indicator identifying that the current block is encoded using zero global motion.

20. The method of claim 12, wherein the identifying the parameterized motion model corresponding to the motion model type comprises:

decoding, from a compressed bitstream, the motion model type; and

determining parameters of the parameterized motion model based on the motion model type.