Upsampling and downsampling of motion maps and other auxiliary maps in a tiered signal quality hierarchy

• US 9,967,568 B2
• Filed: 04/05/2017
• Issued: 05/08/2018
• Est. Priority Date: 11/10/2011
• Status: Active Grant

• Current Assignee: Time Warner Cable Enterprises LLC (Charter Communications Incorporated)
• Original Assignee: Nova International Limited
• Inventors: Rossato, Luca, Meardi, Guido
• Primary Examiner(s): Werner, David N
• Application Number:
  US15/479,966
• Publication Number:
  US 20170208333A1
• Time in Patent Office: 398 Days
• Field of Search: None
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Certain configurations herein include changing the resolution of an auxiliary map (e.g., a motion map, a z-map, etc.) at a first level of quality to obtain an auxiliary map at a second level of quality. For example, changing the resolution can include receiving a respective auxiliary map of one or more vectors at one or more lower levels of quality and progressively refining, via novel operations, the auxiliary map to higher or lower levels of quality in a hierarchy.

Claims

1. A method comprising:

via computer processor hardware, performing image-processing operations of;

receiving a first map of motion vectors defining image motion, the first map of motion vectors defined at a first resolution;

selecting a motion vector from the first map of motion vectors;

identifying a group of motion vectors in the first map of motion vectors, the group including the selected motion vector in the first map of motion vectors;

utilizing the identified group of motion vectors to derive a setting for a given motion vector in a second map of motion vectors, the second map of motion vectors defined at a second resolution, the second resolution higher than the first resolution; and

applying residual data to the second map of motion vectors to adjust settings of corresponding motion vectors in the second map of motion vectors.

2. The method as in claim 1 further comprising:

subdividing the selected motion vector into multiple sub-elements, the given motion vector in the second map of motion vectors being one of the multiple sub-elements of the selected motion vector, each of the multiple sub-elements in the second map of motion vectors being a respective motion vector.

3. The method as in claim 1 further comprising:

generating the setting for the given motion vector to indicate motion to a fraction of an image element at the second resolution.

4. The method as in claim 1, wherein the second map of motion vectors supports a process of reconstructing an image signal based on reference image information at the second resolution.

5. The method as in claim 1, wherein utilizing the identified group of motion vectors to derive the setting for the given motion vector in the second map of motion vectors includes:

upsampling the first map of motion vectors to produce the second map of motion vectors.

6. The method as in claim 1, wherein the identified group of motion vectors includes a particular motion vector from the first map of motion vectors, the particular motion vector being contiguous with respect to the selected motion vector from the first map of motion vectors.

7. The method as in claim 1, wherein identifying the group of motion vectors in the first map of motion vectors includes:

selecting motion vectors in a vicinity of the selected motion vector for inclusion in the group.

8. The method as in claim 1 further comprising:

upsampling the second map of motion vectors into a third map of motion vectors.

9. The method as in claim 8, wherein the third map of motion vectors is defined at a third resolution, the third resolution greater than the second resolution.

10. The method as in claim 9 further comprising:

applying residual data to the third map of motion vectors to adjust settings of corresponding motion vectors in the third map of motion vectors.

11. The method as in claim 1, wherein the first map of motion vectors is applicable to a respective first level of quality of a current image;

and

wherein the second map of motion vectors is applicable to a respective second level of quality of the current image, the second level of quality greater than the first level of quality.

12. A system comprising:

computer processor hardware; and

a hardware storage resource coupled to communicate with the computer processor hardware, the hardware storage resource storing instructions that, when executed by the computer processor hardware, causes the computer processor hardware to;

receive a first map of motion vectors defining image motion, the first map of motion vectors defined at a first resolution;

select a motion vector from the first map of motion vectors;

identify a group of motion vectors in the first map of motion vectors, the group including the selected motion vector in the first map of motion vectors, the second map of motion vectors defined at a second resolution, the second resolution higher than the first resolution;

utilize the identified group of motion vectors to derive a setting for a given motion vector in a second map of motion vectors; and

apply residual data to the second map of motion vectors to adjust settings of corresponding motion vectors in the second map of motion vectors.

13. The system as in claim 12, wherein execution of the instructions further causes the computer processor hardware to:

subdivide the selected motion vector into multiple sub-elements, the given motion vector in the second map of motion vectors being one of the multiple sub-elements of the selected motion vector, each of the multiple sub-elements in the second map of motion vectors being a respective motion vector.

14. The system as in claim 12, wherein execution of the instructions further causes the computer processor hardware to:

generate the setting for the given motion vector to indicate motion to a fraction of an image element at the second resolution.

15. The system as in claim 12, wherein the second map of motion vectors supports a process of reconstructing an image signal based on reference image information at the second resolution.

16. The system as in claim 12, wherein execution of the instructions further causes the computer processor hardware to:

upsample the first map of motion vectors to produce the second map of motion vectors.

17. The system as in claim 12, wherein the identified group of motion vectors includes a particular motion vector from the first map of motion vectors, the particular motion vector being contiguous with respect to the selected motion vector from the first map of motion vectors.

18. The system as in claim 12, wherein execution of the instructions further causes the computer processor hardware to:

select motion vectors in a vicinity of the selected motion vector for inclusion in the group.

19. Computer-readable storage hardware having instructions stored thereon, the instructions, when carried out by computer processor hardware, causing the computer processor hardware to:

receive a first map of motion vectors defining image motion, the first map of motion vectors defined at a first resolution;

select a motion vector from the first map of motion vectors;

identify a group of motion vectors in the first map of motion vectors, the group including the selected motion vector in the first map of motion vectors;

utilize the identified group of motion vectors to derive a setting for a given motion vector in a second map of motion vectors, the second map of motion vectors defined at a second resolution, the second resolution higher than the first resolution; and

apply residual data to the second map of motion vectors to adjust settings of corresponding motion vectors in the second map of motion vectors.