PROCESSING OF MOTION INFORMATION IN MULTIDIMENSIONAL SIGNALS THROUGH MOTION ZONES AND AUXILIARY INFORMATION THROUGH AUXILIARY ZONES

• US 20170310968A1
• Filed: 07/06/2017
• Published: 10/26/2017
• Est. Priority Date: 05/14/2012
• Status: Active Grant

• Current Assignee: Nova International Limited
• Original Assignee: Nova International Limited
• Inventors: Rossato, Luca, Meardi, Guido
• Granted Patent: US 10,750,178 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06T 9/40 Tree coding, e.g. quadtree, octree
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/62 by frequency transforming in three dimensions H04N19/63 takes precedence
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 19/94 Vector quantisation
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Computer processor hardware receives zone information specifying multiple elements of a rendition of a signal belonging to a zone. The computer processor hardware also receives motion information associated with the zone. The motion information can be encoded to indicate to which corresponding element in a reference signal each of the multiple elements in the zone pertains. For each respective element in the zone as specified by the zone information, the computer processor hardware utilizes the motion information to derive a corresponding location value in the reference signal; the corresponding location value indicates a location in the reference signal to which the respective element pertains.

Claims

1. A method comprising:

receiving zone information specifying multiple image elements in a rendition of an image being reproduced, the multiple image elements belonging to a zone in the rendition of the image;

receiving motion information associated with the zone; and

for each respective image element in the zone, utilizing the motion information to derive a corresponding location value in a reference image for the respective image element, the corresponding location value indicating a location in the reference image to which the respective image element pertains.

2. The method as in claim 1, wherein the motion information is encoded to indicate to which corresponding at least one image element in a reference image the respective image element in the zone pertains.

3. The method as in claim 1 further comprising:

selecting a first image element of the multiple image elements in the zone;

utilizing the motion information to calculate a first coordinate location in the reference image;

utilizing the first coordinate location as a basis to identify a first set of image elements in the reference image;

obtaining settings of the image elements in the first set;

calculating a first setting, the first setting calculated based on the settings of the elements in the first set;

assigning the first setting to the first image element in the rendition of the image being reproduced;

selecting a second image element in the zone;

utilizing the motion information to calculate a second coordinate location in the reference image;

utilizing the second coordinate location as a basis to identify a second set of elements in the reference image;

obtaining settings of elements in the second set;

calculating a second setting based on the settings of the elements in the second set; and

assigning the second setting to the second element in the rendition of the image being reproduced.

4. The method as in claim 1, wherein the motion information includes a set of coefficient values;

and

wherein utilizing the motion information to derive the corresponding location value for the respective image element includes;

i) obtaining coordinates for the respective image element, the coordinates specifying a location of the respective image element in the rendition of the image being reproduced; and

ii) performing linear combinations of the coordinates of the respective image element with the coefficient values to produce the corresponding location value.

5. The method as in claim 4, wherein the corresponding location value specifies coordinates in the reference image to which the respective element pertains.

6. The method as in claim 2, wherein the at least one image element includes a group of multiple image elements representing an object in the reference image;

and

wherein the motion information captures motion of the object from a location in the reference image to a location in the rendition of the image being reproduced.

7. The method as in claim 1, wherein the rendition of the image being reproduced is a motion compensated image;

and

wherein the zone in the motion compensated image includes multiple contiguous regions of image elements including a first region of image elements and a second region of image elements, the first region of image elements being non-contiguous with respect to the second region of image elements.

8. The method as in claim 1, wherein a resolution of the rendition of the image being reproduced is substantially different than a resolution of the reference image.

9. The method as in claim 1, wherein a resolution of the rendition of the image being reproduced is substantially the same as a resolution of the reference image.

10. The method as in claim 1, wherein the motion information includes offset adjustment information;

and

wherein utilizing the motion information to derive a corresponding location value for the respective element includes;

i) obtaining coordinates for the respective element, the coordinates specifying a location of the respective element in the rendition of the image being reproduced; and

ii) applying the offset adjustment information to the coordinates to produce the corresponding location information for the respective element, the corresponding location information indicating a location that is offset with respect to the coordinates by an amount as specified by the offset adjustment information.

11. The method as in claim 1, wherein the motion information includes rotation adjustment information;

and

wherein utilizing the motion information to derive a corresponding location value for the respective image element includes;

i) obtaining coordinates for the respective image element, the coordinates specifying a location of the respective image element in the rendition of the image being reproduced; and

ii) applying the rotation adjustment information to the coordinates to produce the corresponding location information for the respective image element, the corresponding location information capturing a rotation of the coordinates by an amount as specified by the rotation adjustment information.

12. The method as in claim 1, wherein the motion information indicates how the multiple image elements in the zone of the rendition of the image are scaled from a corresponding grouping of display elements in the reference image.

13. The method as in claim 1, wherein the corresponding location value has a sufficiently high resolution to specify a fraction of an image element in the reference image.

14. The method as in claim 1, wherein the zone information includes geometric parameters defining a shape of the zone.

15. The method as in claim 1 further comprising:

reproducing the rendition of the image based at least in part on the reference image in a manner as specified by the zone information and the motion information;

receiving reconstruction data specifying adjustments to be applied to the rendition of the image being reproduced; and

applying the adjustments to display elements in the rendition of the image being reproduced as specified by the reconstruction data to refine the reproduced rendition of the image.

16. The method as in claim 1, wherein the rendition of the image is a first image and wherein the reference image is a first reference image from which the first image is at least partially derived, wherein the zone information is first zone information, wherein the motion information is first motion information, the method further comprising:

receiving second zone information and second motion information specifying how to reproduce a second image based on a second reference image;

in accordance with the second zone information and second motion information, reproducing a rendition of the second image based at least in part on the second reference image; and

generating a rendition of a third image using a combination of the first image and the second image as reference images.

17. The method as in claim 1, wherein the zone information is first zone information and wherein the zone is a first zone, the first zone information indicating that a given element resides in the first zone, the method further comprising:

receiving second zone information associated with the rendition of the image, the second zone information indicating that the given element resides in a second zone.

18. The method as in claim 1, wherein the motion information indicates that a particular element in the rendition of the image being reproduced belongs to a motion zone, the particular element assigned a value not based on settings of a group of elements in the reference image.

19. The method as in claim 1, wherein receiving the zone information includes receiving a plane of elements, the elements in the plane specifying attributes associated with corresponding elements in the rendition of the image, a respective setting of each of the elements in the plane including zone data specifying whether a corresponding image element in the rendition of the image is included in the zone.

20. The method as in claim 19 further comprising:

producing the plane of elements in accordance with a tier-based hierarchical decoding method, the tier-based hierarchical decoding method comprising;

decoding a rendition of the plane of elements at a first level of quality;

based at least in part on the rendition of the plane of elements at the first level of quality, deriving a preliminary rendition of the plane of elements at a second level of quality, the second level of quality having a higher resolution than the first level of quality;

obtaining a set of adjustment values associated with the preliminary rendition of the plane of elements at the second level of quality;

applying the adjustment values to the preliminary rendition of the plane of elements at the second level of quality to produce an adjusted rendition of the plane of elements at the second level of quality; and

utilizing the adjusted rendition of the plane of elements at the second level of quality to identify the multiple elements of the rendition of the image that belong to the zone.

21. The method as in claim 20 further comprising:

producing the plane of elements at a first level of quality;

assigning each element that resides in a first portion of the plane a final symbol;

assigning each element that resides in a second portion of the plane a non-final symbol; and

wherein assignment of the final symbol indicates that every child element derived from a respective parent element in the first portion inherits same zone settings as the respective parent element.

23. The method as in claim 1, wherein the motion information is obtained from a motion map at a second level of quality, the motion map at the second level of quality upsampled from a motion map at a first level of quality, the second level of quality being higher in quality than first level of quality.

22. The method as in 1, wherein the corresponding at least one image element of the reference image include multiple display elements defined by a display grid;

and

wherein the corresponding location value for the respective image element in the zone specifies an off-grid image element location with respect to the corresponding at least one image element of the reference image in the display grid.