Directed interpolation and data post-processing

• US 10,194,172 B2
• Filed: 08/16/2017
• Issued: 01/29/2019
• Est. Priority Date: 04/20/2009
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Tourapis, Alexandros, Leontaris, Athanasios, Pahalawatta, Peshala V., Stec, Kevin J.
• Primary Examiner(s): Werner, David N
• Application Number:
  US15/678,982
• Publication Number:
  US 20170347125A1
• Time in Patent Office: 531 Days
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/172 image signals comprising non-image signal components, e.g. headers or format information
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/865 with detection of the former encoding block subdivision in decompressed video
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 19/895 : in combination with error concealment
• CPC Class Codes: H04N 21/4347 : Demultiplexing of several video streams
• CPC Class Codes: H04N 7/0102 : involving the resampling of the incoming video signal
• CPC Class Codes: H04N 7/0117 : involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/0135 : involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59

Abstract

An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Claims

1. A method for decoding a video bitstream, the method comprising:

receiving, at a decoder, the video bitstream comprising at least one region comprising two images, the two images being quincunx sampled and checkerboard multiplexed;

receiving, by the decoder, metadata from the video bitstream indicating a width and height of an image of the two images;

receiving first recommended information in the video bitstream identifying a first post-decoding process to be applied to the at least one region, the first post-decoding process being a spatial process;

receiving second recommended information in the video bitstream identifying second post-decoding process to be applied to the at least one region,

disregarding, by the decoder, the first recommended information without applying the first post-decoding process to the at least one region, and

applying the second post-decoding process identified by the second recommended information to the at least one region;

wherein the video bitstream further comprises a second region different from the at least one region, and

wherein the second post-decoding process is not applied to the second region.

2. The method of claim 1 wherein the second post-decoding process is related to de-blocking.

3. The method of claim 1 wherein the second post-decoding process maps the at least one region to a target device.

4. The method of claim 1 wherein the second post-decoding process was selected during studio work prior to mastering the video bitstream.

5. The method of claim 1 wherein the spatial post-decoding process is at least one of de-interleaving and upsampling.

6. A method for post-decoding processing of a video bitstream, the method comprising:

receiving, at a decoder device, the video bitstream comprising at least one region comprising two images, the two images being quincunx sampled and checkerboard multiplexed;

receiving, by the decoder device, metadata from the video bitstream indicating a width and height of an image of the two images;

receiving, by the decoder device, first recommended information in the video bitstream identifying a first post-decoding process to be applied to the at least one region, the first post-decoding process being a spatial process;

receiving, by the decoder device, second recommended information in the video bitstream identifying second post-decoding process to be applied to the at least one region;

disregarding, by the decoder device, the second recommended information without applying the second post-decoding process to the at least one region, wherein the second recommended information is disregarded based on a predetermined post-decoding process selection criterion; and

applying, by the decoder device, the first post-decoding process identified by the first recommended information to the at least one region including spatially processing the at least one region,

wherein the video bitstream further comprises a second region different from the at least one region, and

wherein the first decoding process is not applied to the second region.

7. The method of claim 6 wherein the second post-decoding process is related to de-blocking.

8. The method of claim 6 wherein the second post-decoding process maps the at least one region to a target device.

9. The method of claim 6 wherein the second post-decoding process was selected during studio work prior to mastering the video bitstream.

10. The method of claim 6 wherein the spatial post-decoding process is at least one of de-interleaving and upsampling.

11. A method for post-decoding processing of a video bitstream, comprising:

receiving, by a decoder device, the video bitstream comprising a sequence of two images, the sequence of two images being quincunx sampled and checkerboard multiplexed and each set of two images comprising at least one region within the two images;

receiving, by the decoder device, processing information in the video bitstream indicating recommended spatial and temporal processing for the video bitstream, the recommended processing adaptively changing according to a set of characteristics of the video bitstream;

selecting, by the decoder device, at a first time, a first spatial processing method for the video bitstream according to the processing information that relates to a first spatial characteristic of the video bitstream;

selecting, by the decoder device, at a second time, a second temporal processing method for the video bitstream according to the processing information that relates to a second temporal characteristic of the video bitstream; and

processing, by the decoder device, the video bitstream using the first spatial processing method based on a predetermined processing method selection criterion, wherein the first spatial processing method and the second temporal processing method are post-decoding processing methods;

wherein each set of the two images further comprises a second region different from the at least one region of the two images within the video bitstream, and

wherein the first decoding processing method is not applied to the second region.

12. The method of claim 11 wherein the first spatial processing method comprises a softer de-blocking method if the at least one region comprises a high textured region.

13. The method of claim 11 wherein the first spatial processing method comprises an aggressive de-blocking method if the at least one region comprises a low textured region.

14. The method of claim 11 wherein the first spatial processing method comprises a interpolation method biased in a given direction depending upon the presence of edges in that given direction.

15. The method of claim 11 wherein the second temporal processing method comprises a motion compensation method if the video bitstream comprises low motion images.

16. The method of claim 11 wherein the second temporal processing method comprises a frame replication method if the video bitstream comprises high motion images.

17. The method of claim 11 wherein the second temporal processing method comprises a frame blending method if the video bitstream comprises high motion images.

18. The method of claim 11 wherein the first spatial processing method comprises a filter depending upon the outcome of a given distortion model applied to the video bitstream.

19. The method of claim 18 wherein the filter is selected for the at least one region according to image characteristics of the at least one region.

20. The method of claim 11, further comprising:

receiving, by the decoder device, processing information in the video bitstream indicating recommended spatial de-noising processing or temporal de-noising processing for the video bitstream;

processing, by the processor of the decoder device, the video bitstream using the spatial de-noising processing information or the temporal de-noising processing information, wherein the spatial de-noising processing and the temporal de-noising processing are post-decoding processing.

21. The method of claim 11, further comprising:

receiving, by the decoder device, processing information in the video bitstream indicating recommended error concealment processing for the video bitstream;

processing, by the processor of the decoder device, the video bitstream using the error concealment processing information, wherein the error concealment processing is post-decoding processing.