Directed Interpolation and Data Post-Processing

• US 20120026288A1
• Filed: 04/20/2010
• Published: 02/02/2012
• Est. Priority Date: 04/20/2009
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Tourapis, Alexandros, Pahalawatta, Peshala V., Leontaris, Athanasios, Stec, Kevin
• Granted Patent: US 9,729,899 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/172 image signals comprising non-image signal components, e.g. headers or format information
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/865 with detection of the former encoding block subdivision in decompressed video
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 19/895 : in combination with error concealment
• CPC Class Codes: H04N 21/4347 : Demultiplexing of several video streams
• CPC Class Codes: H04N 7/0102 : involving the resampling of the incoming video signal
• CPC Class Codes: H04N 7/0117 : involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/0135 : involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59

Abstract

An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bit-stream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Claims

1. A method comprising the step of receiving a signal indicative of a processing technique to be utilized on a data stream comprising a video comprising at least two views.

2. The method according to claim 1, wherein the processing technique comprises resolution conversion comprising at least one of spatial resolution conversion and temporal resolution conversion.

3. The method according to claim 1, wherein the processing technique comprises scaling comprising at least one of de-interlacing, temporal interpolation, and spatial interpolation.

4. The method according to claim 1, wherein the processing technique comprises error concealment comprising a prediction of motion of a region in error.

5. The method according to claim 1, wherein the signal is further indicative of at least one of a spatial and temporal extent to which the processing technique is to be applied, and the data stream is a video data stream and the extent comprises whether the processing technique is applied to at least one of all of the video, portions of the video, pictures of the video, and regions within the video.

6. The method according to claim 1, wherein the views comprise at least one of stereographic views, two different images, a 2D image and depth information, multiple views of 2D scene, and multiple views of a 3D scene.

7. The method according to claim 1, wherein the at least two views are compressed and multiplexed within the data stream in a standardized motion picture format capable of single view video streams.

8. The method according to claim 7, wherein compression of the views comprises at least one of a sampling, filtering, and decimation of the views;

and

compression of the views comprises at least one of horizontal, vertical filtering, and quincunx sampling.

9. The method according to claim 14, wherein the sampling comprises at least one of horizontal, vertical, quincunx, formula based, pattern based, and arbitrary sampling;

and the multiplexing is done in at least one of a checkerboard format, a quadrant based format, a column format, a row format, a side-by-side format, an over-under format, a format based on a pattern, and an alternative format.

10. The method according to claim 1, wherein the signal identifies at least one of at least part of a decoding process and at least part of a post-decoding process to be applied to a region of an image or video scene having characteristics on which the identified process operates efficiently;

the characteristics comprise at least one of texture, contour, edge, contrast, dynamic range, and brightness of the region; and

the signaling comprises a processing technique to be applied to at least one of a sequence, group of pictures (GOP), and region of one of a fixed, arbitrary, or varying size.

11. The method according to claim 1, further comprising the step of selecting at least one processing technique based on the indicative signal and applying the selected processing technique to at least one of the sequence, GOP, and region of the video;

and

wherein the step of selecting is performed by a decoder configured to decode the data stream by applying the selected processing technique to at least one of the sequence, group of pictures (GOP), and region.

12. The method according to claim 1, wherein the processing technique comprises a series of processing techniques comprising at least one of an interpolation process, filtering, and a de-blocking process.

13. The method according to claim 1, wherein the signaling comprises signaling a family of processing techniques, and the method further comprises the steps of selecting at least one processing technique from the signaled family and applying the selected technique(s) to at least one of a sequence, group of pictures (GOP), and a region of one of a fixed, arbitrary, or varying size.

14. The method according to claim 1, wherein the data stream comprises a trans-coded encoded video stream.

15. A post-processing device, comprising:

a video input configured to receive a decoded video stream;

an instruction mechanism configured to one of accept a directive instruction from a side information channel of the video stream and identify a directive instruction from the decoded video stream;

a post-processor configured to post-process the decoded video stream based on the directive instruction; and

a video output configured to output the post-processed video stream.

16. The post-processing device according to claim 15, wherein the post-processing device is integrated into at least one of a display, a set-top box, and a media player.

17. The post-processing device according to claim 15, wherein the directive instruction comprises an identification of at least one region in the video and at least one processing method to be applied to the region, and

the region comprises sub-regions and the directive instruction comprises different parameters to be applied to the processing method for processing each sub-region.

19. The post-processing device according to claim 15, wherein the directive signal comprises a filter identification selected after testing a plurality of filters on the decoded video stream.

20. The post-processing device according to claim 15, wherein the directive signal comprises a filter identification comprising an identification of an entire category of filters, and the post-processor is further configured to select a filter from the category of filters and utilize the selected filter to post-process the decoded video stream.

21. The post-processing device according to claim 20, wherein the post-processor is further configured to select a filter having properties closest to properties of the category of filters and utilize the selected filter as at least part of the post-process.

22. The post-processing device according to claim 19, wherein the post-processor is further configured to select a filter known to have properties that are at least one of close to or better than the filter identified by the filter identification.

23. The post-processing device according to claim 15, further comprising a post-process analyzer configured to analyze at least one of the decoded video stream and the video stream prior to decoding, and, based on the analysis, direct at least part of at least one of a decoding process and the post processor.

18. The post-processing device according to EEE 17, wherein the directive signal identifies regions of different texture qualities and processing methods specifically for each texture quality.

24. A system, comprising:

an encoder configured to select at least one of a processing technique, a category of processing techniques, and a series of processing techniques to be utilized in decoder and encode the selected techniques or an identifier of the selected technique(s) in at least one of an encoded video stream and side information related to the encoded video stream; and

a decoder configured to receive the encoded video stream and decode the video stream using at least one of the selected technique(s).

25. The system according to claim 24, wherein the selected technique(s) are identified by the decoder from at least one of the related side information and the encoded video stream.

26. The system according to claim 24, wherein the decoder uses the selected processing technique to select another processing technique.

27. The system according to claim 24, wherein the decoder maintains an inventory of processing techniques and selects a processing technique or technique(s) from the inventory most closely matching the technique(s) selected by the encoder.