Coding and decoding of interleaved image data

• US 10,362,334 B2
• Filed: 11/30/2017
• Issued: 07/23/2019
• Est. Priority Date: 01/29/2009
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Tourapis, Alexandros, Husak, Walter J., Pahalawatta, Peshala V., Leontaris, Athanasios
• Primary Examiner(s): Haque, Md N
• Application Number:
  US15/827,131
• Publication Number:
  US 20180213256A1
• Time in Patent Office: 600 Days
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/4382 Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002

Abstract

Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.

Claims

1. A video encoding method, comprising:

sub-sampling a first image and a second image into a first sampled image data and a second sampled image data respectively via vertical sampling;

separating each of the first sampled image data and the second sampled image data into a plurality of sub-images, wherein each of the sub-images comprises image data of the corresponding sampled image data selected via a pattern;

packaging at least the plurality of sub-images together into a single image frame of a video stream, wherein the packaging comprises tiling the sub-images together according to an arrangement indicating how the sub-images are tiled together, and wherein the arrangement indicates that the plurality of sub-images corresponding to the first sampled image data are tiled into a first rectangular block of the image frame, and the plurality of sub-images corresponding to the second sampled image data are tiled into a second rectangular block of the image frame, and wherein the first rectangular block is positioned in top-bottom relation with the second rectangular block in the image frame; and

encoding the single image frame, and encoding an identifier of the arrangement.

2. The method according to claim 1, wherein the identifier is a code that is placed in side information of an encoded patterned block and/or that is placed in the image frame.

3. The method according to claim 1, wherein the first image and the second image include any one or more of multiple images from multiple program streams, multiple angles of a same scene, or multiple sets of 3D views of a same or different scene, video game, or program.

7. A video device comprising one or more processor configured to perform steps of a method according to claim 1.

4. A method to decode an encoded video signal with a processor, the method comprising:

decoding the encoded video signal comprising more than one image per frame in the video signal;

de-interleaving groups of video data interleaved in a frame format in the decoded video signal, wherein the groups of video data comprise one or more groups of video data from a first image and one or more groups of video data from a second image, wherein de-interleaving the groups of video data comprises de-interleaving the groups of video data from multiple interleaving formats according to an identifier of an arrangement indicating how the groups of video data are tiled together, and wherein the arrangement indicates that the one or more groups of video data from the first image are tiled into a first rectangular block of an image frame, and the one or more groups of video data from the second image are tiled into a second rectangular block of the frame, and wherein the first rectangular block is positioned in top-bottom relation with the second rectangular block in the image frame; and

up-converting the one or more groups of video data from the first image and the one or more groups of video data from the second image into the first image and the second image respectively according to vertical sampling.

5. The method according to claim 4, wherein the identifier is a code that is placed in side information of an encoded patterned block and/or that is placed in the frame.

6. The method according to claim 4, wherein the first image and the second image include any one or more of multiple images from multiple program streams, multiple angles of a same scene, or multiple sets of 3D views of a same or different scene, video game, or program.

8. A video device comprising one or more processor configured to perform steps of a method according to claim 4.

9. The video device according to claim 8, wherein the processor is part of at least one of a Blue-Ray DVD player, a media player, a set-top box, a cable box, a computer video card or a tuner.