Scalable video coding system

• US 6,292,512 B1
• Filed: 07/06/1998
• Issued: 09/18/2001
• Est. Priority Date: 07/06/1998
• Status: Expired due to Term

• Current Assignee: Funai Electric Co., Ltd. (Funai Electric Holdings Company Limited)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Radha, Hayder, Chen, Yingwei, Cohen, Robert A.
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/110,616
• Time in Patent Office: 1,170 Days
• Field of Search: 375/240.05, 375/240.01, 375/240.19, 375/240.11, 375/240.1
• US Class Current: 375/240.1
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server

Abstract

A system for coding video data comprised of one or more frames codes a portion of the video data using a frame-prediction coding technique, and generates residual images based on the video data and the coded video data. The system then codes the residual images using a fine-granular scalability coding technique, and outputs the coded video data and at least one of the coded residual images to a receiver.

Claims

1. A method of coding video data comprised of one or more frames, the method comprising:

a first coding step for producing coded video data by coding a portion of the video data using a frame-prediction coding technique;

a generating step for generating residual images based on the video data and the coded video data;

a second coding step for producing coded residual images by coding the residual images using a fine-granular scalability coding technique; and

an outputting step for outputting the coded video data and one or more of the coded residual images to a receiver;

wherein the first coding step comprises coding the portion of the video data at a plurality of different bit rates so as to produce multiple versions of the coded video data;

wherein the generating step comprises generating a plurality of residual images for each version of the coded video data;

wherein the second coding step comprises coding the plurality of residual images for each version of the coded video data using a fine-granular scalability coding technique; and

wherein the outputting step comprises outputting one version of the coded video data together with at least one corresponding coded residual image therefor;

wherein the outputting step comprises the steps of;

determining variations in a bandwidth of the receiver over time; and

selecting which one of the multiple-versions of the coded video data and which of the coded residual images to output over time based on the variations in the bandwidth of the receiver;

wherein, for a receiver bandwidth increasing from B₁ to B₂, where B₁<

B₂, the selecting step selects a first version of the coded video data and successively selects coded residual images corresponding to each frame of the first version of the coded video data, which are coded at successively higher bit rates; and

wherein for a receiver bandwidth increasing from B₂ to B₃, where B₂<

B₃, the selecting step selects a second version of the coded video data and successively selects coded residual images corresponding to each frame of the second version of the coded video data, which are coded at successively higher bit rates.

2. A method of coding video data comprised of one or more frames, the method comprising:

a first coding step for producing coded video data by coding a portion of the video data using a frame-prediction coding technique;

a generating step for generating residual images based on the video data and the coded video data;

a second coding step for producing coded residual images by coding the residual images using a fine-granular scalability coding technique; and

an outputting step for outputting the coded video data and one or more of the coded residual images to a receiver;

wherein the first coding step comprises coding the portion of the video data at a plurality of different bit rates so as to produce multiple versions of the coded video data;

wherein the generating step comprises generating a plurality of residual images for each version of the coded video data;

wherein the second coding step comprises coding the plurality of residual images for each version of the coded video data using a fine-granular scalability coding technique; and

wherein the outputting step comprises outputting one version of the coded video data together with at least one corresponding coded residual image therefor;

wherein the outputting step comprises the steps of;

determining variations in a bandwidth of the receiver over time; and

selecting which one of the multiple versions of the coded video data and which of the coded residual images to output over time based on the variations in the bandwidth of the receiver;

wherein, for a receiver bandwidth decreasing from B₃ to B₂, where B₃>

B₂, the selecting step selects a first version of the coded video data and successively selects coded residual images corresponding to each frame of the first version of the coded video data, which are coded at successively lower bit rates; and

wherein, for a receiver bandwidth decreasing from B₂ to B₁, where B₂>

B₁, the selecting step selects a second version of the coded video data and successively selects coded residual images corresponding to each frame of the second version of the coded video data, which are coded at successively lower bit rates.

3. An apparatus for coding video data comprised of one or more frames, the apparatus comprising:

a memory which stores computer-executable process steps; and

a processor which executes the process steps stored in the memory so as (i) to produce coded video data by coding a portion of the video data using a frame-prediction coding technique, (ii) to generate residual images based on the video data and the coded video data, (iii) to produce coded residual images by coding the residual images using a fine-granular scalability coding technique, and (iv) to output the coded video data and at least one of the coded residual images to a receiver;

wherein the processor (i) codes the portion of the video data at a plurality of different bit rates so as to produce multiple versions of the coded video data, (ii) generates a plurality of residual images for each version of the coded video data, (iii) codes the plurality of residual images for each version of the coded video data using a fine-granular scalability coding technique, and (iv) outputs one version of the coded video data together with at least one corresponding coded residual image therefor;

wherein the processor outputs the one version of the coded video data together with at least one corresponding coded residual image therefor by (i) determining variations in a bandwidth of the receiver over time, and (ii) selecting which one of the multiple versions of the coded video data and which of the coded residual images to output over time based on the variations in the bandwidth of the receiver;

wherein, for a receiver bandwidth increasing from B₁ to B₂, where B₁<

B₂, the processor selects a first version of the coded video data and successively selects coded residual images corresponding to each frame of the first version of the coded video data, which are coded at successively higher bit rates; and

wherein, for a receiver bandwidth increasing from B₂ to B₃, where B₂<

B₃, the processor selects a second version of the coded video data and successively selects coded residual images corresponding to each frame of the second version of the coded video data, which are coded at successively higher bit rates.

4. An apparatus for coding video data comprised of one or more frames, the apparatus comprising:

a memory which stores computer-executable process steps; and

a processor which executes the process steps stored in the memory so as (i) to produce coded video data by coding a portion of the video data using a frame-prediction coding technique, (ii) to generate residual images based on the video data and the coded video data, (iii) to produce coded residual images by coding the residual images using a fine-granular scalability coding technique, and (iv) to output the coded video data and at least one of the coded residual images to a receiver;

wherein the processor (i) codes the portion of the video data at a plurality of different bit rates so as to produce multiple versions of the coded video data, (ii) generates a plurality of residual images for each version of the coded video data, (iii) codes the plurality of residual images for each version of the coded video data using a fine-granular scalability coding technique, and (iv) outputs one version of the coded video data together with at least one corresponding coded residual image therefor;

wherein the processor outputs the one version of the coded video data together with at least one corresponding coded residual image therefor by (i) determining variations in a bandwidth of the receiver over time, and (ii) selecting which one of the multiple versions of the coded video data and which of the coded residual images to output over time based on the variations in the bandwidth of the receiver;

wherein, for a receiver bandwidth decreasing from B₃ to B₂, where B₃>

B₂, the processor selects a first version of the coded video data and successively selects coded residual images corresponding to each frame of the first version of the coded video data, which are coded at successively lower bit rates; and

wherein for a receiver bandwidth decreasing from B₂ to B₁, where B₂>

B₁, the processor selects a second version of the coded video data and successively selects coded residual images corresponding to each frame of the second version of the coded video data, which are coded at successively lower bit rates.

5. A method for coding video data and for outputting coded video data to a plurality of receivers, the method comprising the steps of:

coding a first portion of the video data using a frame-prediction coding technique to produce a first bitstream;

coding a second portion of the video data using a fine-granular scalability coding technique to produce a second bitstream;

outputting the first bitstream to the plurality of receivers;

dividing the second bitstream into two or more sub-streams; and

outputting the two or more sub-streams to the plurality of receivers;

further comprising the step of determining a maximum bit-rate among the plurality of receivers (R_MAX);

wherein the first portion of the video data is coded at a first bit-rate (R_BL), and the dividing step divides the second bitstream into sub-streams ranging from 0 bits to R_MAX−

R_BL bits.

6. An apparatus for coding video data and for outputting coded video data to a plurality of receivers, the apparatus comprising:

a memory which stores process steps; and

a processor which executes the process steps stored in the memory so as (i) to code a first portion of the video data using a frame-prediction coding technique to produce a first bitstream, (ii) to code a second portion of the video data using a fine-granular scalability coding technique to produce a second bitstream, (iii) to output the first bitstream to the plurality of receivers, (iv) to divide the second bitstream into two or more sub-streams, and (v) to output the two or more sub-streams to the plurality of receivers;

wherein the processor determines a maximum bit-rate among the plurality of receivers (R_MAX); and

wherein the first portion of the video data is coded at a first bit-rate (R_BL), and the processor divides the second bitstream into sub-streams ranging from 0 bits to R_MAX−

R_BL bits.