Method for transmission control in hybrid temporal-SNR fine granular video coding

• US 6,785,334 B2
• Filed: 08/15/2001
• Issued: 08/31/2004
• Est. Priority Date: 08/15/2001
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Radha, Hayder, van der Schaar, Mihaela
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): Bugg, George A
• Application Number:
  US09/930,671
• Publication Number:
  US 20030035480A1
• Time in Patent Office: 1,112 Days
• Field of Search: 375/240.01, 375/240.11, 375/240.16, 375/240.19, 348/397.1, 348/398.1
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]

Abstract

In a Fine Granular Video encoding system, a method for determining the number of transmission bits of SNR encoded and temporally encoded video data within a frame to balance image quality and object motion is presented. In accordance with the principles of the invention, a number of transmission bits at a known bit-rate for a quality enhanced video frame and a temporal enhanced video frame is determined to balance image quality and object motion smoothness. In one aspect of the invention, the number of bits transmitted in each frame is determined by comparing a ratio of a measure of video encoded information within the quality enhanced video frame and a measure of video encode information within the quality enhanced video frame and the temporally enhanced video frame to a known threshold level. The number of transmission bits in each enhancement layer is then determined using a first method when the ratio is above a known threshold and using a second method otherwise. In a second aspect of the invention, the number of bits is determined by first determining a measure of motion activity and complexity. The number of transmission bits is determined using a first method when the motion activity is below a known measure or the complexity if above a known threshold. Otherwise a second method is used to determine the number of transmission bits in each enhancement layer.

Claims

1. A method for determining a number of transmission bits, at a known bit-rate, for each of a current quality enhancement layer frame and a current temporal enhancement layer frame in a transmission of a fine granular enhanced encoded video frame, said method comprising the steps of:

comparing a ratio of a first measure of said quality enhancement layer frame and a second measure of said quality enhancement layer frame and said temporal enhancement layer frame to a known threshold; and

determining said number of transmission bits using a first method when said ratio is below a known threshold and using a second method otherwise.

2. The method as recited in claim 1 wherein said first measure is a difference in quality measure between said current quality enhancement layer frame and at least one next quality enhancement layer frame, said current frame quality measure and said at least one next frame quality measure is determined using a number of bits determined in accordance with said first method.

3. The method as recited in claim 1 wherein said second measure is a difference in quality measure between said current quality enhancement layer frame and at least one next temporal enhancement layer frame, said current frame quality measure is determined using a number of bits determined in accordance with said first method and said at least one next temporal enhancement layer frame quality measure is determined using a number of bits determined in accordance with said second method.

4. The method as recited in claim 1 wherein said threshold is selected in accordance with said temporal enhancement layer frame measure, wherein said temporal enhancement layer frame measure is determined using a number of bits determined in accordance with said second method.

5. The method as recited in claim 4 wherein said threshold monotonically decreases as said temporal enhancement layer frame measure increases.

6. The method as recited in claim 1 further comprising the steps of:

comparing a measure of at least one next temporal enhancement layer frame to a known level; and

determining said number of transmission bits in accordance with said first method when said measure is below said known level.

7. The method as recited in claim 6 wherein said measure of at least one temporal enhancement layer frame is determined using a number of bits determined in accordance with said second method.

8. The method as recited in claim 1 further comprising the steps of:

comparing a measure of said current temporal enhancement layer frame with a second known level; and

determining said number of transmission bits in accordance with said second method when said measure is above said second known level.

9. The method as recited in claim 8 wherein said measure of said current temporal enhancement layer frame is determined using a number of bits determined in accordance with said second method.

10. The method as recited in claim 1 wherein said first method is determined in accordance with $F_{\mathrm{eA}}^{\mathrm{FGS}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{R-R_{\mathrm{BL}}}{f_{\mathrm{BL}}}$

where:

F^FGS_eA is the number of bits for each FGS enhancement frame;

F^FGST_eA=0;

R_enh is the bit-rate available for the enhancement layer at transmission time;

f_t is the transmission frame rate

R is the total bandwidth available at transmission time;

f_BL is the base layer frame rate and;

R_BL is the base-layer bit-rate.

11. The method as recited in claim 1 wherein said second method is determined in accordance with $F_{\mathrm{eB}}^{\mathrm{FGS}}=F_e^{\mathrm{FGST}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{F_{\mathrm{eA}}^{\mathrm{FGS}}}{f_{\mathrm{BL}}+f_{\mathrm{EL}}}$

where

F^FGS_eB is the number of bits for an FGS frame;

F^FGST_eB is the number of bits for an FGST frame; and

f_EL is the over all enhancement-layer bit-rate.

19. The device as recited in claim 1 further including:

code for comparing a measure of said current temporal enhancement layer frame with a second known level; and

code for determining said number of transmission bits in accordance with said second method when said measure is above said second known level.

20. The device as recited in claim 19 wherein said current temporal enhancement layer frame measure is determined using a number of bits determined in accordance with said second method.

23. The method as recited in claim 10 wherein said number of transmission bits in said temporal layer enhancement frame is zero.

12. A device for determining a number of transmission bits, at a known bit-rate, for each of a current quality enhancement layer frame and a current temporal enhancement layer frame in a transmission of a fine granular encoded video frame, said device including:

code for comparing a ratio of a first measure of said quality enhancement layer frame and a second measure of said quality enhancement layer frame and said temporal enhancement layer frame to a known threshold; and

code for determining said number of transmission bits using a first method when said ratio is above a known threshold and using a second method otherwise.

13. The device as recited in claim 12 further including:

code for determining said first measure as a difference in quality between said current quality enhancement layer frame and at least one next quality enhancement layer frame, wherein said quality measure of said current frame and said at least one next frame is determined using a number of bits determined in accordance with said first method.

14. The device as recited in claim 12 further including:

code for determining said second measure as a difference in quality measure between said current quality enhancement layer frame and at least one next temporal enhancement layer frame wherein said current frame quality measure is determined using a number of bits determined in accordance with said first method and said at least one next temporal enhancement layer frame quality measure is determined using a number of bits determined in accordance with said second method.

15. The device as recited in claim 12 further including:

code for determining said threshold in accordance with said temporal enhancement layer frame measure, wherein said temporal enhancement layer frame measure is determined using a number of bits determined in accordance with said second method.

17. The device as recited in claim 12 further including:

code for comparing a measure of at least one next temporal enhancement layer frame to a known level; and

code for determining said number of transmission bits in accordance with said first method when said measure is below said known level.

18. The device as recited in claim 17 wherein said measure of at least one temporal enhancement layer frame is determined using a number of bits determined in accordance with said second method.

21. The device as recited in claim 12 further including:

code for determining said first method in accordance with $F_{\mathrm{eA}}^{\mathrm{FGS}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{R-R_{\mathrm{BL}}}{f_{\mathrm{BL}}}$

where;

F^FGS_eA is the number of bits for each FGS enhancement frame;

F^FGST_eA=0;

R_enh is the bit-rate available for the enhancement layer at transmission time;

f_t is the transmission frame rate

R is the total bandwidth available at transmission time;

f_BL is the base layer frame rate and;

R_BL is the base-layer bit-rate.

22. The device as recited in claim 12 further including:

code for determining said second method in accordance with $F_{\mathrm{eB}}^{\mathrm{FGS}}=F_e^{\mathrm{FGST}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{F_{\mathrm{eA}}^{\mathrm{FGS}}}{f_{\mathrm{BL}}+f_{\mathrm{EL}}}$

where

F^FGS_eB is the number of bits for an FGS frame;

F^FGST_eB is the number of bits for an FGST frame; and

f_EL is the over all enhancement-layer bit-rate.

24. The device as recited in claim 21 further including:

code to set said number of transmission bits in said temporal enhancement layer frame to zero.

16. The device as recited 15 wherein said threshold monotonically decreases as said temporal enhancement layer frame measure increases.

25. An memory module containing code therein for determining the number of transmission bits in each of a current quality enhancement layer frame and a current temporal enhancement layer frame in a transmission of a fine granular scalability encoded video frame, said memory module code being operable to:

compare a measure of at least one next temporal enhancement layer frame to a known level;

determine said number of transmission bits in accordance with a first method when said measure is below said known level;

compare a measure of a current temporal enhancement layer frame with a second known level;

determine said number of transmission bits in accordance with a second method when said second measure is above said second known level;

compare a ratio of a measure of said quality enhancement layer frame and a measure of said quality enhancement layer frame and said temporal enhancement layer frame to a known threshold; and

determine said number of transmission bits in accordance with said first method when said ratio is above a known threshold and in accordance with said second method otherwise.

26. The memory module code as recited in claim 25 further operable to:

determine said at least one temporal enhancement layer frame using a number of bits determined in accordance with said second method.

27. The memory module code as recited in claim 25 further operable to:

determine said current temporal enhancement layer frame measure using a number of bits determined in accordance with said second method.

28. The memory module code as recited in claim 25 further operable to:

determine a quality measure of said current quality enhancement layer frame using a number of bits determined in accordance with said first method;

determine a quality measure of at least one next quality enhancement layer frame using a number of bits determined in accordance with said first method; and

determine said quality enhancement layer frame measure as a difference in said current quality enhancement layer frame quality measure and at least one next quality enhancement layer frame quality measure.

29. The memory module code as recited in claim 25 further operable to:

determine a quality measure of said current quality enhancement layer frame using a number of bits determined in accordance with said first method;

determine a quality measure of said at least one next temporal enhancement layer frame using a number of bits determined in accordance with said second method; and

determine said quality enhancement layer frame and said temporal enhancement layer frame measure as a difference in said current quality enhancement layer frame quality measure and said at least one next temporal enhancement layer frame quality measure.

30. The memory module code as recited in claim 25 further operable to determine said first method in accordance with $F_{\mathrm{eA}}^{\mathrm{FGS}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{R-R_{\mathrm{BL}}}{f_{\mathrm{BL}}}$

where:

F^FGS_eA is the number of bits for each FGS enhancement frame;

F^FGST_eA=0;

R_enh is the bit-rate available for the enhancement layer at transmission time;

f_t is the transmission frame rate

R is the total bandwidth available at transmission time;

f_BL is the base layer frame rate and;

R_BL is the base-layer bit-rate.

31. The memory module code as recited in claim 30 further operable to:

determine said number of transmission bits in said temporal enhancement layer frame to zero.

32. The memory module code as recited in claim 25 further operable to determine said second method in accordance with $F_{\mathrm{eB}}^{\mathrm{FGS}}=F_e^{\mathrm{FGST}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{F_{\mathrm{eA}}^{\mathrm{FGS}}}{f_{\mathrm{BL}}+f_{\mathrm{EL}}}$

where

F^FGS_eB is the number of bits for an FGS frame;

F^FGST_eB is the number of bits for an FGST frame; and

f_EL is the over all enhancement-layer bit-rate.

33. A method for determining a number of transmission bits, at a known bit-rate, for each of a current quality enhancement layer frame and a current temporal enhancement layer frame in a transmission of a fine granular enhanced encoded video frame, said method comprising the steps of:

determining said number of transmission bits using a first method when a quality layer frame measure is above an known threshold;

determining said number of transmission bits using said first method when a temporal enhancement layer frame measure is below an acceptable measure;

otherwise, determining said number of transmission bits using a second method.

34. The method as recited in claim 33 wherein said second method is determined in accordance with $F_{\mathrm{eB}}^{\mathrm{FGS}}=F_e^{\mathrm{FGST}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{F_{\mathrm{eA}}^{\mathrm{FGS}}}{f_{\mathrm{BL}}+f_{\mathrm{EL}}}$

where

F^FGS_eB is the number of bits for an FGS frame;

F^FGST_eB is the number of bits for an FGST frame; and

f_EL is the over all enhancement-layer bit-rate.

35. The method as recited in claim 33 wherein said first method is determined in accordance with $F_{\mathrm{eA}}^{\mathrm{FGS}}=\frac{R_{\mathrm{enh}}}{f_t}=\frac{R-R_{\mathrm{BL}}}{f_{\mathrm{BL}}}$

where:

F^FGS_eA is the number of bits for each FGS enhancement frame;

F^FGST_eA=0;

R_enh is the bit-rate available for the enhancement layer at transmission time;

f_t is the transmission frame rate

R is the total bandwidth available at transmission time;

f_BL is the base layer frame rate and;

R_BL is the base-layer bit-rate.

36. The method as recited in claim 33 wherein said temporal enhancement layer frame measure is determined from a motion activity measure in said frame.

37. The method as recited in claim 33 wherein said quality layer frame measure is determined from a complexity measure of said frame.

38. The method as recited in claim 36, wherein said motion activity measure is determined in relation to motion vectors of said frame.