Encoding and decoding video signals using adaptive filter switching criteria

• US 5,488,570 A
• Filed: 06/29/1994
• Issued: 01/30/1996
• Est. Priority Date: 11/24/1993
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Agarwal, Rohit
• Primary Examiner(s): Ramirez, Ellis B.
• Application Number:
  US08/268,270
• Time in Patent Office: 580 Days
• Field of Search: 364/514 R, 364/514 A, 348/390, 348/396, 348/391, 348/394, 348/395, 348/404, 348/409, 348/412, 348/19, 348/12, 348/13, 348/607, 395/162
• US Class Current: 345/501
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/895 : in combination with error concealment
• CPC Class Codes: H04N 19/91 : Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/152 : Multipoint control units therefor
• CPC Class Codes: H04N 7/52 : Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434
• CPC Class Codes: H04Q 11/0435 : Details
• CPC Class Codes: H04Q 2213/13034 : A/D conversion, code compression/expansion
• CPC Class Codes: H04Q 2213/13103 : Memory
• CPC Class Codes: H04Q 2213/13106 : Microprocessor, CPU
• CPC Class Codes: H04Q 2213/13152 : Callback
• CPC Class Codes: H04Q 2213/1319 : Amplifier, attenuation circuit, echo suppressor
• CPC Class Codes: H04Q 2213/13209 : ISDN
• CPC Class Codes: H04Q 2213/1324 : Conference call
• CPC Class Codes: H04Q 2213/13299 : Bus
• CPC Class Codes: H04Q 2213/13337 : Picturephone, videotelephony

Abstract

Reference frames are generated by selectively filtering blocks of decoded video frames. The decision whether to filter a block is based on a comparison of an energy measure value generated for the block and an energy measure threshold value corresponding to the quantization level used to encode the block. The energy measure threshold value for a given quantization level is selected by analyzing the results of encoding and decoding training video frames using that quantization level. The reference frames are used in encoding and decoding video frames using interframe processing.

Claims

1. A computer-implemented process for encoding video signals, comprising the steps of:

(a) encoding one or more training video frames using a selected quantization level to generate one or more encoded training video frames;

(b) decoding the encoded training video frames to generate one or more decoded training video frames;

(c) generating one or more energy measure values corresponding to the decoded training video frames;

(d) performing steps (a)-(c) for a plurality of quantization levels;

(e) selecting an energy measure threshold value for each of the quantization levels in accordance with the decoded training video frames;

(f) generating a first reference frame corresponding to a first video frame;

(g) encoding a block of a second video frame using the first reference frame and a selected quantization level to generate a block of an encoded second video frame;

(h) decoding the block of the encoded second video frame to generate a block of a second reference frame, wherein step (h) comprises the steps of;

(1) generating an energy measure value corresponding to the block of the encoded second video frame;

(2) comparing the energy measure value of step (h)(1) with the energy measure threshold value of step (e) corresponding to the selected quantization level for the block; and

(3) applying a filter to generate the block of the second reference frame in accordance with the comparison of step (h)(2); and

(i) encoding a third video frame using the second reference frame.

2. The process of claim 1, wherein:

step (g) comprises the steps of;

(1) generating pixel differences between the block of the second video frame and the first reference frame; and

(2) encoding the block of the second video frame in accordance with the pixel differences to generate the block of the encoded second video frame; and

step (h)(1) comprises the steps of;

(i) decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

3. The process of claim 2, wherein step (h)(3) comprises the steps of:

(i) applying the filter to a block of the first reference frame; and

(ii) adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

4. The process of claim 2, wherein step (h)(3) comprises the steps of:

(i) adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) applying the filter to the reconstructed block to generate the block of the second reference frame.

5. The process of claim 2, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

6. The process of claim 1, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

7. The process of claim 1, wherein the filter comprises a spatial filter.

8. The process of claim 1, wherein the energy measure comprises a sum of absolute differences.

9. An apparatus for encoding video signals, comprising:

(a) means for encoding one or more training video frames using a selected quantization level to generate one or more encoded training video frames;

(b) means for decoding the encoded training video frames to generate one or more decoded training video frames;

(c) means for generating one or more energy measure values corresponding to the decoded training video frames, wherein the processing of means (a)-(c) is performed for a plurality of quantization levels and an energy measure threshold value is selected for each of the quantization levels in accordance with the decoded training video frames;

(d) means for generating a first reference frame corresponding to a first video frame;

(e) means for encoding a block of a second video frame using the first reference frame and a selected quantization level to generate a block of an encoded second video frame;

(f) means for decoding the block of the encoded second video frame to generate a block of a second reference frame, wherein means (f) comprises;

(1) means for generating an energy measure value corresponding to the block of the encoded second video frame;

(2) means for comparing the energy measure value of means (f)(1) with the energy measure threshold value corresponding to the selected quantization level for the block; and

(3) means for applying a filter to generate the block of the second reference frame in accordance with the comparison of means (f)(2); and

(i) means for encoding a third video frame using the second reference frame.

10. The apparatus of claim 9, wherein:

means (e) comprises;

(1) means for generating pixel differences between the block of the second video frame and the first reference frame; and

(2) means for encoding the block of the second video frame in accordance with the pixel differences to generate the block of the encoded second video frame; and

means (f)(1) comprises;

(i) means for decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) means for generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

11. The apparatus of claim 10, wherein means (f)(3) comprises:

(i) means for applying the filter to a block of the first reference frame; and

(ii) means for adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

12. The apparatus of claim 10, wherein means (f)(3) comprises:

(i) means for adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) means for applying the filter to the reconstructed block to generate the block of the second reference frame.

13. The apparatus of claim 10, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

14. The apparatus of claim 10, wherein the apparatus comprises a pixel processor, the pixel processor is electrically connected to a bus, and the bus is electrically connected to a memory device.

15. The apparatus of claim 9, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

16. The apparatus of claim 9, wherein the filter comprises a spatial filter.

17. The apparatus of claim 9, wherein the energy measure comprises a sum of absolute differences.

18. The apparatus of claim 9, wherein the apparatus comprises a pixel processor, the pixel processor is electrically connected to a bus, and the bus is electrically connected to a memory device.

19. A computer-implemented process for encoding video signals, comprising the steps of:

(a) generating a first reference frame corresponding to a first video frame;

(b) encoding a block of a second video frame using the first reference frame and a selected quantization level to generate a block of an encoded second video frame;

(c) decoding the block of the encoded second video frame to generate a block of a second reference frame, wherein step (c) comprises the steps of;

(1) generating an energy measure value corresponding to the block of the encoded second video frame;

(2) comparing the energy measure value of step (c)(1) with an energy measure threshold value corresponding to the selected quantization level for the block; and

(3) applying a filter to generate the block of the second reference frame in accordance with the comparison of step (c)(2); and

(d) encoding a third video frame using the second reference frame, wherein the energy measure threshold value corresponding to the selected quantization level for the block having been determined by;

encoding one or more training video frames using each of a plurality of quantization levels to generate a plurality of encoded training video frames;

decoding the encoded training video frames to generate a plurality of decoded training video frames;

generating a plurality of energy measure values corresponding to the decoded training video frames; and

selecting an energy measure threshold value for each of the quantization levels in accordance with the decoded training video frames.

20. The process of claim 19, wherein:

step (b) comprises the steps of;

(1) generating pixel differences between the block of the second video frame and the first reference frame; and

(2) encoding the block of the second video frame in accordance with the pixel differences to generate the block of the encoded second video frame; and

step (c)(1) comprises the steps of;

(i) decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

21. The process of claim 20, wherein step (c)(3) comprises the steps of:

(i) applying the filter to a block of the first reference frame; and

(ii) adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

22. The process of claim 20, wherein step (c)(3) comprises the steps of:

(i) adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) applying the filter to the reconstructed block to generate the block of the second reference frame.

23. The process of claim 20, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

24. The process of claim 19, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

25. The process of claim 19, wherein the filter comprises a spatial filter.

26. The process of claim 19, wherein the energy measure comprises a sum of absolute differences.

27. An apparatus for encoding video signals, comprising:

(a) means for generating a first reference frame corresponding to a first video frame;

(b) means for encoding a block of a second video frame using the first reference frame and a selected quantization level to generate a block of an encoded second video frame;

(c) means for decoding the block of the encoded second video frame to generate a block of a second reference frame, wherein means (c) comprises;

(1) means for generating an energy measure value corresponding to the block of the encoded second video frame;

(2) means for comparing the energy measure value of means (c)(1) with an energy measure threshold value corresponding to the selected quantization level for the block; and

(3) means for applying a filter to generate the block of the second reference frame in accordance with the comparison of means (c)(2); and

(d) means for encoding a third video frame using the second reference frame, wherein the energy measure threshold value corresponding to the selected quantization level for the block having been determined by;

encoding one or more training video frames using each of a plurality of quantization levels to generate a plurality of encoded training video frames;

decoding the encoded training video frames to generate a plurality of decoded training video frames;

generating a plurality of energy measure values corresponding to the decoded training video frames; and

selecting an energy measure threshold value for each of the quantization levels in accordance with the decoded training video frames.

28. The apparatus of claim 27, wherein:

means (b) comprises;

(1) means for generating pixel differences between the block of the second video frame and the first reference frame; and

(2) means for encoding the block of the second video frame in accordance with the pixel differences to generate the block of the encoded second video frame; and

means (c)(1) comprises;

(i) means for decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) means for generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

29. The apparatus of claim 28, wherein means (c)(3) comprises:

(i) means for applying the filter to a block of the first reference frame; and

(ii) means for adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

30. The apparatus of claim 28, wherein means (c)(3) comprises:

(i) means for adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) means for applying the filter to the reconstructed block to generate the block of the second reference frame.

31. The apparatus of claim 28, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

32. The apparatus of claim 28, wherein the apparatus comprises a pixel processor, the pixel processor is electrically connected to a bus, and the bus is electrically connected to a memory device.

33. The apparatus of claim 27, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

34. The apparatus of claim 27, wherein the filter comprises a spatial filter.

35. The apparatus of claim 27, wherein the energy measure comprises a sum of absolute differences.

36. The apparatus of claim 27, wherein the apparatus comprises a pixel processor, the pixel processor is electrically connected to a bus, and the bus is electrically connected to a memory device.

37. A computer-implemented process for decoding video signals, comprising the steps of:

(a) decoding an encoded first video frame to generate a first reference frame;

(b) decoding a block of an encoded second video frame to generate a block of a second reference frame, wherein step (b) comprises the steps of;

(1) generating an energy measure value corresponding to the block of the encoded second video frame;

(2) comparing the energy measure value of step (b)(1) with an energy measure threshold value corresponding to a selected quantization level for the block; and

(3) applying a filter to generate the block of the second reference frame in accordance with the comparison of step (b)(2); and

(c) decoding an encoded third video frame using the second reference frame, wherein the energy measure threshold value corresponding to the selected quantization level for the block having been determined by;

encoding one or more training video frames using each of a plurality of quantization levels to generate a plurality of encoded training video frames;

decoding the encoded training video frames to generate a plurality of decoded training video frames;

generating a plurality of energy measure values corresponding to the decoded training video frames; and

selecting an energy measure threshold value for each of the quantization levels in accordance with the decoded training video frames.

38. The process of claim 37, wherein step (b)(1) comprises the steps of:

(i) decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

39. The process of claim 38, wherein step (b)(3) comprises the steps of:

(i) applying the filter to a block of the first reference frame; and

(ii) adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

40. The process of claim 38, wherein step (b)(3) comprises the steps of:

(i) adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) applying the filter to the reconstructed block to generate the block of the second reference frame.

41. The process of claim 38, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

42. The process of claim 37, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

43. The process of claim 37, wherein the filter comprises a spatial filter.

44. The process of claim 37, wherein the energy measure comprises a sum of absolute differences.

45. An apparatus for decoding video signals, comprising:

(a) means for decoding an encoded first video frame to generate a first reference frame;

(b) means for decoding a block of an encoded second video frame to generate a block of a second reference frame, wherein means (b) comprises;

(1) means for generating an energy measure value corresponding to the block of the encoded second video frame;

(2) means for comparing the energy measure value of means (b)(1) with an energy measure threshold value corresponding to a selected quantization level for the block; and

(3) means for applying a filter to generate the block of the second reference frame in accordance with the comparison of means (b)(2); and

(c) means for decoding an encoded third video frame using the second reference frame, wherein the energy measure threshold value corresponding to the selected quantization level for the block having been determined by;

encoding one or more training video frames using each of a plurality of quantization levels to generate a plurality of encoded training video frames;

decoding the encoded training video frames to generate a plurality of decoded training video frames;

generating a plurality of energy measure values corresponding to the decoded training video frames; and

selecting an energy measure threshold value for each of the quantization levels in accordance with the decoded training video frames.

46. The apparatus of claim 45, wherein means (b) (1) comprises:

(i) means for decoding the block of the encoded second video frame to generate decoded pixel differences; and

(ii) means for generating the energy measure value corresponding to the block of the encoded second video frame using the pixel differences.

47. The apparatus of claim 46, wherein means (b) (3) comprises:

(i) means for applying the filter to a block of the first reference frame; and

(ii) means for adding the decoded pixel differences to the filtered block of the first reference frame to generate the block of the second reference frame.

48. The apparatus of claim 46, wherein means (b)(3) comprises:

(i) means for adding the decoded pixel differences to a block of the first reference frame to generate a reconstructed block; and

(ii) means for applying the filter to the reconstructed block to generate the block of the second reference frame.

49. The apparatus of claim 46, wherein:

the encoded second video frame is generated using motion estimation;

the second reference frame is generated using motion compensation;

the filter comprises a spatial filter; and

the energy measure comprises a sum of absolute differences.

50. The apparatus of claim 46, wherein the apparatus comprises a host processor, the host processor is electrically connected to a bus, and the bus is electrically connected to a memory device.

51. The apparatus of claim 45, wherein the encoded second video frame is generated using motion estimation and the second reference frame is generated using motion compensation.

52. The apparatus of claim 45, wherein the filter comprises a spatial filter.

53. The apparatus of claim 45, wherein the energy measure comprises a sum of absolute differences.

54. The apparatus of claim 45, wherein the apparatus comprises a host processor, the host processor is electrically connected to a bus, and the bus is electrically connected to a memory device.