Video data compression with integrated lossy and lossless compression

• US 8,194,736 B2
• Filed: 04/15/2008
• Issued: 06/05/2012
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Youn, Jeongnam
• Primary Examiner(s): NGUYEN, HANH N
• Application Number:
  US12/103,427
• Publication Number:
  US 20090257485A1
• Time in Patent Office: 1,512 Days
• Field of Search: 375/240, 375/240.01, 375/240.03, 375/240.12, 375/240.23, 375/240.24, 375/240.25, 370/410, 370/412, 370/413, 370/451, 370485-487, 348/397, 348/398, 348/420, 348/421, 348/405, 348/411
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/428 Recompression, e.g. by spatial or temporal decimation
• CPC Class Codes: H04N 19/433 characterised by techniques for memory access
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A method and apparatus to compress video data to smaller size and embed extra information into the compressed data using the integrated lossy and lossless compressions is described. The method can be used to compress reference frames of a video codec (coder/decoder) combines codec where the small noise is critical and to reduce bus bandwidth. Data transfer between an encoder and an external frame memory connected via an external bus in a video codec is reduced by compressing data from the encoder prior to inputting into the frame memory over the external bus, and decompressing the compressed data from the frame memory after retrieving over the external bus. Reference frames are compressed to variable size without causing any considerable artifact to reduce bus bandwidth between the encoder core and external memory. In the method, lossy and lossless compression is integrated to maximize the compression efficiency.

Claims

1. A method for reducing video data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

dividing each frame of video data into blocks, and segmenting each block into blocks of smaller size;

compressing video data from an encoder prior to inputting into a frame memory over an external bus;

wherein each block is compressed by first predicting and compressing each original block, and then determining if a number of bits used during compression is within an allowed range;

decompressing compressed video data from the frame memory after retrieving over the external bus;

compressing the video data by lossy compression if the number of bits used during compression is not within an allowed range; and

generating a noise compensation bit to indicate an amount of noise introduced during compression.

2. The method of claim 1, wherein the video data is compressed by lossless compression.

3. The method of claim 1, wherein the video data is compressed by lossy compression.

4. The method of claim 3:

wherein in said lossy compression, compression noise is compensated by estimating noise in compression and embedding noise information in a compressed block; and

wherein noise is compensated during decompression.

5. The method of claim 3, wherein during compression the least significant bit is truncated and the majority truncated bit is set as a noise compensation bit, and during decompression the least significant bits are set to the majority truncated bit.

6. The method of claim 1, wherein the video data is compressed by combination of lossless compression and lossy compression.

7. The method of claim 6, further comprising embedding extra information in each compressed block without losing original information in the block, at least one said extra information comprising a compressed/uncompressed block indication, and at least one said extra information comprising a lossy/lossless indication.

8. The method of claim 6, further comprising determining a maximum precision that is needed to represent the residual data in each block, and this maximum precision is applied for the compression of the block.

9. The method of claim 6, further comprising:

wherein said blocks of smaller size comprise subblocks;

determining a maximum precision for each subblock; and

compressing each subblock using the maximum precision for that subblock.

10. The method of claim 1, wherein the frames are luminance frames and chrominance frames.

11. The method of claim 1, further comprising:

wherein said blocks of smaller size comprise subblocks;

applying a different level of compression to said subblocks;

wherein inside one compression block, both lossy and lossless subblocks are integrated together.

12. The method of claim 1, further comprising packing the compressed video data into a packet and storing the packet in the frame memory if the number of bits used during compression is within said allowed range.

13. The method of claim 12, wherein the compressed video data from frame memory is decompressed by first determining if each block is compressed, and if compressed, decoding each block using the same precision used in compressing the block.

14. The method of claim 13, wherein determining if each block is compressed is performed by reading an embedded compressed/uncompressed bit.

15. The method of claim 13, further comprising determining if the block is compressed by lossy compression, and if so, compensating the decompressed block to reduce noise.

16. The method of claim 15, wherein determining if the block is compressed by lossy compression is performed by reading an embedded lossy/lossless bit.

17. The method of claim 15, wherein compensating the decompressed block comprises reading a noise compensation bit.

18. The method of claim 13, further comprising, if a block is not compressed, determining if the video data was compressed by lossy compression, and if so decoding the data and if not reading all the data.

19. The method of claim 1, wherein lossy compression is performed by reducing precision of the original video data.

20. The method of claim 1, wherein the video data is processed by an uncompression process if lossy compression is not allowed.

21. The method of claim 20, wherein the uncompression process is performed by first processing the first pixel by reading the first pixel, taking some bits out of the first pixel and sending the removed bits to FIFO, reading additional bits of header and attaching the header bits to the remaining bits of the first pixel, storing the result in a compression buffer, decompressing the processed first pixel, and storing the decompressed first pixel in a decompression buffer.

22. The method of claim 21, wherein the uncompression process is further performed starting with the second pixel by predicting the current pixel value from the previous decompressed pixel, computing the residual between the current and predicted values, determining the number of bits required for the residual, and embedding a stop flag bit and a contained previous bit and any FIFO data in available bits.

23. A method for reducing video data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

dividing each frame of video data into blocks, and segmenting each block into blocks of smaller size;

compressing data from an encoder prior to inputting into a frame memory over an external bus;

wherein each block is compressed by first predicting and compressing each original block, and then determining if a number of bits used during compression is within an allowed range;

decompressing compressed data from the frame memory after retrieving over the external bus;

compressing the video data by lossy compression if the number of bits used during compression is not within an allowed range; and

wherein the video data is processed by an uncompression process if lossy compression is not allowed.

24. The method as recited in claim 23, wherein the video data is compressed by lossless compression, lossy compression or a combination of lossless compression and lossy compression.

25. The method as recited in claim 24, further comprising embedding extra information in each compressed block without losing original information in the block, at least one said extra information comprising a compressed/uncompressed block indication, and at least one said extra information comprising a lossy/lossless indication.

26. The method as recited in claim 24, further comprising determining a maximum precision that is needed to represent residual data in each block, and applying this for the compression of the block.

27. The method as recited in claim 24, further comprising:

segmenting each block into subblocks;

determining a maximum precision for each subblock; and

compressing each subblock using the maximum precision for that subblock.

28. The method as recited in claim 24, wherein the uncompression process is performed by first processing the first pixel by reading the first pixel, taking some bits out of the first pixel and sending the removed bits to FIFO, reading additional bits of header and attaching the header bits to the remaining bits of the first pixel, storing the result in a compression buffer, decompressing the processed first pixel, and storing the decompressed first pixel in a decompression buffer.

29. The method as recited in claim 28, wherein the uncompression process is further performed starting with the second pixel by predicting the current pixel value from the previous decompressed pixel, computing the residual between the current and predicted values, determining the number of bits required for the residual, and embedding a stop flag bit and a contained previous bit and any FIFO data in available bits.

30. The method as recited in claim 24:

wherein in said lossy compression, compression noise is compensated by estimating noise in compression and embedding noise information in a compressed block; and

wherein noise is compensated during decompression.

31. The method as recited in claim 24, wherein during compression the least significant bit is truncated and the majority truncated bit is set as a noise compensation bit, and during decompression the least significant bits are set to the majority truncated bit.

32. The method as recited in claim 23, wherein said frames comprise luminance frames and chrominance frames.

33. The method as recited in claim 23:

wherein said blocks of smaller size comprise subblocks;

wherein said subblocks have a different level of compression; and

wherein inside one compression block, lossy and lossless subblocks can be integrated together.

34. The method as recited in claim 23, further comprising packing the compressed video data into a packet and storing the packet in the frame memory if the number of bits used during compression is within an allowed range.

35. The method as recited in claim 34, wherein the compressed data from frame memory is decompressed by first determining if each block is compressed, and if compressed, decoding each block using the same precision used in compressing the block.

36. The method as recited in claim 35, wherein determining if each block is compressed is performed by reading an embedded compressed/uncompressed bit.

37. The method as recited in claim 35, further comprising determining if the block is compressed by lossy compression, and if so, compensating the decompressed block to reduce noise.

38. The method as recited in claim 37, wherein determining if the block is compressed by lossy compression is performed by reading an embedded lossy/lossless bit.

39. The method as recited in claim 37, wherein compensating the decompressed block comprises reading a noise compensation bit.

40. The method as recited in claim 35, further comprising, if a block is not compressed, determining if the video data was compressed by lossy compression, and if so decoding the video data and if not reading all the video data.

41. The method as recited in claim 23, wherein lossy compression is performed by reducing the precision of the original video data.

42. The method as recited in claim 23, further comprising generating a noise compensation bit to indicate the amount of noise introduced during compression.

43. A method for reducing video data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

dividing each frame of video data into blocks, and segmenting each block into blocks of smaller size;

compressing data from an encoder prior to inputting into a frame memory over an external bus;

wherein each block is compressed by first predicting and compressing each original block, and then determining if a number of bits used during compression is within an allowed range;

decompressing compressed data from the frame memory after retrieving over the external bus;

compressing the video data by lossy compression if the number of bits used during compression is not within an allowed range;

wherein the video data is processed by an uncompression process if lossy compression is not allowed; and

wherein the uncompression process is performed by first processing a first pixel by reading the first pixel, taking some bits out of the first pixel and sending the removed bits to FIFO, reading additional bits of header and attaching the header bits to the remaining bits of the first pixel, storing the result in a compression buffer, decompressing the processed first pixel, and storing the decompressed first pixel in a decompression buffer.

44. The method of claim 43, wherein the uncompression process is further performed starting with a second pixel by predicting a current pixel value from the previous decompressed pixel, computing a residual between current and predicted values, determining a number of bits required for the residual, and embedding a stop flag bit and a contained previous bit and any FIFO data in available bits.

45. A method for reducing video data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

dividing each frame of video data into blocks, and segmenting each block into subblocks;

determining a maximum precision for each subblock;

compressing each subblock using a maximum precision for that subblock;

said subblocks are compressed from an encoder prior to inputting into a frame memory over an external bus;

wherein the video data is compressed by combination of lossless compression and lossy compression; and

decompressing compressed video data from the frame memory after retrieving over the external bus.

46. The method as recited in claim 45, wherein said video data comprises luminance frames and chrominance frames.

47. The method as recited in claim 45, further comprising generating a noise compensation bit to indicate an amount of noise introduced during compression.

48. A method for reducing video data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

dividing each frame of video into blocks;

compressing video data from an encoder prior to inputting into a frame memory over an external bus;

wherein each block is compressed by first predicting and compressing each original block, and then determining if a number of bits used during compression is within an allowed range;

packing compressed video data into a packet and storing the packet in the frame memory if the number of bits used during compression is within an allowed range;

decompressing the compressed video data from the frame memory after retrieving over the external bus; and

determining if the block is compressed by lossy compression, and if so, compensating the decompressed block by reading a noise compensation bit to reduce noise;

wherein the compressed video data from frame memory is decompressed by first determining if each block is compressed, and if compressed, decoding each block using an identical precision used in compressing the block.

49. A method for reducing data transfer between an encoder and an external frame memory connected via an external bus in a video codec, comprising:

compressing video data, by lossy compression, from an encoder prior to inputting into a frame memory over an external bus; and

decompressing compressed video data from the frame memory after retrieving over the external bus;

wherein in said lossy compression, compression noise is compensated by estimating noise in compression and embedding noise information in a compressed block; and

wherein noise is compensated during decompression.

50. The method as recited in claim 49, wherein said video data comprises luminance frames and chrominance frames.

51. The method as recited in claim 49, further comprising:

wherein frames of said video data are divided into blocks;

segmenting said blocks into subblocks having a different level of compression.

52. The method as recited in claim 49, further comprising generating a noise compensation bit to indicate an amount of noise introduced during compression.

53. The method as recited in claim 49, wherein the video data is processed by an uncompression process if lossy compression is not allowed.

54. A method for reducing video data transfer between an encoder and an external frame memory connected through an external bus in a video codec, comprising:

compressing video data, by lossy compression, from an encoder prior to inputting into a frame memory over a external bus; and

decompressing compressed video data from the frame memory after retrieving over the external bus;

wherein during compression a least significant bit is truncated and a majority truncated bit is set as a noise compensation bit, and during decompression the least significant bits are set to the majority truncated bit.

55. The method as recited in claim 54, wherein said video data comprises luminance frames and chrominance frames.

56. The method as recited in claim 54, further comprising:

wherein frames of said video data are divided into blocks;

segmenting said blocks into subblocks having a different level of compression.

57. The method as recited in claim 54, further comprising generating a noise compensation bit to indicate an amount of noise introduced during compression.

58. The method as recited in claim 54, wherein the video data is processed by an uncompression process if lossy compression is not allowed.