METHODS AND APPARATUS FOR IMAGE COMPRESSION AND DECOMPRESSION USING GRAPHICS PROCESSING UNIT (GPU)

• US 20080304738A1
• Filed: 01/17/2008
• Published: 12/11/2008
• Est. Priority Date: 06/11/2007
• Status: Active Grant

• Current Assignee: Mercury Systems, Incorporated
• Original Assignee: Mercury Computer Systems Incorporated
• Inventors: Beilloin, David
• Granted Patent: US 8,019,151 B2
• Time in Patent Office: Days
• US Class Current: 382/166
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/60 using transform coding

Abstract

The invention provides, in some aspects, methods for image compression that utilize the central processing unit (CPU) of a digital data processor and its associated graphics processing unit (GPU), together, in order to compress an image. In related aspects of the invention, the GPU is adapted to render at least a portion of an image to an associated texture buffer (i.e., a texture buffer of the GPU) and to transform the resulting image values in that texture buffer to a frequency-based representation. The GPU can, further, order coefficients comprising that frequency-based representation and transfer at least a portion of those coefficients to the associated CPU. That CPU can, in turn, effect transfer of the downloaded coefficients to another digital data processor (or image processing apparatus). Still other aspects of the invention provide methods of image decompression that utilize the central processing unit (CPU) of a digital data processor and its associated graphics processing unit (GPU), together, in order to decompress an image. These methods parallel the compression methods discussed above, albeit in substantially reversed order. Other aspects of the invention provide digital data processing apparatus for image compression, decompression and/or remote image display operating in accord with the methods described above.

Claims

1. A method of image compression in a digital data processor having a CPU that is coupled to an associated GPU comprising the steps of:

A. transforming, with the GPU, image values from a color representation to a frequency-based representation,

B. with the GPU, re-ordering coefficients comprising the frequency-based representation such that coefficients of like order from multiple sub-blocks of the frequency-based representation of the image are ordered together,

C. transferring the re-ordered coefficients to the CPU, the CPU at least one of storing and transmitting said coefficients.

2. The method of claim 1, comprising using the CPU to compress the re-ordered coefficients.

3. The method of claim 1, comprising using the CPU to transmit the compressed coefficients to a further digital data processor in communications coupling with the CPU.

4. The method of claim 3, wherein the GPU and the CPU are co-housed, and the further digital data processor is remote from said GPU and CPU.

5. The method of claim 1, further comprising

D. with the CPU, compressing the coefficients via a run length encoding-based compression technique.

6. The method of claim 1, wherein step (A) further comprises transforming, with the GPU, the image values from an RGB color space to a YCbCr color space before transforming said image values to the frequency-based representation.

7. The method of claim 1, wherein step (A) further comprises transforming, with the GPU, the image values from the color representation to the frequency-based representation via a Fourier-related transform.

8. The method of claim 1, wherein step (A) further comprises transforming, with the GPU, the image values from the color representation to the frequency-based representation via a discrete cosine transform (DCT).

9. The method of claim 8, wherein step (A) further comprises transforming, with the GPU, the image values from a color representation into a frequency-based representation via a first pass of the DCT transform and a second pass of the DCT transform, wherein the first pass is performed on lines of the image values, and the second pass is performed on columns of the image values.

10. The method of claim 9, wherein step (A) further comprises quantizing the coefficients with the GPU.

11. The method of claim 1, wherein step (B) further comprises re-ordering the coefficients so that coefficients of a first order are followed by coefficients of a second order, which are followed by coefficients of a third order, and so forth.

12. A method of image decompression in a digital data processor having a CPU that is coupled to an associated GPU comprising the steps of:

A. receiving compressed image values in memory associated with the CPU,

B. transferring the compressed image values to the GPU, the compressed image values comprising coefficients of like order from multiple sub-blocks of a frequency-based representation of an image that are ordered together,

C. with the GPU, re-ordering the compressed image values such that coefficients of like sub-blocks of the frequency-based representation of the image are stored together.

13. The method of claim 12, wherein step (B) further includes decompressing, with the CPU, the compressed image values before transferring the image values to the GPU.

14. The method of claim 12, wherein the step (A) comprises receiving compressed quantized image values in a memory associated with the CPU, wherein the quantized image values comprise coefficients having a data size reduced from a first size to a second size.

15. The method of claim 14, further comprising

D. with the GPU, unquantizing the compressed quantized image values such that the data size of the coefficients are expanded from the second size to the first size.

16. The method of claim 12, further comprising

D. with the GPU, transforming the image values from a frequency-based representation to a color space representation.

17. The method of claim 16, wherein step (D) transforms the image values from a frequency-based representation to a YCbCr color space via an inverse Discrete Cosine Transform (DCT).

18. The method of claim 17, wherein step (D) further comprises transforming, with the GPU, the image values from the YCbCr color space to an RGB color space.

19. The method of claim 17 further comprising

E. with the GPU, generating a three-dimensional image (14D) from the RGB color space image values, and storing that image in a memory.

20. In a digital data processing system, the improvement for image compression comprising:

A. a graphics processing unit (GPU) coupled to a central processing unit (CPU),

B. the GPU transforming image values from a color representation to a frequency-based representation and re-ordering coefficients comprising that frequency-based representation such that coefficients of like order from multiple sub-blocks of the frequency-based representation are ordered together,

C. the CPU receiving the re-ordered coefficients from the GPU and at least any of storing and transmitting the re-ordered coefficients.

21. The digital data processing system of claim 20, wherein the CPU compresses the re-ordered coefficients prior to storing and/or transmitting said coefficients.

22. The digital data processing system of claim 21, wherein the GPU compresses the re-ordered coefficients via any of a run length encoding-based compression technique and Zlib algorithm.

23. The digital data processing system of claim 20, wherein the GPU transforms the image values from an RGB color space to a YCbCr color space before said GPU transforms the image values to the frequency-based representation.

24. The digital data processing system of claim 23, wherein the GPU transforms the image values from the YCbCr color space to the frequency-based representation via a Fourier-related transform.

25. The digital data processing system of claim 23, wherein the GPU transforms the image values from the YCbCr color space to the frequency-based representation via a discrete cosine transform (DCT).

26. The digital data processing system of claim 25, wherein the GPU applies a quantization factor to said coefficients.

27. The digital data processing system of claim 20, wherein the GPU renders the image values to a texture stored in a buffer associated with the GPU.

28. In a digital data processing system, the improvement for image decompression comprising:

A. a graphics processing unit (GPU) coupled to a central processing unit (CPU),

B. the CPU receiving compressed image values in a memory associated with the CPU,

C. the CPU transferring the compressed image values to the GPU, the compressed image values comprising coefficients of like order from multiple sub-blocks of a frequency-based representation of an image that are ordered together,

D. the GPU re-ordering the compressed image values such that coefficients of like sub-blocks of the frequency-based representation of the image are stored together.

29. The digital data processing system of claim 28, wherein the CPU decompresses the compressed image values before transferring the image values to the GPU.

30. The digital data processing system of claim 28, wherein the CPU receives compressed quantized image values in a memory associated with the CPU, wherein the quantized image values comprise coefficients having a data size reduced from a first size to a second size.

31. The digital data processing system of claim 30, further comprising the GPU unquantizing the compressed quantized image values such that the data size of the coefficients are expanded from the second size to the first size.

32. The digital data processing system of claim 28, further comprising the GPU transforming the image values from a frequency-based representation to a color space representation.

33. The digital data processing system of claim 32, wherein the GPU transforms the image values from a frequency-based representation to a YCbCr color space via an inverse Discrete Cosine Transform (DCT).

34. The digital data processing system of claim 33, wherein the GPU transforms the image values from the YCbCr color space to an RGB color space.

35. The digital data processing system of claim 33, wherein the GPU generates a three-dimensional image (4D) from the RGB color space image values and stores that generated image in a memory.

36. A digital data processing system for remote visualization comprising:

A. a digital data processor having a graphics processing unit (GPU) associated with a central processing unit (CPU)

B. the GPU transforming image values from a color representation to a frequency-based representation and ordering coefficients comprising that frequency-based representation such that coefficients of like order from multiple sub-blocks of the frequency-based representation are ordered together,

C. the CPU receiving the ordered coefficients from the GPU and transmitting image values values comprising said coefficients to a remote digital data processor.

37. The digital data processing system of claim 36, wherein the remote digital data processor comprises a remote GPU associated with a remote CPU that receives the transmitted image values.

38. The digital data processing system of claim 37, wherein the remote CPU transfers the image values to the remote GPU, the remote GPU re-ordering the image values such that coefficients of like sub-blocks of the frequency-based representation are stored together.

39. The digital data processing system of claim 38, wherein the remote GPU transforms the image values from a frequency-based representation to a color space representation.

40. The digital data processing system of claim 39, wherein the remote GPU transforms the image values from a frequency-based representation to a YCbCr color space via an inverse Discrete Cosine Transform (DCT).

41. The digital data processing system of claim 40, wherein the remote GPU transforms the image values from the YCbCr color space to an RGB color space.

42. The digital data processing system of claim 38, wherein the remote GPU renders an image from the color space image values and outputs that rendered image to a display coupled to the remote digital data processor.