Hardware accelerated compression

• US 7,051,126 B1
• Filed: 08/19/2003
• Issued: 05/23/2006
• Est. Priority Date: 08/19/2003
• Status: Active Grant

• Current Assignee: F5 Networks, Inc. (F5 Incorporated)
• Original Assignee: F5 Networks, Inc. (F5 Incorporated)
• Inventors: Franklin, Jason G.
• Primary Examiner(s): Fleming, Fritz
• Assistant Examiner(s): PATEL, NIKETA I
• Application Number:
  US10/643,459
• Time in Patent Office: 1,008 Days
• Field of Search: 710/65, 710/66, 710/68, 709/247
• US Class Current: 710/68
• CPC Class Codes: G06F 13/122 where hardware performs an I/O function other than control of data transfer
• CPC Class Codes: H03M 7/3084 using adaptive string matching, e.g. the Lempel-Ziv method

Abstract

A compression system is arranged to use software and/or hardware accelerated compression techniques to increase compression speeds and enhance overall data throughput. A logic circuit is arranged to: receive a data stream from a flow control processor, buffer the data stream, select a hardware compressor (e.g., an ASIC), and forward the data to the selected hardware compressor. Each hardware compressor performs compression on the data (e.g., LZ77), and sends the compressed data back to the logic circuit. The logic circuit receives the compressed data, converts the data to another compressed format (e.g., GZIP), and forwards the converted and compressed data back to the flow control processor. History associated with the data stream can be stored in memory by the flow control processor, or in the logic circuit.

Claims

1. An apparatus for compressing data from a data stream comprising:

a hardware compressor that is arranged to provide compressed data in response to buffered data, wherein the compressed data is encoded according to a first compression format with a first byte window size;

a logic block that is arranged to;

receive data associated with the data stream, store the data as the buffered data, forward the buffered data to the hardware compressor, receive the compressed data from the hardware compressor, convert the compressed data to a second compression format with a second byte window size that is different from the first byte window size, and provide the converted compressed data; and

a flow controller that is arranged to;

monitor system resources associated with the flow controller, algorithmically compress selected data streams with the flow controller when the resources associated with the flow controller are below a minimum criteria, and forward data streams to the logic block for compression when the resources associated with the flow controller exceed the minimum criteria such that compression is offloaded to the logic block;

wherein the minimum criteria comprises at least one of;

a processor usage criteria, a memory usage criteria, and a network traffic loading criteria.

2. The apparatus of claim 1, wherein the first compression format is an ALDC format and the second format is a GZIP format.

3. The apparatus of claim 1, wherein the first compression format is an LZS format and the second format is a GZIP format.

4. The apparatus of claim 1, wherein the first byte window size is 512 bytes and the second byte window size is 32 K bytes.

5. The apparatus of claim 1, wherein the first byte window size is 4 K bytes and the second byte window size is 32 K bytes.

6. The apparatus of claim 1, the logic block comprising:

a compression controller that is arranged in cooperation with a memory, wherein the compression controller is arranged to control the interaction of the logic block with the hardware compressor.

7. The apparatus of claim 6, wherein context switching is maintained in the memory and the compression controller is configured to provide history information to the hardware compressor based on a context stored in the memory.

8. The apparatus of claim 6, further comprising another hardware compressor that is arranged to provide compressed data in response to another buffered data, wherein the other compressed data is encoded according to the first compression format with the first byte window size;

and wherein the logic block further comprises a multiplexer that is arranged to select one of the hardware compressors for operating on the buffered data.

9. The apparatus of claim 1, wherein the flow controller is further arranged to receive requests for data, retrieve the requested data, and provide compressed data in response to the requested data.

10. The apparatus of claim 1, wherein the flow controller includes a flow control processor and a context memory, wherein context switching is maintained by the flow controller in the context memory, and wherein the flow controller is arranged to provide history information to the logic block based on a context stored in the context memory.

11. A method for compressing data associated with a data stream comprising:

receiving data from a flow controller with a logic block when resources associated with the flow controller exceed a minimum criteria, wherein the minimum criteria comprises at least one of;

a processor usage associated with the flow controller, a memory usage associated with the flow controller, and a network traffic loading associated with the flow controller;

storing the received data as buffered data;

selecting a hardware compressor for processing the buffered data;

sending the buffered data from the logic block to the selected hardware compressor;

compressing the buffered data according to a first compression format with the selected hardware compressor, wherein the first compression format has a first history window size;

retrieving the compressed data from the selected hardware compressor with the logic block;

converting the compressed data to a second compression format with the logic block, wherein the second compression format has a second history window size;

assembling a header with the logic block, wherein the header associates the converted compressed data with the data from the flow controller; and

sending the header and the converted compressed data to the flow controller.

12. The method of claim 11, wherein the first history window size is different from the second history window size.

13. The method of claim 11, wherein the first compression format is at least one of an ALDC format, an LZ77 format, and an LZS format.

14. The method of claim 11, wherein the second compression format is a GZIP deflate format.

15. The method of claim 11, further comprising:

extracting history information from the data received by the logic block from the flow controller, sending the history information to the selected hardware compressor, updating the history information in response to the first compressed data with the logic block, and sending the updated history information to the flow controller.

16. The method of claim 15, wherein the history information is extracted from another header that is provided by the flow controller to the logic block, wherein the other header includes at least one of a context location flag, a reply ID, a context data, a first block flag, and a byte count.

17. The method of claim 11, further comprising:

retrieving history information that is associated with the data stream from a memory that is associated with the logic block, sending the history information to the selected hardware compressor, updating the history information in response to the first compressed data with the logic block,.

18. The method of claim 17, further comprising:

assembling another header with the logic block such that the history information is associated with at least one of a context location flag, a reply ID, a context data, a first block flag, and a byte count, wherein sending the buffered data includes sending the other header to the selected hardware compressor.

19. The method of claim 11, further comprising:

compressing another buffered data according to the first compression format with another selected hardware compressor, wherein the hardware compressor and the other hardware compressor are operable in parallel for enhanced compression speeds.

20. A system for compressing data associated with a data stream, comprising:

a first hardware compressor that is arranged to compress data according to a first compression format;

a second hardware compressor that is arranged to compress data according to the first compression format;

a flow controller that is arranged to monitor system resources associated with the flow controller, wherein the flow controller is also arranged to offload compression based upon the monitored system resource, wherein the monitored system resources comprise at least one of;

a processor usage in the flow controller, a memory usage in the flow controller, and a network traffic loading effect in the flow controller; and

a logic block that includes;

a first means for receiving that is arranged to receive data from the flow controller when the monitored process loading exceeds a processing criteria associated with the flow controller;

a means for storing the received data as buffered data;

a means for selecting one of the first and second hardware compressors for processing the buffered data;

a first means for sending that is arranged to send buffered data to the selected hardware compressor;

a second means for receiving that is arranged to receive compressed data from the selected hardware compressor, wherein the compressed data is arranged according to a first compression format;

a means for converting the compressed data to a second compression format that is different from the first compression format; and

a second means for sending that is arranged to send converted compressed data to the flow controller such that the converted compressed data is associated with the data received from the flow controller.

21. The system of claim 20, wherein the flow controller is arranged to:

compress data according to the second compression format when sufficient resources in the flow controller are available, and send data to the logic block when insufficient resources in the flow controller are available.

22. A method for compressing data associated with a flow, comprising:

receiving the data with a flow controller;

evaluating system resources associated with the flow controller;

selecting a software based compression method when sufficient system resources are available in the flow controller;

selecting a hardware based compression method when insufficient system resources are available in the flow controller; and

compressing the data according to the selected compression method based on the evaluated system resources.

23. The method of claim 22, wherein evaluating system resources associated with the flow controller comprises at least one of:

evaluating processor usage associated with the flow controller, evaluating processor idle time associated with the flow controller, evaluating memory usage associated with the flow controller, evaluating traffic loading associated with the flow controller, and evaluating history buffer sizes associated with the flow controller.

24. The method of claim 22, further comprising:

maintaining context information with the flow controller, wherein the context information is associated with each flow.

25. The method of claim 22, farther comprising:

maintaining context information with the flow controller, wherein the context information is associated with each flow, and passing the context information to a logic block when the hardware based compression method is selected.

26. The method of claim 22, wherein the hardware compression method comprises:

receiving data from the flow controller when insufficient system resources are available in the flow controller for software based compression;

storing the received data as buffered data;

selecting a hardware compressor for processing the buffered data;

compressing the buffered data according to a compression format with the selected hardware compressor;

converting the compressed data to another compression format; and

sending the converted compressed data to the flow controller.

27. The method of claim 26, further comprising:

compressing another buffered data according to the compression format with another selected hardware compressor, wherein the hardware compressor and the other hardware compressor are operable in parallel for enhanced compression speeds.

28. The method of claim 26, wherein the compression format and the other compression format are different from one another.

29. An apparatus for compressing a data stream associated with a flow, comprising:

a hardware compressor that is arranged to provide compressed data in response to the data stream;

a software compressor that is arranged to provide compressed data in response to the data stream; and

a flow controller that is arranged to;

evaluate system resources associated with the flow controller, select the software compressor for compression of the data stream when sufficient system resources are available in the flow controller, and select the hardware compressor for compression when insufficient system resources are available in the flow controller.

30. The apparatus of claim 29, wherein the flow controller is arranged to evaluate system resources by evaluating at least one of:

processor usage associated with the flow controller, and idle time associated with the flow controller.

31. The apparatus of claim 29, wherein the flow controller is arranged to evaluate system resources by evaluating memory usage associated with the flow controller.

32. The apparatus of claim 29, wherein the flow controller is arranged to evaluate system resources by evaluating traffic loading associated with the flow controller.

33. The apparatus of claim 29, wherein the flow controller is arranged to evaluate system resources by evaluating history buffer sizes associated with the flow controller.

34. The apparatus of claim 29, wherein the flow controller is further arranged to maintain context information that is associated with each flow.

35. The apparatus of claim 34, further comprising:

a logic block that is arranged to receive the context information from the flow controller when the hardware compressor is selected by the flow controller for compression.

36. The apparatus of claim 29, further comprising:

a logic block that is arranged in cooperation with the hardware compressor to;

receive data from the flow controller when insufficient system resources are available in the flow, store the received data as buffered data;

forward the buffered data to the hardware compressor for compression, receive compressed data from the hardware compressor, convert the compressed data received from the hardware compressor to another compression format with the logic block, and send the converted compressed data to the flow controller.

37. The apparatus of claim 36, further comprising:

another hardware compressor that is arranged to operate in parallel with the hardware compressor for enhanced compression speeds.

38. The apparatus of claim 36, wherein a compression format of the hardware compressor has a first byte window size, and wherein the other compression format has a second byte windows size that is different from the first byte window size.