System and methods for accelerated data storage and retrieval

• US 9,116,908 B2
• Filed: 06/12/2014
• Issued: 08/25/2015
• Est. Priority Date: 03/11/1999
• Status: Expired due to Term

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J.
• Primary Examiner(s): Tran, Philip B
• APPLICATION: US 20150012507A1
• Time in patent office: 439 Days
• Field of Search: 709/231, 709/247
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 17/30153 using compression, e.g. sparse files details of compression H03M7/30; protocols for data compression H04L69/04
• CPC Class Codes: G06F 17/30371 Ensuring data consistency and integrity
• CPC Class Codes: H04L 69/04 Protocols for data compression compression in general H03M7/30; reduction of the amount or size of exchanged application data at an intermediate network processing stage H04L67/2828; optimizing, e.g. header compression, information sizing in wireless communication networks H04W28/06
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/843 on film
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal

Abstract

Systems and methods for providing accelerated data storage and retrieval utilizing lossless data compression and decompression. A data storage accelerator includes one or a plurality of high speed data compression encoders that are configured to compress data. The compressed data is subsequently stored in a target memory or other storage device whose input data storage bandwidth is lower than the original input data stream bandwidth. Similarly, a data retrieval accelerator includes one or a plurality of high speed data decompression decoders that are configured to decompress data at a rate equivalent to or faster than the input data stream from the target memory or storage device. The decompressed data is then output at rate data that is greater than the output rate from the target memory or data storage device.

Claims

1. A system comprising:

a memory device; and

a data accelerator configured to compress;

(i) a first data block with a first compression technique to provide a first compressed data block; and

(ii) a second data block with a second compression technique, different from the first compression technique, to provide a second compressed data block;

wherein the compressed first and second data blocks are stored on the memory device, and the compression and storage occurs faster than the first and second data blocks are able to be stored on the memory device in uncompressed form.

2. The system of claim 1, wherein the data accelerator stores a first data descriptor on the memory device indicative of the first compression technique such that the first descriptor is capable of being utilized to decompress at least a portion of the first data block.

3. The system of claim 2, wherein the data accelerator stores a second data descriptor on the memory device indicative of the second compression technique such that the second descriptor is capable of being utilized to decompress at least a portion of the second data block.

4. The system of claim 2, wherein the data accelerator retrieves the first descriptor and the first compressed data block from the memory device.

5. The system of claim 1, wherein the data accelerator retrieves the first compressed and second data blocks from the memory device.

6. The system of claim 1, wherein the data accelerator retrieves the first compressed data block from the memory device and decompresses the first compressed data block.

7. The system of claim 6, wherein the retrieval of the first compressed data block from the memory device and the decompression occurs faster than the first data block is able to be retrieved from the memory device in uncompressed form.

8. The system of claim 1, wherein the data accelerator is coupled to the memory device via an industry standard disk interface.

9. The system of claim 1, wherein the first compression technique applied to the first data block is a fonn of dictionary compression and the second compression technique applied to the second data block is a form of Lempel-Ziv compression.

10. The system of claim 1, wherein the first compression technique includes compressing with Lempel-Ziv encoding.

11. The system of claim 1, wherein the first compression technique includes compressing with a form of dictionary encoding.

12. The system of claim 1, wherein the first compression technique includes compressing with a plurality of encoders in a serial configuration.

13. The system of claim 1, wherein the first compression technique includes compressing with a plurality of encoders in a parallel configuration, each of the plurality of encoders having an identical type.

14. The system of claim 1, wherein the data accelerator is configured to compress a third data block with a third compression technique to provide a third compressed data block.

15. The system of claim 14 wherein the first compression technique is content dependent, the second compression technique is a form of dictionary compression, and the third compression technique is a different form of dictionary compression.

16. The system of claim 15, wherein the third data block is compressed by the third compression technique in real-time.

17. The system of claim 15, wherein the third data block is compressed by the third compression technique not in real-time.

18. The system of claim 1, wherein the first and second data blocks comprise audio or video information.

19. The system of claim 1, wherein the first and second data blocks are received over a communications channel.

20. The system of claim 1 wherein the first compression technique is content dependent and the second compression technique is a form of dictionary compression.

21. A method for accelerated data storage of data, comprising:

compressing a first data block with a first data compression technique to provide a first compressed data block; and

compressing a second data block with a second data compression technique to provide a second compressed data block, wherein the first data compression technique and the second data compression technique are different;

storing the first and second data compressed blocks on a memory device wherein the compression and storage occurs faster than the first and second data blocks are able to be stored on the memory device in uncompressed form.

22. The method of claim 21, further comprising:

storing, on the memory device, a first data descriptor indicative of the first compression technique such that the first descriptor is capable of being utilized to decompress the first compressed data block.

23. The method of claim 21, wherein retrieval of the first compressed data block from the memory device and decompression of the first compressed data block occurs faster than the first data block is able to be retrieved from the memory device in uncompressed form.

24. The system of claim 21, wherein the first compression technique applied to the first data block is a form of dictionary compression and the second compression technique applied to the second data block is a form of Lempel-Ziv compression.

25. A method for accelerated storage of data, comprising:

receiving a first and a second data block over a communications channel;

compressing the first data block with a first data compression technique to provide a first compressed data block;

compressing the second data block with a second data compression technique to provide a second compressed data block, wherein the first data compression technique and the second data compression technique are different; and

storing the first and second data compressed blocks on a memory device wherein the compression and storage occurs faster than the first and second data blocks are able to be stored on the memory device in uncompressed form.

26. The method of claim 25, further comprising:

determining a bandwidth at which the first and the second data blocks are received; and

adjusting a system parameter to make a bandwidth of the first and compressed second data blocks compatible with a bandwidth of the memory device.

27. The method of claim 25, wherein a data token is associated with the first and second compressed data blocks, and wherein the token includes values corresponding to a one or more encoding techniques that were applied to either or both of the first and second compressed data blocks.

28. The method of claim 25, wherein retrieval of compressed first data block from the memory device and decompression occurs faster than the first data block is able to be retrieved from the memory device in uncompressed form.

29. A method for accelerated retrieval of data comprising:

retrieving a first compressed data block and a second compressed data block from a memory device; and

decompressing the first compressed data block and the second compressed data block, wherein the retrieval and decompression occurs faster than the first data block is able to be retrieved from the memory device in uncompressed form;

wherein the first compressed data block was compressed with a first data compression technique to provide a first compressed data block;

wherein the second compressed data block was compressed with a second data compression technique to provide a second compressed data block, wherein the first data compression technique and the second data compression techniques are different; and

wherein the first and second data compression blocks were stored on the memory device.

30. The method of claim 29, wherein the first compression technique applied to the first data block was a form of dictionary compression and the second compression technique applied to the second data block was a form of Lempel-Ziv compression.