DATA COMPRESSION SYSTEMS AND METHODS

• US 20140022098A1
• Filed: 09/24/2013
• Published: 01/23/2014
• Est. Priority Date: 12/11/1998
• Status: Active Grant

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: FALLON, James J.
• Granted Patent: US 8,717,203 B2
• Time in Patent Office: Days
• US Class Current: 341/87
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/3088 employing the use of a dictionary, e.g. LZ78
• CPC Class Codes: H03M 7/40 Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

Systems and methods for providing fast and efficient data compression using a combination of content independent data compression and content dependent data compression. In one aspect, a method for compressing data comprises the steps of: analyzing a data block of an input data stream to identify a data type of the data block, the input data stream comprising a plurality of disparate data types; performing content dependent data compression on the data block, if the data type of the data block is identified; performing content independent data compression on the data block, if the data type of the data block is not identified.

Claims

1. A method of decompressing in a data decompression engine one or more compressed data blocks included in one or more data packets, the one or more data packets being transmitted in sequence from one of an internal or an external source to the data decompression engine, the method comprising:

applying a plurality of decompression techniques to the one or more compressed data blocks using the data decompression engine;

identifying one or more associated recognizable data tokens of a data packet from among the one or more data packets,

wherein the one or more associated recognizable data tokens identifies a selected compression encoder used to compress the one or more compressed data blocks associated with the data packet, and

wherein the selected compression encoder was selected based on content of a data block on which a compression algorithm was applied to provide the compressed data block;

applying one or more decompression decoders to the compressed data block corresponding to the data packet based on the one or more associated recognizable data tokens;

decompressing the compressed data block with an appropriate decompression decoder if the one or more associated recognizable data tokens indicates that the data block was encoded utilizing content dependent data compression; and

decompressing the compressed data block with an appropriate decompression decoder if the one or more associated recognizable data tokens indicates that the data block was encoded utilizing content independent data compression.

2. The method of claim 1, wherein the decompressing is performed by an entity different than the compression encoder performing the compression.

3. The method of claim 1, wherein the one or more recognizable data tokens include values corresponding to one or more applied decoding techniques to be applied by the one or more decompression decoders.

4. The method of claim 1, wherein the step of decompressing the compressed data block with the appropriate decompression decoder when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content independent data compression occurs prior to the step of decompressing the compressed data block with the appropriate decompression decoder when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content dependent data compression.

5. The method of claim 1, wherein the applying the one or more decompression decoders further comprises:

applying a plurality of the one or more decompression decoders to decompress a plurality of compressed data blocks from among the one or more compressed data blocks corresponding to the data packet.

6. The method of claim 5, wherein the applying the one or more decompression decoders further comprises:

applying a plurality of the one or more decompression decoders to decompress a plurality of the one or more data packets to provide a plurality of decompressed data packets.

7. The method of claim 6, further comprising:

generating one or more decompressed data blocks in sequence from the plurality of decompressed data packets.

8. The method of claim 1, wherein the method of decompressing the one or more compressed data blocks is performed in real-time.

9. The method of claim 1, wherein the one or more decompression decoders and their associated compression encoders utilize lossy techniques to permit data that was selected and compressed by the selected compression encoder to be fully recovered when decompressed by the one or more decompression decoders.

10. The method of claim 1, wherein the one or more decompression decoders and their associated compression encoders utilize lossy techniques.

11. The method of claim 1, wherein the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content independent data compression applies a different decompression algorithm than the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content dependent data compression.

12. The method of claim 1, wherein the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content independent data compression applies the same decompression algorithm than the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content dependent data compression.

13. The method of claim 1, wherein the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content independent data compression is the same decompression decoder as the appropriate decompression decoder selected when the one or more recognizable data tokens indicate that the compressed data block was encoded utilizing content dependent data compression.

14. A system for decompressing, one or more compressed data blocks included in one or more data packets using a data decompression engine, the one or more data packets being transmitted in sequence from a source that is internal or external to the data decompression engine, the system comprising:

a data decompression processor configured to analyze a data packet from among the one or more data packets to identify one or more recognizable data tokens associated with the data packet, the one or more recognizable data identifying a selected encoder used to compress one or more data blocks to provide the one or more compressed data blocks, the encoder being selected based on content of the one or more data blocks on which a compression algorithm was applied;

one or more decompression decoders configured to decompress a compressed data block from among the one or more compressed data blocks associated with the data packet based on the one or more recognizable data tokens;

wherein;

the one or more decompression decoders are further configured to decompress the compressed data block utilizing content dependent data decompression to provide a first decompressed data block when the one or more recognizable data tokens indicate that the data block was encoded utilizing content dependent data compression; and

the one or more decompression decoders are further configured to decompress the compressed data block utilizing content independent data decompression to provide a second decompressed data block when the one or more recognizable data tokens indicate that the data block was encoded utilizing content independent data compression; and

an output interface, coupled to the data decompression engine, configured to output a decompressed data packet including the first or the second decompressed data block.

15. The system of claim 14, wherein the one or more decompression decoders are separate from the selected encoder used to compress the one or more data blocks.

16. The system of claim 14, wherein the one or more recognizable data tokens include values corresponding to one or more applied decoding techniques utilized by the one or more decompression decoders.

17. The system of claim 14, wherein the one or more decompression decoders are further configured to decompress the compressed data block utilizing one or more appropriate decoders for the data block encoded with content independent data compression prior to further decoding the data block utilizing one or more appropriate decoders for the data block encoded with content dependent data compression.

18. The system of claim 14, wherein the one or more decompression decoders are further configured to decompress a plurality of the one or more compressed data blocks corresponding to the data packet.

19. The system of claim 18, wherein the one or more decompression decoders are further configured to decompress a plurality of the one or more data packets.

20. The system of claim 14, wherein the one or more decompression decoders are further configured to decompress the compressed data block in real-time.

21. A method of compressing a plurality of data blocks residing in data packets, the data packets being transmitted in sequence from a source that is internal or external to a source of the compression method, the one or more compressed data packets including one or more compressed data blocks, the method comprising:

applying a plurality of compression techniques to the plurality of data blocks using a data compression engine;

analyzing content of a data block from among the plurality of data blocks to determine any characteristic, attribute, or parameter of the data block;

selecting one or more encoders based on the characteristic, attribute, or parameter of the data block and a file, wherein the file indicates the characteristic, attribute, or parameter of the data block and their associated encoders;

compressing the data block with the selected one or more encoders utilizing content dependent data compression when the characteristic, attribute, or parameter of the data residing within the data block is recognized as associated with an encoder utilizing content dependent data compression;

compressing the data block with the selected one or more encoders utilizing content independent data compression when the characteristic, attribute, or parameter of the data residing within the data block is not recognized as associated with an encoder utilizing content dependent data compression; and

providing a recognizable data token which identifies the selected one or more encoders utilized for compression of the data block in the one or more data blocks transmitted in sequence.

22. The method of claim 21, wherein a plurality of the one or more encoders are applied to compress the plurality of data blocks.

23. The method of claim 21, further comprising:

transmitting one or more data packets in sequence from outputs of the one or more encoders.

24. The method of claim 21, wherein the data blocks contain financial data.

25. The method of claim 21, wherein the method of compressing the plurality of data blocks is performed in real-time.

26. The method of claim 21, wherein the encoders are lossy, and wherein the compression permits the compressed data block to be recovered in full when decompressed.

27. A system for compressing a plurality of data blocks residing in a data packet, the data packets being transmitted in sequence from a source that is internal or external to the compression system, the system comprising:

a data compression processor configured to analyze content of a data block to determine any characteristic, attribute, or parameter of the data block;

one or more compression encoders configured to be selected based on the characteristic, attribute, or parameter of the data block and a file, wherein the file indicates the characteristic, attribute, or parameter of the data block and their associated compression encoders;

wherein;

the data block is compressed by the selected one or more compression encoders utilizing content dependent data compression when the characteristic, attribute, or parameter of the data block is recognized as being associated with an encoder utilizing content dependent data compression; and

the data block is compressed by the selected one or more compression encoders utilizing content independent data compression when the characteristic, attribute, or parameter of the data block is not recognized as being associated with an encoder utilizing content dependent data compression; and

an output interface, coupled to the data compression processor, configured to output a recognizable data token identifying the selected one or more compression encoders, the recognizable data token including any recognizable data token representative of one or more values in the compressed data packet.

28. The system of claim 27, wherein the one or more compression encoders are further configured to compress a plurality of data blocks associated with the compressed data packet.

29. The system of claim 28, wherein the one or more compression encoders are further configured to output a plurality of compressed data blocks.

30. The system of claim 27, wherein the one or more compression encoders are further configured to compress the data blocks in real-time.