System and method for data compression

• US 9,859,919 B2
• Filed: 05/02/2017
• Issued: 01/02/2018
• Est. Priority Date: 10/03/2000
• Status: Expired due to Fees

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J., Pickel, Paul F., McErlain, Stephen J., Melone, II, Carlton J.
• Primary Examiner(s): JEANGLAUDE, JEAN BRUNER
• Application Number:
  US15/584,865
• Publication Number:
  US 20170237446A1
• Time in Patent Office: 245 Days
• Field of Search: 341 50, 341 51
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/10 Tax strategies
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H10K 50/80 Constructional details

Abstract

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

Claims

1. A method for compressing data in one or more data blocks, comprising:

analyzing a data block to determine a parameter, attribute, or value of the data block, wherein the analyzing excludes only reading a descriptor or data token associated with the data block;

selecting at least one lossless encoder associated with the determined parameter, attribute, or value;

compressing data in the data block with the selected at least one lossless encoder to produce a compressed data block, having a size over 10 times smaller than the data block; and

storing the compressed data block, wherein the time of the compressing the data block and the storing the compressed data block is less than the time of storing the data block in uncompressed form.

2. The method of claim 1, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a global state machine, the method further comprising selectively storing information regarding the data block in a global state machine such that the information is available to encode additional data blocks.

3. The method of claim 1, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a global state machine, the method further comprising selectively storing the data block in a global state machine such that the data block is available to encode additional data blocks.

4. The method of claim 1, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a local state machine.

5. The method of claim 1, wherein the compressing utilizes at least two or more state machines, and wherein at least one of the two or more state machines is a global state machine, and wherein at least one of the two or more state machines is a local state machine.

6. The method of claim 1, further comprising a plurality of data blocks, wherein the:

analyzing includes analyzing the plurality of data blocks to determine a parameter, attribute, or value of the plurality of data blocks, wherein the analyzing excludes only reading a descriptor or data token associated with the plurality of data blocks;

compressing includes compressing data in the plurality of data blocks with the selected at least one lossless encoder to produce a plurality of compressed data blocks, having a size over 10 times smaller than the plurality of data blocks; and

storing includes storing the plurality of compressed data blocks, wherein the time of the compressing the plurality of data blocks and the storing the plurality of compressed data blocks is less than the time of storing the plurality of data blocks in uncompressed form.

7. The method of claim 1, wherein the selected at least one lossless encoder utilizes a form of mathematical analysis.

8. The method of claim 1, wherein the selected at least one lossless encoder utilizes a dynamically loaded library.

9. A system for compressing data in one or more data blocks, comprising:

a data storage server implemented on one or more processors and one or more memory systems and configured to;

analyze a data block to determine a parameter, attribute, or value of the data block;

wherein the analyzing excludes only reading a descriptor or data token associated with the data block;

select at least one lossless encoder associated with the determined parameter, attribute, or value;

compress data in the data block with the selected at least one lossless encoder to produce a compressed data block, having a size over 10 times smaller than the data block; and

store the compressed data block,

wherein the time of the compressing the data block and the storing the compressed data block is less than the time of storing the data block in uncompressed form.

10. The system of claim 9, wherein to compress the data in the data block, the data storage server is configured to utilize one or more state machines, and wherein at least one of the one or more state machines is a global state machine, and wherein the data storage server is further configured to selectively store information regarding the data block in a global state machine such that the information is available to encode additional data blocks.

11. The system of claim 9, wherein to compress the data in the data block, the data storage server is configured to utilize one or more state machines, and wherein at least one of the one or more state machines is a global state machine, and wherein the data storage server is further configured to selectively store the data block in a global state machine such that the data block is available to encode additional data blocks.

12. The system of claim 9, wherein to compress the data in the data block, the data storage server is further configured to utilize one or more state machines, and wherein at least one of the one or more state machines is a local state machine.

13. The system of claim 9, wherein to compress the data in the data block, the data storage server is further configured to utilize at least two or more state machines, and wherein at least one of the two or more state machines is a global state machine, and wherein at least one of the two or more state machines is a local state machine.

14. The system of claim 9, further comprising a plurality of data blocks, wherein the data storage server is further configured to:

analyze the plurality of data blocks to determine a parameter, attribute, or value of the plurality of data blocks, wherein the analyzing excludes only reading a descriptor or data token associated with the plurality of data blocks;

compress data in the plurality of data blocks with the selected at least one lossless encoder to produce a plurality of compressed data blocks, having a size over 10 times smaller than the plurality of data blocks; and

store the plurality of compressed data blocks, wherein the time of the compressing the plurality of data blocks and the storing the plurality of compressed data blocks is less than the time of storing the plurality of data blocks in uncompressed form.

15. The system of claim 9, wherein the selected at least one lossless encoder utilizes a form of mathematical analysis.

16. The system of claim 9, wherein the selected at least one lossless encoder utilizes a library.

17. A method for compressing data in one or more data blocks, comprising:

analyzing a data block to determine a parameter, attribute, or value of the data block;

wherein the analyzing excludes only reading a descriptor or data token associated with the data block;

selecting at least one lossless compression encoder associated with the determined parameter, attribute, or value;

compressing data in the data block with the selected at least one lossless compression encoder to produce a compressed data block, having a size over 10 times smaller than the data block;

transmitting the compressed data block; and

storing the compressed data block, wherein the time of the compressing, the transmitting, and the storing of the data block is less than the time of transmitting and storing the data block in uncompressed form.

18. The method of claim 17, wherein transmitting the compressed data block further comprises transmitting one or more data packets to a client, wherein the one or more data packets includes control information and compressed data information.

19. The method of claim 17, wherein the data block is transmitted in one or more data packets utilizing Transmission Control Protocol/Internet Protocol (TCP/IP), the one or more data packets including control information and compressed data information.

20. The method of claim 17, wherein transmitting the compressed data block further comprises:

sequencing the compressed data block with one or more additional compressed data blocks to produce a sequence of aggregated data blocks; and

transmitting the sequence of aggregated data blocks.

21. The method of claim 17, further comprising:

after transmitting the compressed data block and before storing the compressed data block, decompressing the compressed data block, wherein the time to compress, transmit, decompress, and store is less than the time of transmitting and storing the data block in uncompressed form.

22. The method of claim 17, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a global state machine, the method further comprising selectively storing information regarding the data block in a global state machine such that the information is available to encode additional data blocks.

23. The method of claim 17, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a global state machine, the method further comprising selectively storing the data block in a global state machine such that the data block is available to encode additional data blocks.

24. The method of claim 17, wherein the compressing utilizes one or more state machines, and wherein at least one of the one or more state machines is a local state machine.

25. The method of claim 17, wherein the compressing utilizes at least two or more state machines, and wherein at least one of the two or more state machines is a global state machine, and wherein at least one of the two or more state machines is a local state machine.

26. The method of claim 17, further comprising a plurality of data blocks, wherein the:

analyzing includes analyzing the plurality of data blocks to determine a parameter, attribute, or value of the plurality of data blocks, wherein the analyzing excludes only reading a descriptor or data token associated with the plurality of data blocks;

compressing includes compressing data in the plurality of data blocks with the selected at least one lossless encoder to produce a plurality of compressed data blocks, having a size over 10 times smaller than the plurality of data blocks; and

storing includes storing the plurality of compressed data blocks, wherein the time of the compressing the plurality of data blocks and the storing the plurality of compressed data blocks is less than the time of storing the plurality of data blocks in uncompressed form.