System and method for data feed acceleration and encryption

• US 20040042506A1
• Filed: 05/07/2003
• Published: 03/04/2004
• Est. Priority Date: 10/03/2000
• Status: Active Grant

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J., Pickel, Paul F., McErlain, Stephen J., Melone, Carlton J.
• Granted Patent: US 7,417,568 B2
• Time in Patent Office: Days
• US Class Current: 370/521
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 50/188 Electronic negotiation
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

Systems and methods for providing accelerated transmission of broadcast data, such as financial data and news feeds, over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency associated with data transmission of conventional systems.

Claims

1. A method for compressing data, wherein one or more types of encoding are applied to a data stream depending on identifiable data fields in the data stream, the method comprising the steps of:

recognizing a data field type of a data field in a data stream;

selecting an encoder associated with the recognized data field type;

encoding the data in the data field with the selected encoder; and

appending a descriptor to the encoded data which identifies the selected encoder.

2. The method of claim 1, wherein the step of selecting an encoder associated with the recognized data field type is further based on a packet type.

3. The method of claim 2, comprising the step of using a packet independent encoder for encoding a UDP (User Datagram Protocol) data packet.

4. The method of claim 2, comprising the step of selecting packet independent or dependent encoders based on a packet type.

5. The method of claim 1, comprising the step of compressing data of an unrecognized data field using content independent data compression.

6. The method of claim 1, comprising the step of compressing data of an unrecognized packet type using content independent data compression.

7. The method of claim 1, comprising the step of using arithmetic encoding with single character codes to encode data of single character fields.

8. The method of claim 1, comprising the step of using Huffman encoding with small set multiple character codes to encode data of multiple character fields.

9. The method of claim 1, comprising the step of using variable length encoding with large set multiple character codes to encode data of multiple character fields.

10. The method of claim 9, wherein large set multiple character codes utilize truncation of trailing spaces.

11. The method of claim 1, comprising the step of using table-based encoding with large set multiple character codes to encode data of multiple character fields.

12. The method of claim 1, comprising the step of using variable length encoding with variable length codes to encode absolute number values.

13. The method of claim 1, comprising the step of using numerical difference encoding to encode relative numeric values.

14. The method of claim 1, comprising the step of encoding time data based on a difference between a previous time data.

15. The method of claim 1, comprising the step of difference encoding based on a first absolute values within a same data packet for packet independent data compression.

16. The method of claim 1, comprising the step of using difference encoding to encode a message sequence number.

17. The method of claim 1, wherein a date within a packet is encoded only one time if all messages have the same date.

18. The method of claim 17, comprising the step of including date data as part of a message only if the message has a different date.

19. The method of claim 1, comprising the step of including error detection codes in a compressed data packet.

20. A method for creating a data feed dependent data compression routine, comprising the steps of:

creating a description file that describes one or more data fields and one or more encoders associated with each data field;

processing the description file with a data compression compiler; and

outputting an executable file that is used to process a stream of data by recognizing data field types in the data stream and applying encoders associated with the recognized data field types to encode the data stream.

21. The method of claim 20, wherein the output file comprises a dynamic link library.

22. The method of claim 20, wherein the executable file comprises instructions for performing content independent data compression for unrecognized data fields.

23. The method of claim 22, wherein encoder types that are used for the content independent data compression are specified in the description file.

24. A method for creating a data feed dependent data compression routine, comprising the steps of:

creating a description file that describes one or more data fields and one or more encoders associated with each data field;

processing the description file with a data compression compiler; and

outputting source code that is used to process a stream of data by recognizing data field types in the data stream and applying encoders associated with the recognized data field types to encode the data stream.