Methods for encoding and decoding data

• US 8,742,958 B2
• Filed: 09/24/2013
• Issued: 06/03/2014
• Est. Priority Date: 10/03/2000
• Status: Expired due to Fees

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J., Pickel, Paul F., McErlain, Stephen J., Melone, II, Carlton J.
• Primary Examiner(s): JEANGLAUDE, JEAN BRUNER
• Application Number:
  US14/035,719
• Publication Number:
  US 20140028480A1
• Time in Patent Office: 252 Days
• Field of Search: 341/50, 341/51, 341/67, 341/79, 341/75, 341/87
• US Class Current: 341/51
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/10 Tax strategies
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H10K 50/80 Constructional details

Abstract

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

Claims

1. A method of decoding, on a packet independent basis using a data decoding engine, one or more encoded messages residing in one or more data blocks of a data packet, the data packet comprising a header containing control information followed by one or more encoded messages of the data packet, the one or more encoded messages including financial information transmitted in sequence from a source, the source internal or external to the decoding engine, wherein at least one of a plurality of decompression techniques are applied to the one or more encoded messages, the method comprising:

identifying one or more recognizable data tokens in an encoded message among the one or more encoded messages, the one or more recognizable data tokens being associated with the data packet and including any recognizable data token indicative of any characteristic, attribute, or parameter of data residing in the one or more data blocks, and wherein encoders were selected based on analysis of content of the data block, wherein the analyzing of the content of the data block excludes analyzing based on a descriptor that is indicative of the any characteristic, attribute, or parameter of the data residing in the one or more data blocks;

applying one or more decoders to a data block from among the one or more data blocks associated with the encoded message based on the one or more recognizable data tokens and a description table, the description table including the characteristics, attributes, or parameters of the data residing in the one or more data blocks and their associated decoders;

decoding the data block with an associated decoder when the recognizable data token indicates that the data block was encoded utilizing content dependent data compression; and

decoding the data block with an associated decoder when the recognizable data token indicates that the data block was encoded utilizing content independent data compression.

2. The method of claim 1, wherein the source of the one or more data blocks containing financial data transmitted in sequence is a different entity than an entity operating the data decoding engine.

3. The method of claim 1, wherein the source of the one or more data blocks containing financial data transmitted in sequence is the same entity as an entity operating the data decoding engine.

4. The method of claim 1, wherein at least one of the decoding the data block steps include decoding when the recognizable data token is included with the data packet.

5. The method of claim 1, wherein at least one of the decoding the data block steps include decoding when the recognizable data token is not included with the data packet.

6. The method of claim 1, wherein the one or more data blocks of financial data transmitted in sequence includes a synchronization point, the method further comprising:

receiving the one or more encoded messages using the synchronization point.

7. The method of claim 1, further comprising:

using one or more global state machines and one or more adaptive local state machines in the decoding process.

8. The method of claim 7, further comprising:

storing, in one or more of the global state machines, data blocks that are likely to repeat in the one or more data blocks of financial data transmitted in sequence based on a priori knowledge of the one or more data blocks transmitted in sequence; and

storing, in one or more of the adaptive local state machines, a decoded data block such that the data field is available to facilitate decoding of the one or more of the data fields corresponding to the decoded data block.

9. The method of claim 1, wherein the decoding is performed in real-time.

10. The method of claim 1, wherein decoders used to decode the one or more data blocks encoded with content independent data compression apply different algorithms than decoders used to decode one or more data blocks encoded with content dependent data compression.

11. The method of claim 1, wherein at least one of the decoders used to decode one or more data blocks encoded with content independent data compression apply algorithms that are the same as decoders used to decode one or more data blocks encoded with content dependent data compression.

12. The method of claim 1, wherein at least one of the decoders used to decode the one or more data blocks encoded with content independent data compression apply the same algorithm as decoders used to decode one or more data blocks encoded with content independent data compression.

13. A system for encoding a plurality of data blocks to create an encoded data packet from one or more data blocks of financial data, wherein the data packet comprises a header containing control information followed by one or more encoded messages of the data packet and the one or more data blocks of financial data are from a source that is external or internal to the encoding system, the system comprising:

a memory with a fixed table of data blocks based on a priori knowledge of the financial data and an adaptive table of data blocks, the memory including a computer readable program code of instructions;

a data encoding engine configured to execute the instructions to analyze data residing in a data block to determine any characteristic, attribute, or parameter of the data block, wherein the analyzing of the data excludes analyzing based on a descriptor that is indicative of the any characteristic, attribute, or parameter of the data block;

one or more encoders selected based upon the characteristic, attribute, or parameter of the data block and a list, wherein the list indicates the characteristic, attribute, or parameter of the data block and its associated encoders, and

wherein the one or more selected encoders are configured to;

encode the one or more data blocks using a data block in the adaptive table identified by the selected encoder, if available, to provide an encoded data packet; and

otherwise

encode the one or more data blocks using a data block in the fixed table identified by the selected encoder and storing the data block in the adaptive table such that the data block is available to encode one or more other data blocks, and

an output interface configured to output the data packet with one or more recognizable data tokens having one or more values, the recognizable data tokens being indicative of the selected one or more encoders.

14. The system of claim 13, wherein the source of the one or more data blocks of financial data transmitted in sequence is a different entity than an entity operating the data encoding engine.

15. The system of claim 13, wherein the source of the one or more data blocks of financial data transmitted in sequence is the same entity as an entity operating the data encoding engine.

16. The system of claim 13, wherein the one or more recognizable data tokens are included with the data packet.

17. The system of claim 13, wherein the one or more recognizable data tokens are not included with the data packet.

18. The system of claim 13, wherein the one or more encoders are further configured to encode one or more encoded messages in real-time.

19. The system of claim 13, wherein a first and a second set of encoders from among the one or more encoders are configured to encode the one or more data blocks using content independent data compression and content dependent data compression, respectively.

20. The system of claim 19, wherein at least one of the first set of encoders is configured to apply a different algorithm than applied by the second set of encoders.

21. The system of claim 19, wherein at least one of the first set of encoders is configured to apply a same algorithm as applied by at least one of the second set of encoders.

22. A method of decoding using a data decoding engine, on a packet independent basis, one or more encoded data packets in one or more data blocks, wherein a data packet among the one or more encoded data packets comprises a header containing control information followed by one or more encoded messages of the data packet and the one or more data blocks including financial information transmitted in sequence from a source that is internal or external to where decoding is performed, the method comprising:

applying, based on one or more recognizable data tokens, one or more decoders to a data block associated with an encoded data packet from among the one or more encoded data packets, the one or more recognizable data tokens having one or more values indicative of encoders used to encode data blocks associated with the encoded data packet;

decoding the data block with an associated decoder when the recognizable data token indicates the data block was encoded utilizing content dependent data compression; and

decoding the data block with an associated decoder when the recognizable data token indicates the data block was encoded utilizing content independent data compression.

23. The method of claim 22, wherein the source of the one or more data blocks of financial data transmitted in sequence is a different entity than an entity performing the data decoding.

24. The method of claim 22, wherein the source of the one or more data blocks of financial data transmitted in sequence is the same entity as an entity performing the data decoding.

25. A system for decoding one or more encoded messages of an encoded data packet within financial data, the financial data comprising one or more encoded data packets including the encoded data packet transmitted in sequence to a data decoding engine, and wherein the encoded data packet comprises a header containing control information followed by the one or more encoded messages of the encoded data packet and the decoding method is performed on a packet independent basis, and wherein at least one of a plurality of decompression decoders are applied to an encoded message, the system comprising:

a data decoding engine configured to decode an encoded message in a data packet, the data packet including one or more data blocks associated with the encoded message and one or more recognizable data tokens associated with the data packet, wherein the recognizable data token is indicative of any characteristic, attribute, or parameter of data residing in the one or more data blocks, and wherein the encoders used to encode the encoded financial data were selected based upon an analysis of content of the one or more data blocks;

wherein the analyzing of the data within the data field excluded analyzing based on a descriptor that is indicative of the any characteristic, attribute, or parameter of the data residing in the one or more data blocks, and

one or more decoders applied to the one or more data blocks associated with the encoded message based on the one or more recognizable data tokens and a description file, the description file including the characteristic, attribute, or parameter of the one or more data blocks and their associated decoders,

wherein the one or more associated decoders are utilized to decode the one or more data blocks based on the one or more recognizable data tokens, the one or more recognizable data tokens being indicative of the one or more data blocks that were encoded utilizing content dependent data compression, content independent data compression, or both.

26. The system of claim 25, wherein the source of the one or more encoded data packets transmitted in sequence is a different entity than an entity operating the data decoding engine.

27. The system of claim 25, wherein the source of the one or more data packets transmitted in sequence is the same entity as an entity operating the data decoding engine.

28. The system of claim 25, wherein the one or more recognizable data tokens are included with the encoded data packet.

29. The system of claim 25, wherein the one or more encoded data packets transmitted in sequence include a plurality of synchronization points, and wherein the one or more encoded messages are received using a synchronization point from among the plurality of synchronization points.

30. The system of claim 25, wherein the one or more recognizable data tokens are not included with the encoded data packet.