SYSTEM AND METHOD FOR DATA FEED ACCELERATION AND ENCRYPTION

• US 20090287839A1
• Filed: 06/02/2008
• Published: 11/19/2009
• Est. Priority Date: 10/03/2000
• Status: Active Grant

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J., Pickel, Paul F., McErlain, Stephen J., Melone, Carlton J.
• Granted Patent: US 7,777,651 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 50/188 Electronic negotiation
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

Claims

1-28. -28. (canceled)

29. A method of encoding a plurality of data blocks to create an encoded data packet in a financial data stream using a data encoding engine, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data blocks, the method comprising:

receiving a data block from the plurality of data blocks;

analyzing content of the data block to determine a data block type;

selecting one or more lossless encoders based on the data block type and a computer file, wherein the computer file indicates data block types and associated encoders;

encoding the data block with a selected lossless encoder utilizing content dependent data compression, if the data block type is recognized as associated with a lossless encoder utilizing content dependent data compression;

encoding the data block with a selected lossless encoder utilizing content independent data compression, if the data block type is not recognized as associated with a lossless encoder utilizing content dependent data compression; and

providing a descriptor for the encoded data packet in the financial data stream, wherein the descriptor indicates the one or more selected lossless encoders for the encoded data block.

30. The method of claim 29, wherein the data block corresponds to a data field of a message.

31. The method of claim 29, further comprising including a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

32. The method of claim 31, wherein the encoded data packet in the financial data stream is broadcast to a plurality of client systems.

33. The method of claim 32, wherein the encoded data packet is a User Datagram Protocol (UDP) data packet.

34. The method of claim 29, wherein the encoding comprises packet independent data encoding.

35. The method of claim 29, further comprising providing one or more global state machines and one or more adaptive local state machines.

36. The method of claim 35, further comprising:

storing in one or more of the global state machines data blocks that are likely to repeat in the financial data stream based on a priori knowledge of the data stream; and

storing in one or more of the adaptive local state machines the received data block such that the data block is available to encode one or more other data blocks.

37. The method of claim 36, further comprising resetting one or more of the adaptive local state machines at a determinate point of the encoded data packet.

38. The method of claim 29, wherein encoding the data block utilizing content dependent data compression comprises using a difference between data blocks in the encoded data packet.

39. The method of claim 29, wherein the time of encoding the plurality of data blocks and transmitting the encoded data packet is less than the time to transmit the plurality of data blocks in unencoded form.

40. The method of claim 29, wherein the plurality of data blocks includes one or more of stock, options, and futures information.

41. The method of claim 29, wherein the descriptor comprises values corresponding to a single applied compression technique or multiple compression techniques applied in a specific order.

42. The method of claim 29, wherein the method of encoding a plurality of data blocks to create an encoded data packet in a financial data stream is performed in real-time.

43. A method of encoding a plurality of data blocks to create an encoded data packet for a financial data stream using a data encoding engine, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data blocks, the method comprising:

providing a fixed table of data blocks based on a priori knowledge of the financial data stream;

providing an adaptive table of data blocks;

receiving a data block from the plurality of data blocks;

analyzing content of the data block to determine a data block type;

selecting one or more lossless encoders for the data block based on the data block type and a computer file, wherein the computer file indicates data block types and associated encoders;

encoding the data block with a selected lossless encoder using a data block in said adaptive table identified by said selected lossless encoder, if available, otherwise using a data block in said fixed table identified by said selected lossless encoder;

storing the data block in said adaptive table, such that the data block is available to encode one or more other data blocks; and

providing a descriptor for the encoded data packet, wherein the descriptor indicates the selected one or more lossless encoders for the encoded data block.

44. The method of claim 43, wherein the data block corresponds to a data field of a message.

45. The method of claim 43, further comprising resetting the adaptive table at a determinate point of the encoded data packet.

46. The method of claim 43, further comprising including a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

47. The method of claim 43, wherein encoding the data block comprises packet independent data encoding.

48. The method of claim 43, further comprising encoding one or more of the plurality of data blocks with a selected lossless encoder utilizing content independent data compression, if the data block type is associated with a lossless encoder utilizing content independent data compression.

49. The method of claim 43, wherein encoding the data block with a selected lossless encoder comprises using a difference between the data block and a data block in the adaptive table.

50. The method of claim 43, wherein the method of encoding a plurality of data blocks to create an encoded data packet in a financial data stream achieves a compression ratio of at least 10:

1.

51. The method of claim 43, wherein the method of encoding a plurality of data blocks to create an encoded data packet in a financial data stream is performed in real-time.

52. A method of encoding one or more messages to create an encoded data packet for a financial data stream using a data encoding engine, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data fields of a message, the method comprising:

providing a fixed table of data fields based on a priori knowledge of the financial data stream;

providing an adaptive table of data fields;

receiving a message from the one or more messages;

analyzing content of a data field in the message to determine a data field type;

selecting one or more lossless encoders for the data field based on the data field type and a computer file, wherein the computer file indicates data block types and associated encoders;

encoding the data field with a selected lossless encoder using a data field in said adaptive table identified by said selected lossless encoder, if available, otherwise using a data field in said fixed table identified by said selected lossless encoder;

storing the data field in said adaptive table, such that the data field is available to encode one or more other data fields; and

providing a descriptor for the encoded data packet, wherein the descriptor indicates the selected one or more lossless encoders for the encoded data field.

53. The method of claim 52, further comprising resetting the adaptive table at a determinate point of the data packet.

54. The method of claim 52, further comprising including a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

55. The method of claim 52, wherein encoding the data field comprises packet independent data encoding.

56. The method of claim 52, further comprising encoding one or more data fields with a selected lossless encoder utilizing content independent data compression, if the data field type is associated with a lossless encoder utilizing content independent data compression.

57. The method of claim 52, wherein encoding the data field with a selected lossless encoder comprises using a difference between the data field and a data field in the adaptive table.

58. The method of claim 52, wherein the method of encoding one or more messages to create an encoded data packet for a financial data stream is performed in real-time.

59. The method of claim 52, wherein the method of encoding one or more messages to create an encoded data packet for a financial data stream achieves a compression ratio of at least 10:

1.

60. A system for encoding a plurality of data blocks to create an encoded data packet in a financial data stream, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data blocks, the system comprising:

an input interface that receives a data block;

a data encoding engine operatively connected to said input interface having a computer readable program code of instructions executable by the data encoding engine, said instructions comprising instructions to;

analyze content of the data block to determine a data block type;

select one or more lossless encoders based on the data block type and a computer file, wherein the computer file indicates data block types and associated encoders;

encode the data block with a selected lossless encoder utilizing content dependent data compression, if the data block type is recognized as associated with a lossless encoder utilizing content dependent data compression; and

encode the data block with a selected lossless encoder utilizing content independent data compression, if the data block type is not recognized as associated with a lossless encoder utilizing content dependent data compression; and

an output interface operatively connected to said data encoding engine that outputs a descriptor comprising one or more values in the encoded data packet in the financial data stream, wherein the descriptor indicates the one or more selected lossless encoders.

61. The system of claim 60, wherein the data block corresponds to a data field of a message.

62. The system of claim 60, wherein the system for encoding includes a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

63. The system of claim 60, wherein the data encoding engine performs packet independent data encoding.

64. The system of claim 60, further comprising one or more global state machines and one or more adaptive local state machines operatively connected to said data encoding engine.

65. The system of claim 64, wherein the one or more global state machines store data blocks that are likely to repeat in the financial data stream based on a priori knowledge of the data stream and the one or more adaptive local state machines store the received data block such that the data block is available to encode one or more other data blocks.

66. The system of claim 65, wherein the one or more adaptive local state machines reset at a determinate point of the data packet.

67. A system for encoding one or more messages to create an encoded data packet in a financial data stream, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data fields of a message, the system comprising:

an input interface that receives a message, wherein the message comprises a plurality of data fields;

a data encoding engine operatively connected to said input interface having a computer readable program code of instructions executable by the data encoding engine, said instructions comprising instructions to;

analyze content of a data field of the message to determine a data field type;

select one or more lossless encoders based on the data field type and a description file, wherein the description file indicates data field types and associated encoders;

encode the data field with a selected lossless encoder utilizing content dependent data compression, if the data block type is recognized as associated with a lossless encoder utilizing content dependent data compression; and

encode the data field with a selected lossless encoder utilizing content independent data compression, if the data block type is not recognized as associated with a lossless encoder utilizing content dependent data compression; and

an output interface operatively connected to said data encoding engine that outputs a descriptor in the encoded data packet in the financial data stream, wherein the descriptor indicates the selected one or more lossless encoders.

68. The system of claim 67, wherein the system for encoding includes a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

69. The system of claim 68, wherein the encoded data packet is included in a broadcast to a plurality of client systems.

70. The system of claim 67, wherein the data encoding engine performs packet independent data encoding.

71. The system of claim 67, further comprising one or more global state machines and one or more adaptive local state machines operatively connected to said data encoding engine.

72. The system of claim 71, wherein the one or more global state machines store data fields that are likely to repeat in the financial data stream based on a priori knowledge of the data stream and the one or more adaptive local state machines store the data field such that the data field is available to encode one or more other data fields.

73. The system of claim 72, wherein the one or more adaptive local state machines reset at a determinate point of the data packet.

74. The system of claim 67, wherein the instructions to encode the data field utilizing content dependent data compression comprises using a difference between the content of data fields in the encoded data packet.

75. The system of claim 67, wherein the time of encoding the one or more messages and transmitting the encoded data packet in the financial data stream is less than the time to transmit the one or more messages in unencoded form.

76. The system of claim 67, wherein the one or more messages include one or more of stock, options, and futures information.

77. The system of claim 67, wherein the one or more messages include financial news.

78. The system of claim 67, wherein the system for encoding one or more messages to create an encoded data packet in a financial data stream achieves a compression ratio of at least 10:

1.

79. The system of claim 67, wherein the system for encoding one or more messages to create an encoded data packet in a financial data stream operates in real-time.

80. A system for encoding a plurality of data blocks to create an encoded data packet in a financial data stream, wherein multiple encoders applying a plurality of lossless compression techniques are applied to a plurality of data blocks, the system comprising:

an input interface that receives a data block from the plurality of data blocks;

a memory with a fixed table of data blocks based on a priori knowledge of the financial data stream and an adaptive table of data blocks;

a data encoding engine operatively connected to said input interface and said memory having a computer readable program code of instructions executable by the data encoding engine, said instructions comprising instructions to;

analyze content of the data block to determine a data block type;

select one or more lossless encoders for the data block based on the data block type and a computer file, wherein the computer file indicates data block types and associated encoders;

encode the data block with a selected lossless encoder using a data block in said adaptive table identified by said selected lossless encoder, if available, otherwise using a data block in said fixed table identified by said selected lossless encoder; and

store the data block in said adaptive table, such that the data block is available to encode one or more other data blocks; and

an output interface operatively connected to said data encoding engine that outputs the encoded data packet with a descriptor comprising one or more values, wherein the descriptor indicates the selected one or more lossless encoders.

81. The system of claim 80, wherein the data block corresponds to a data field of a message.

82. The system of claim 80, wherein the memory resets the adaptive table at a determinate point of the data packet.

83. The system of claim 80, wherein the system for encoding includes a plurality of synchronization points in the financial data stream for initiating decoding the financial data stream.

84. The system of claim 80, further comprising instructions executable by the data encoding engine to encode one or more data blocks with a selected lossless encoder utilizing content independent data compression, if the data block type is associated with a lossless encoder utilizing content independent data compression.

85. The system of claim 80, wherein the data encoding engine performs packet independent data encoding.

86. The system of claim 80, wherein the instructions to encode the data block with a selected lossless encoder comprises using a difference between the data block and a data block in the adaptive table.

87. The system of claim 80, wherein the system for encoding a plurality of data blocks to create an encoded data packet in a financial data stream achieves a compression ratio of at least 10:

1.

88. The system of claim 80, wherein the system for encoding a plurality of data blocks to create an encoded data packet in a financial data stream operates in real-time.

89. A method of decoding one or more encoded data packets of a financial data stream using a data decoding engine, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded data packet, the method comprising:

receiving an encoded data packet from the financial data stream having one or more descriptors comprising one or more values, wherein the one or more descriptors indicate lossless encoders used to encode data blocks associated with the encoded data packet, and further wherein the lossless encoders are selected based on analyses of content of the data blocks;

analyzing the encoded data packet of the financial data stream to identify a descriptor;

selecting one or more lossless decoders for a data block associated with the data packet, wherein the selecting is based on the descriptor;

decoding the data block with a selected lossless decoder utilizing content dependent data decompression, if the descriptor indicates the data block is encoded utilizing content dependent data compression; and

decoding the data block with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data block is encoded utilizing content independent data compression.

90. The method of claim 89, wherein the data block corresponds to a data field associated with a message in the encoded data packet of the financial data stream.

91. The method of claim 89, wherein the descriptor comprises values corresponding to a single applied decompression technique or multiple decompression techniques applied in a specific order.

92. The method of claim 89, wherein decoding the data block utilizing content independent data decompression occurs prior to decoding the data block utilizing content dependent data decompression.

93. The method of claim 89, further comprising initiating the method of decoding one or more encoded data packets of a financial data stream using a synchronization point, wherein the financial data stream includes a plurality of synchronization points.

94. The method of claim 93, wherein the one or more encoded data packets of the financial data stream are broadcast to a plurality of client systems.

95. The method of claim 94, wherein the one or more encoded data packets are User Datagram Protocol (UDP) data packets.

96. The method of claim 89, wherein decoding the data block comprises packet independent data decoding.

97. The method of claim 89, further comprising providing one or more global state machines and one or more adaptive local state machines.

98. The method of claim 97, further comprising:

storing in one or more of the global state machines data blocks that are likely to repeat in the financial data stream based on a priori knowledge of the data stream; and

storing in one or more of the adaptive local state machines the decoded data block such that the data block is available to decode one or more other data blocks.

99. The method of claim 98, further comprising resetting one or more of the adaptive local state machines at a determinate point of the data packet.

100. The method of claim 89, wherein decoding the data block utilizing content dependent data decompression comprises using a difference between data blocks in the encoded data packet.

101. The method of claim 89, wherein the time of receiving and decoding the one or more encoded data packets is less than the time to receive the one or more encoded data packets in unencoded form.

102. The method of claim 89, wherein the one or more encoded data packets include one or more of stock, options, and futures information.

103. The method of claim 89, wherein the one or more encoded data packets include financial news.

104. The method of claim 89, wherein the method of decoding one or more encoded data packets of a financial data stream achieves an expansion ratio of at least 1:

10.

105. The method of claim 89, wherein the method of decoding one or more encoded data packets of a financial data stream is performed in real-time.

106. The method of claim 89, wherein the one or more encoded data packets of the financial data stream is transmitted to one or more client systems utilizing TCP/IP.

107. A method of decoding one or more encoded messages of a data packet in a financial data stream using a data decoding engine, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded message, the method comprising:

receiving an encoded message in a data packet of the financial data stream having a plurality of data fields associated with the encoded message and one or more descriptors comprising one or more values, wherein the one or more descriptors indicate data field types of the data fields and lossless encoders used to encode the data fields, and further wherein the lossless encoders are selected based on analyses of content of the data fields;

analyzing the encoded message to identify a descriptor;

selecting one or more lossless decoders for a data field associated with the encoded message, wherein the selecting is based on the descriptor and a description file, and further wherein the description file comprises data field types and associated lossless decoders;

decoding the data field with a selected lossless decoder utilizing content dependent data decompression, if the descriptor indicates the data field is encoded utilizing content dependent data compression; and

decoding the data field with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data field is encoded utilizing content independent data compression.

108. The method of claim 107, wherein the one or more lossless decoders are further selected based upon the specific ordering of the data field in the encoded message.

109. The method of claim 107, wherein the descriptor comprises values corresponding to a single applied decompression technique or multiple decompression techniques applied in a specific order.

110. The method of claim 107, further comprising initiating the method of decoding one or more encoded messages of a data packet in a financial data stream using a synchronization point, wherein the financial data stream includes a plurality of synchronization points.

111. The method of claim 110, wherein the one or more encoded messages of a data packet are included in a broadcast to a plurality of client systems.

112. The method of claim 107, wherein decoding the data field comprises packet independent data decoding.

113. The method of claim 107, further comprising providing one or more global state machines and one or more adaptive local state machines.

114. The method of claim 113, further comprising:

storing in one or more of the global state machines data fields that are likely to repeat in the financial data stream based on a priori knowledge of the data stream; and

storing in one or more of the adaptive local state machines the decoded data field such that the data field is available to decode one or more other data fields.

115. The method of claim 114, further comprising resetting one or more of the adaptive local state machines at a determinate point of the data packet.

116. The method of claim 107, wherein the time of receiving and decoding the one or more encoded messages of a data packet is less than the time to receive the one or more encoded messages of a data packet in unencoded form.

117. The method of claim 107, wherein the method of decoding one or more encoded messages of a data packet in a financial data stream achieves an expansion ratio of at least 1:

10.

118. The method of claim 107, wherein the method of decoding one or more encoded messages of a data packet in a financial data stream is performed in real-time.

119. A method of decoding one or more encoded data packets in a financial data stream using a data decoding engine, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded data packet, the method comprising:

providing a fixed table of data blocks based on a priori knowledge of the financial data stream;

providing an adaptive table of data blocks;

receiving an encoded data packet from the financial data stream having one or more descriptors comprising one or more values, wherein the one or more descriptors indicate lossless encoders used to encode data blocks associated with the encoded data packet, and further wherein the lossless encoders are selected based on analyses of content of the data blocks;

analyzing the encoded data packet to identify a descriptor;

selecting one or more lossless decoders for a data block associated with the encoded data packet, wherein the selecting is based on the descriptor;

decoding the data block with a selected lossless decoder using a data block in said adaptive table identified by said selected lossless decoder, if available, otherwise using a data block in said fixed table identified by said selected lossless decoder; and

storing the decoded data block in said adaptive table, such that the decoded data block is available to decode one or more other data blocks.

120. The method of claim 119, wherein the data block corresponds to a data field associated with a message in the encoded data packet of the financial data stream.

121. The method of claim 119, further comprising resetting the adaptive table at a determinate point of the encoded data packet.

122. The method of claim 119, further comprising initiating the method of decoding one or more encoded data packets in a financial data stream using a synchronization point, wherein the financial data stream includes a plurality of synchronization points.

123. The method of claim 119, wherein decoding the data block comprises packet independent data decoding.

124. The method of claim 119, further comprising decoding one or more data blocks with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data block is encoded utilizing content independent data compression.

125. The method of claim 119, wherein decoding the data block with a selected lossless decoder comprises using a difference between the data block and a data block in the adaptive table.

126. A system for decoding one or more encoded data packets of a financial data stream, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded data packet, the system comprising:

an input interface that receives an encoded data packet from the financial data stream having one or more descriptors comprising one or more values, wherein the one or more descriptors indicate lossless encoders used to encode data blocks associated with the encoded data packet, and further wherein the lossless encoders are selected based on analyses of content of the data blocks;

a data decoding engine operatively connected to said input interface having a computer readable program code of instructions executable by the data decoding engine, said instructions comprising instructions to;

analyze the encoded data packet of the financial data stream to identify a descriptor;

select one or more lossless decoders for a data block associated with the encoded data packet, wherein the selecting is based on the descriptor;

decode the data block with a selected lossless decoder utilizing content dependent data decompression, if the descriptor indicates the data block is encoded utilizing content dependent data compression; and

decode the data block with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data block is encoded utilizing content independent data compression; and

an output interface operatively connected to said data decoding engine that outputs data from the data packet.

127. The system of claim 126, wherein the data block corresponds to a data field associated with a message in the encoded data packet of the financial data stream.

128. The system of claim 126, wherein the descriptor comprises values corresponding to a single applied decompression technique or multiple decompression techniques applied in a specific order.

129. The system of claim 126, wherein the system for decoding initiates decoding the one or more encoded data packets of the financial data stream using a synchronization point, and further wherein the financial data stream includes a plurality of synchronization points.

130. The system of claim 126, wherein the data decoding engine performs packet independent data decoding.

131. The system of claim 126, further comprising one or more global state machines and one or more adaptive local state machines operatively connected to said data decoding engine.

132. The system of claim 131, wherein the one or more global state machines store data blocks that are likely to repeat in the financial data stream based on a priori knowledge of the data stream and the one or more adaptive local state machines store the decoded data block such that the data block is available to decode one or more other data blocks.

133. The system of claim 132, wherein the one or more adaptive local state machines reset at a determinate point of the encoded data packet.

134. The system of claim 126, wherein the system for decoding one or more encoded data packets of a financial data stream achieves an expansion ratio of at least 1:

10.

135. The system of claim 126, wherein the system for decoding one or more encoded data packets of a financial data stream operates in real-time.

136. A system for decoding one or more data packets in a financial data stream, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded data packet, the system comprising:

an input interface that receives an encoded data packet from the financial data stream having one or more descriptors comprising one or more values, wherein the one or more descriptors indicate lossless encoders used to encode data blocks associated with the data packet, and further wherein the lossless encoders are selected based on analyses of content of the data blocks;

a memory with a fixed table of data blocks based on a priori knowledge of the financial data stream and an adaptive table of data blocks;

a data decoding engine operatively connected to said input interface and said memory having a computer readable program code of instructions executable by the data decoding engine, said instructions comprising instructions to;

analyze the encoded data packet to identify a descriptor;

select one or more lossless decoders for a data block associated with the encoded data packet, wherein the selecting is based on the descriptor;

decode the data block with a selected lossless decoder using a data block in said adaptive table identified by said selected lossless decoder, if available, otherwise using a data block in said fixed table identified by said selected lossless decoder; and

store the decoded data block in said adaptive table, such that the decoded data block is available to decode one or more other data blocks; and

an output interface operatively connected to said data decoding engine that outputs data from the data packet.

137. The system of claim 136, wherein the data block corresponds to a data field associated with a message in the encoded data packet of the financial data stream.

138. The system of claim 136, wherein the memory resets the adaptive table at a determinate point of the data packet.

139. The system of claim 136, wherein the system for decoding initiates decoding the one or more data packets in a financial data stream using a synchronization point, and further wherein the financial data stream includes a plurality of synchronization points.

140. The system of claim 136, wherein the data decoding engine performs packet independent data decoding.

141. The system of claim 136, further comprising instructions executable by the data decoding engine to decode one or more data blocks with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data block is encoded utilizing content independent data compression.

142. The system of claim 136, wherein the instructions to decode the data block with a selected lossless decoder comprises using a difference between the data block and a data block in the adaptive table.

143. A system for decoding one or more encoded messages of a data packet in a financial data stream, wherein multiple decoders applying a plurality of lossless decompression techniques are applied to an encoded message, the system comprising:

an input interface that receives an encoded message in a data packet from the financial data stream having a plurality of data fields associated with the encoded message and one or more descriptors comprising one or more values, wherein the one or more descriptors indicate data field types of the data fields and lossless encoders used to encode the data fields, and further wherein the lossless encoders are selected based on analyses of content of the data fields;

a memory with a fixed table of data fields based on a priori knowledge of the financial data stream and an adaptive table of data fields;

a data decoding engine operatively connected to said input interface and said memory having a computer readable program code of instructions executable by the data decoding engine, said instructions comprising instructions to;

analyze the encoded message to identify a descriptor;

select one or more lossless decoders for a data field associated with the encoded message, wherein the selecting is based on the descriptor and a description file, and further wherein the description file comprises data field types and associated lossless decoders;

decode the data field with a selected lossless decoder using a data field in said adaptive table identified by said selected lossless decoder, if available, otherwise using a data field in said fixed table identified by said selected lossless decoder; and

store the decoded data field in said adaptive table, such that the decoded data field is available to decode one or more other data fields; and

an output interface operatively connected to said data decoding engine that outputs data from the data packet.

144. The system of claim 143, wherein the memory resets the adaptive table at a determinate point of the data packet.

145. The system of claim 143, wherein the system for decoding initiates decoding the one or more encoded messages using a synchronization point, wherein the financial data stream includes a plurality of synchronization points.

146. The system of claim 143, wherein the data decoding engine performs packet independent data decoding.

147. The system of claim 143, wherein the instructions to select one or more lossless decoders is further based upon the specific ordering of the data field in the encoded message.

148. The system of claim 143, further comprising instructions executable by the data decoding engine to decode the data field with a selected lossless decoder utilizing content independent data decompression, if the descriptor indicates the data block is encoded utilizing content independent data compression.

149. The system of claim 143, wherein the instructions to decode the data block with a selected lossless decoder comprises using a difference between the data field and a data field in the adaptive table.

150. The system of claim 143, wherein the system for decoding one or more encoded messages of a data packet in a financial data stream achieves an expansion ratio of at least 1:

10.

151. The system of claim 143, wherein the system for decoding one or more encoded messages of a data packet in a financial data stream operates in real-time.