Data feed acceleration

• US 9,141,992 B2
• Filed: 02/23/2012
• Issued: 09/22/2015
• Est. Priority Date: 10/03/2000
• Status: Expired due to Term

• Current Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Original Assignee: Realtime Data LLC (Realtime Data Compression Systems, Inc.)
• Inventors: Fallon, James J., Pickel, Paul F., McErlain, Stephen J., Melone, Carlton J. II
• Primary Examiner(s): JEANGLAUDE, JEAN BRUNER
• Application Number:
  US13/403,785
• Publication Number:
  US 20120194362A1
• Time in Patent Office: 1,307 Days
• Field of Search: 341/51, 341/50, 341/87, 370/355, 380/212, 709/238, 358/400, 345/419, 714/752
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/10 Tax strategies
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H10K 50/80 Constructional details

Abstract

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

Claims

1. A method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form.

2. The method of claim 1, wherein the compressed data stream includes a plurality of compressed data fields and a plurality of synchronization points.

3. The method of claim 2, wherein the compressed data stream is transmitted in User Datagram Protocol (UDP) packets and each UDP packet has at least one synchronization point.

4. The method of claim 3, wherein the at least one synchronization point occurs at the beginning of at least some of the UDP packets.

5. The method of claim 4, wherein each synchronization point is a predetermined byte sequence.

6. The method of claim 3, wherein the at least one synchronization point occurs at the end of at least some of said UDP packets.

7. The method of claim 6, wherein each synchronization point is a predetermined byte sequence.

8. The method of claim 2, wherein the compressed data stream includes discrete messages and each message has at least one synchronization point.

9. The method of claim 1, wherein the compressed data is transmitted in a packetized data stream, and further comprising using a packet independent encoder for compressing the data field for a packet.

10. The method of claim 1, wherein the compressed data is transmitted in a stream of User Datagram Protocol (UDP) packets, and further comprising resetting the state machine for each UDP packet.

11. The method of claim 1, wherein the state machine includes one or more global state machines, and further comprising selectively storing data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

12. The method of claim 1, wherein the state machine includes one or more local state machines, and further comprising storing the data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

13. The method of claim 12, wherein the compressed data is transmitted in a packetized data stream, and further wherein the one or more local state machines are reset at a predetermined point of each data packet in the packetized data stream.

14. The method of claim 1, wherein the state machine is a fixed table, and further comprising storing data in the fixed table based on a priori knowledge of the packet structure.

15. The method of claim 1, wherein the state machine is an adaptive table, and further comprising storing data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields.

16. The method of claim 15, wherein the compressed data is transmitted in a packetized data stream, and further wherein the adaptive table is reset at a point of each data packet in the packetized data stream.

17. A method comprising:

recognizing a parameter, attribute, or value, of a data field in a packet, the packet having, control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine includes one or more global state machines, and the method further comprises selectively storing data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

18. A m method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine includes one or more local state machines, and the method further comprises storing the data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

19. A method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine is an adaptive table, and the method further comprises storing data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields wherein the compressed data is transmitted in a packetized data stream, and further wherein the adaptive table is reset at a point of each data packet in the packetized data stream.

20. A system comprising:

a data server configured to use a processor and memory to;

recognize a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

select an encoder associated with the recognized parameter, attribute, or value;

compress the data in the data field with the selected encoder; and

initiate point-to-point transmission of a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compression utilizes a state machine and the time of the compression and transmission is less than the time to transmit the data in uncompressed form.

21. The system of claim 20, wherein the compressed data stream includes a plurality of compressed data fields and a plurality of synchronization points.

22. The system of claim 21, further comprising a transmitter that point-to-point transmits the compressed data stream in User Datagram Protocol (UDP) packets and each UDP packet has at least one synchronization point.

23. The system of claim 22, wherein the at least one synchronization point occurs at the beginning of at least some of the UDP packets.

24. The system of claim 23, wherein each synchronization point is a predetermined byte sequence.

25. The system of claim 22, wherein the at least one synchronization point occurs at the end of at least some of said UDP packets.

26. The system of claim 25, wherein each synchronization point is a predetermined byte sequence.

27. The system of claim 21, wherein the compressed data stream includes discrete messages and each message has at least one synchronization point.

28. The system of claim 20, wherein the compressed data is transmitted in a packetized data stream, and the data server is further configured to use a packet independent encoder for compressing the data field for a packet.

29. The system of claim 20, wherein the compressed data is transmitted in a stream of User Datagram Protocol (UDP) packets, and the data server is further configured to reset the state machine for each UDP packet.

30. The system of claim 20, wherein the state machine includes one or more global state machines, and the data server is further configured to selectively store data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

31. The system of claim 20, wherein the state machine includes one or more local state machines, and the data server is further configured to store the data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

32. The system of claim 31, wherein the compressed data is transmitted in a packetized data stream, and the one or more local state machines are reset at a predetermined point of each data packet in the packetized data stream.

33. The system of claim 20, wherein the state machine is a fixed table, and the data server is further configured to store data in the fixed table based on a priori knowledge of the packet structure.

34. The system of claim 20, wherein the state machine is an adaptive table, and the data server is further configured to selectively store data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields.

35. The system of claim 34, wherein the compressed data is transmitted in a packetized data stream, and the adaptive table is reset at a point of each data packet in the packetized data stream.

36. A system comprising:

a data server configured to use a processor and memory to;

recognize a parameter, attribute, or value of a data field in a packet, the packet having;

control and data information;

select an encoder associated with the recognized parameter, attribute, or value;

compress the data in the data field with the selected encoder; and

initiate point-to-point transmission of a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and

wherein the compression utilizes a state machine and the time of the compression and transmission is less than the time to transmit the data in uncompressed form, wherein the state machine includes one or more global state machines, and

the data server is further configured to use the memory to selectively store data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

37. A system comprising:

a data server configured to use a processor and memory to;

recognize a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

select an encoder associated with the recognized parameter, attribute, or value;

compress the data in the data field with the selected encoder; and

initiate point-to-point transmission of a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and

wherein the compression utilizes a state machine and the time of the compression and transmission is less than the time to transmit the data in uncompressed form and the state machine includes one or more local state machines, and

the data server is further configured to use the memory to store he data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

38. A system comprising:

a data server configured to use a processor and memory to;

recognize a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

select an encoder associated with the recognized parameter, attribute, or value;

compress the data in the data field with the selected encoder; and

initiate point-to-point transmission of a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and

wherein the compression utilizes a state machine and the time of the compression and transmission is less than the time to transmit the data in uncompressed form, and the state machine is an adaptive table, and

the data server is further configured to use the memory to store data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields wherein the compressed data is transmitted in a packetized data stream, and further wherein the adaptive table is reset at a point of each data packet in the packetized data stream.

39. A non-transitory machine-readable storage medium encoded with machine executable instructions for performing a method on a device, the method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form.

40. The machine-readable storage medium of claim 39, wherein the compressed data stream includes a plurality of compressed data fields and a plurality of synchronization points.

41. The machine-readable storage medium of claim 40, wherein the compressed data stream is transmitted in User Datagram Protocol (UDP) packets and each UDP packet has at least one synchronization point.

42. The machine-readable storage medium of claim 41, wherein the at least one synchronization point occurs at the beginning of at least some of the UDP packets.

43. The machine-readable storage medium of claim 42, wherein each synchronization point is a predetermined byte sequence.

44. The machine-readable storage medium of claim 41, wherein the at least one synchronization point occurs at the end of at least some of said UDP packets.

45. The machine-readable storage medium of claim 40, wherein each synchronization point is a predetermined byte sequence.

46. The machine-readable storage medium of claim 40, wherein the compressed data stream includes discrete messages and each message has at least one synchronization point.

47. The machine-readable storage medium of claim 39, wherein the compressed data is transmitted in a packetized data stream, and further comprising using a packet independent encoder for compressing the data field for a packet.

48. The machine-readable storage medium of claim 39, wherein the compressed data is transmitted in a stream of User Datagram Protocol (UDP) packets, and further comprising resetting the state machine for each UDP packet.

49. The machine-readable storage medium of claim 39, wherein the state machine includes one or more global state machines, and further comprising selectively storing data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

50. The machine-readable storage medium of claim 39, wherein the state machine includes one or more local state machines, and further comprising storing the data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

51. The machine-readable storage medium of claim 50, wherein the compressed data is transmitted in a packetized data stream, and further wherein the one or more local state machines are reset at a predetermined point of each data packet in the packetized data stream.

52. The machine-readable storage medium of claim 39, wherein the state machine is a fixed table, and further comprising storing data in the fixed table based on a priori knowledge of the packet structure.

53. The machine-readable storage medium of claim 39, wherein the state machine is an adaptive table, and further comprising storing data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields.

54. The machine-readable storage medium of claim 53, wherein the compressed data is transmitted in a packetized data stream, and further wherein the adaptive table is reset at a point of each data packet in the packetized data stream.

55. A non-transitory machine-readable storage medium encoded with machine executable instructions for performing a method on a device, the method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine includes one or more global state machines, and the method further comprises selectively storing data fields in at least one of the one or more global state machines based on a priori knowledge of the packet'"'"'s structure.

56. A non-transitory machine-readable storage medium encoded with machine executable instructions for performing a method on device, the method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine includes one or more local state machines, and the method further comprises storing the data in the data field in at least one of the one or more local state machines such that the data is available to encode one or more other data fields.

57. A non-transitory machine-readable storage medium encoded with machine executable instructions for performing a method on a device, the method comprising:

recognizing a parameter, attribute, or value of a data field in a packet, the packet having control and data information;

selecting an encoder associated with the recognized parameter, attribute, or value;

compressing the data in the data field with the selected encoder; and

point-to-point transmitting a compressed data stream to a client, wherein the compressed data stream includes the compressed data, and wherein the compressing utilizes a state machine and the time of the compressing and transmitting is less than the time to transmit the data in uncompressed form, wherein the state machine is an adaptive table, and the method further comprises storing data from selected data fields in the adaptive table such that the data is available to encode one or more other data fields wherein the compressed data is transmitted in a packetized data stream, and further wherein the adaptive table is reset at a point of each data packet in the packetized data stream.