Data feed acceleration

• US 9,667,751 B2
• Filed: 09/14/2015
• Issued: 05/30/2017
• Est. Priority Date: 10/03/2000
• Status: Expired due to Term

• Current Assignee: N/A
• Original Assignee: N/A
• Inventors: Pickel, Paul F., McErlain, Stephen J., Fallon, James J., Melone, II, Carlton J.
• Primary Examiner(s): JEANGLAUDE, JEAN BRUNER
• Application Number:
  US14/853,581
• Publication Number:
  US 20160127513A1
• Time in Patent Office: 624 Days
• Field of Search: 341 50, 341 51
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/10 Tax strategies
• CPC Class Codes: G06T 9/005 Statistical coding, e.g. Huffman, run length coding
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H10K 50/80 Constructional details

Abstract

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

Claims

1. A method for compressing data comprising:

analyzing content of a data block to identify a parameter, attribute, or value of the data block that excludes analyzing based solely on reading a descriptor;

selecting an encoder associated with the identified parameter, attribute, or value;

compressing data in the data block with the selected encoder to produce a compressed data block, wherein the compressing includes utilizing a state machine; and

storing the compressed data block;

wherein the time of the compressing the data block and the storing the compressed data block is less than the time of storing the data block in uncompressed form.

2. The method of claim 1, further comprising transmitting the compressed data block in a data packet to a client, the data packet including both control information and compressed data information.

3. The method of claim 2, wherein the compressed data block is transmitted utilizing Transmission Control Protocol/Internet Protocol (TCP/IP).

4. The method of claim 3, wherein the compressed data block is one of a plurality of compressed data blocks transmitted in sequence.

5. The method of claim 2, wherein the compressed data block is one of a plurality of compressed data blocks transmitted in sequence with a plurality of synchronization points.

6. The method of claim 5, wherein the plurality of compressed data blocks in sequence is transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the beginning of at least one of the UDP Packets.

7. The method of claim 5, wherein the plurality of compressed data blocks in sequence is transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the end of at least some of the UDP Packets.

8. The method of claim 5, wherein the plurality of compressed data blocks in sequence is transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the end of at least one of the UDP Packets.

9. The method of claim 5, wherein the one or more compressed data blocks in sequence have at least one synchronization point per data block.

10. The method of claim 9, wherein the at least one synchronization point is a predetermined byte sequence.

11. The method of claim 1, further comprising transmitting the compressed data block in a packetized data stream having data packets that include control information and compressed data information, and wherein the selected encoder is a packet independent encoder.

12. The method of claim 11, further comprising resetting the state machine for each data packet.

13. The method of claim 1, further comprising:

transmitting the compressed data block and a plurality of other compressed data blocks in a stream of Transmission Control Protocol/Internet Protocol (TCP/IP) packets, the TCP/IP data packets including control information and compressed data information, and

resetting the state machine used in the compressing for each TCP/IP packet.

14. The method of claim 1, wherein the state machine includes one or more global state machines, and further comprising selectively storing data blocks in at least one of the one or more global state machines based on a priori knowledge of data block structure.

15. The method of claim 14, further comprising:

transmitting the compressed data blocks in a packetized data stream of data packets having control and compressed data information, and

resetting the one or more local state machines at a predetermined point of each data packet in the packetized data stream.

16. The method of claim 1, wherein the state machine includes one or more local state machines, and further comprising storing the data in the data block in at least one of the one or more local state machines such that the data is available to encode one or more other data blocks.

17. The method of claim 1, wherein the state machine is a fixed table, and further comprising storing data in the fixed table based on a priori knowledge of data block structure.

18. The method of claim 1, wherein the state machine is an adaptive table, and further comprising storing data from selected data block in the adaptive table such that the data is available to encode one or more other data blocks.

19. The method of claim 18, wherein the compressed data block is to be transmitted in a packetized data stream of data packets having control and compressed data information, and further comprising resetting the adaptive table at a point of each data packet in the packetized data stream.

20. The method of claim 1, wherein the state machine is an adaptive table and the compressed data block is to be transmitted in a packetized data stream of data packets having control and compressed data information, and further comprising resetting the adaptive table at a point of each data packet in the packetized data stream.

21. The method of claim 1, wherein the selected encoder is lossless.

22. The method of claim 1, further comprising compressing the data block with a default lossless encoder when the analyzing of the content of the data block is unable to identify a parameter, attribute, or value of the data block.

23. The method of claim 1, wherein the storing the compressed data block further comprises storing the compressed data block in a buffer.

24. The method of claim 1, further comprising transmitting the compressed data block, wherein the time of the compressing the data block, storing the compressed data block, and transmitting the compressed data block is less than the time of transmitting the data block in an uncompressed form.

25. A system for compressing data comprising:

a data server implemented on one or more processors and one or more memory systems and configured to;

analyze content of a data block to identify a parameter, attribute, or value of the data block that excludes analysis based solely on reading a descriptor;

select an encoder associated with the identified parameter, attribute, or value;

compress data in the data block with the selected encoder to produce a compressed data block, wherein the compression utilizes a state machine; and

store the compressed data block;

wherein the time of the compressing the data block and the storing the compressed data block is less than the time of storing the data block in uncompressed form.

26. The system of claim 25, wherein the data server is further configured to output the compressed data block in a data packet for transmission to a client, the data packet including both control information and compressed data information.

27. The system of claim 26, wherein the compressed data block is to be transmitted in Transmission Control Protocol/Internet Protocol (TCP/IP) Packets and each (TCP/IP) Packet has at least one synchronization point.

28. The system of claim 27, wherein the compressed data block is one of a plurality of compressed data blocks to be transmitted in sequence.

29. The system of claim 26, wherein the compressed data block is one of a plurality of compressed data blocks to be transmitted in sequence with a plurality of synchronization points.

30. The system of claim 29, wherein the plurality of compressed data blocks in sequence is to be transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the beginning of at least one of the UDP Packets.

31. The system of claim 29, wherein the plurality of compressed data blocks in sequence is to be transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the end of at least some of the UDP Packets.

32. The system of claim 29, wherein the plurality of compressed data blocks in sequence is to be transmitted in User Datagram Protocol (UDP) Packets and wherein at least one synchronization point occurs at the end of at least one of the UDP Packets.

33. The system of claim 29, wherein the one or more compressed data blocks in sequence have at least one synchronization point per data block.

34. The system of claim 25, wherein the data server is further configured to transmit the data compressed data block in a packetized data stream having data packets that include control information and compressed data information, and wherein the selected encoder is a packet independent encoder.

35. The system of claim 33, wherein the at least one synchronization point is a predetermined byte sequence.

36. The system of claim 34, wherein the state machine is reset for each data packet.

37. The system of claim 25, wherein a plurality of the compressed data blocks are to be transmitted in a stream of Transmission Control Protocol/Internet Protocol (TCP/IP) packets, the TCP/IP data packets including control information and compressed data information, and

wherein the state machine is reset for each TCP/IP packet.

38. The system of claim 25, wherein the state machine includes one or more global state machines, and the data server is further configured to selectively store data blocks in at least one of the one or more global state machines based on a priori knowledge of data block structure.

39. The system of claim 38, wherein the compressed data blocks are to be transmitted in a packetized data stream, the packets having control and compressed data information and wherein the one or more local state machines are reset at a predetermined point of each data packet in the packetized data stream.

40. The system of claim 25, wherein the state machine includes one or more local state machines, and the data server is further configured to store the data in the data block in at least one of the one or more local state machines such that the data is available to encode one or more other data blocks.

41. The system of claim 25, wherein the state machine is a fixed table, and further comprising storing data in the fixed table based on a priori knowledge of the data block structure.

42. The system of claim 25, wherein the state machine is an adaptive table, and the data server is further configured to store data from selected data block in the adaptive table such that the data is available to encode one or more other data blocks.

43. The system of claim 42, wherein the compressed data blocks are to be transmitted in a packetized data stream, the packets having control and compressed data information, and wherein the adaptive table is reset at a point of each data packet in the packetized data stream.

44. The system of claim 25, wherein the compressed data block is to be transmitted in a packetized data stream, the packets having control and compressed data information and wherein the state machine is an adaptive table reset at a point of each data packet in the packetized data stream.

45. The system of claim 25, wherein the selected encoder is lossless.

46. The system of claim 25, wherein the data server is further configured to compress the data block with a default lossless encoder when the analysis of the content of the data block is unable to identify a parameter, attribute, or value of the data block.

47. The system of claim 25, wherein the data server is further configured to store the compressed data block in a buffer.

48. The system of claim 25, wherein the data server is further configured to transmit the compressed data block, wherein the time of compression of the data block, storage of the compressed data block, and transmission of the compressed data block is less than the time of transmission of the data block in an uncompressed form.