Method and system for processing raw financial data streams to produce and distribute structured and validated product offering data to subscribing clients

• US 20020016839A1
• Filed: 05/31/2001
• Published: 02/07/2002
• Est. Priority Date: 08/04/2000
• Status: Active Grant

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Cowan, Randy, Thomas, Deane, Smith, Andrew J.R.
• Granted Patent: US 7,958,251 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/06 Asset management; Financial planning or analysis

Abstract

A method and system is disclosed for processing information provided from various content providers in the form objects having states subject to periodic updates. Formatted information indicating a current state of the objects are delivered to subscribing clients in substantially real-time. An information manager module is configured to receiving raw data objects from the content providers, format the information, and broadcast current state information on various broadcast data stream. A client manager module supports multiple client communication sessions and connects to at least one broadcast data stream. Current state information received on a broadcast data stream is processed and transmitted to specific clients in accordance with processing rules specified in the client profiles.

Claims

1. A method for processing information provided from at least one content provider about the state of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the method comprising the steps of:

in an information manager;

receiving raw data objects on at least one raw data stream input;

generating a formatted data object from a received raw data object;

storing a current state of the formatted data object in an object storage pool; and

broadcasting the current state of the formatted data object on a particular broadcast data stream;

in a client manager;

establishing communication sessions with a plurality of clients;

connecting to at least one broadcast data stream;

receiving on a connected broadcast data stream a current state for a specific data object;

updating an object pool cache to reflect the current state of the specific data object; and

transmitting the current state of the specific data object to a set of clients selected from the plurality of clients.

2. The method of claim 1, wherein the step of broadcasting the current state of the formatted data object comprises:

determining if a prior version of the formatted data object was present in the object storage pool,

if a prior version of the formatted data object was present, determining a data differential between the prior version and the current state of the formatted data object and broadcasting the data differential on the particular broadcast data stream;

otherwise, broadcasting the current state of the formatted data object on the particular broadcast data stream;

3. The method of claim 1, wherein each client has an associated profile comprising data indicating data stream subscriptions and at least one object rule associated with the subscribed data streams;

the step of transmitting the current state of the specific data object to a set of clients comprising the steps of;

for each respective client subscribed to the particular input data stream, evaluating from the client profile associated with the respective client the object rules associated with the particular input data stream against the specific data object; and

transmitting the current state of the specific data object to the respective client in response to a positive evaluation.

4. The method of claim 1, wherein the step of connecting to at least one broadcast data stream comprises the steps of:

connecting to a first broadcast data stream from a first information manager; and

connecting to a second data broadcast stream from a second information manager.

5. The method of claim 1, wherein the step of broadcasting the current state comprises broadcasting a corresponding sequence number associated with the current state.

6. The method of claim 1, further comprising the step of determining an object type of the raw data object;

the step of generating the formatted data object comprising the step of applying a set of formatting rules to the received raw data object in accordance with the object type.

7. The method of claim 6, further comprising the step of translating the raw data object into a raw event comprising at least one name-value pair prior to performing the steps of determining an object type of the raw data object and generating a formatted data object.

8. The method of claim 1, further comprising the step of determining an object type of the raw data object;

the particular broadcast data stream being selected from a plurality of broadcast data streams according to the object type.

9. The method of claim 1, further comprising the steps of:

validating the contents of the raw data object; and

upon a failed validation, preventing subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

10. The method of claim 1, wherein the raw data object comprises information related to a financial product offering.

11. The method of claim 1, further comprising the step of, in the client manager:

after connecting to a particular broadcast data stream, initializing the object pool cache with an initial state of data objects carried on the particular broadcast data stream.

12. The method of claim 1, further comprising the step of obtaining the initial state of data objects from the information manager generating the particular broadcast data stream.

13. The method of claim 11, further comprising the step of, after establishing a communication session with a particular client, delivering to the particular client a snapshot of a set of data objects in the object pool cache which are carried on broadcast data streams to which the particular client is subscribed.

14. The method of claim 1, further comprising the step of, in the client manager:

in response to a detection that a particular client has subscribed to a new broadcast data stream not in a set of connected broadcast data streams, connecting to the new broadcast data stream.

15. The method of claim 14, further comprising the steps of, in the client manager:

initializing the object pool cache with an initial state of data objects carried on the new input data stream; and

delivering to the particular client a snapshot of a set of the data objects in the object pool cache associated with the new data stream.

16. The method of claim 1, wherein each connected client has a respective client event queue, the step of transmitting the current state of the specific data object to the set of clients comprises the steps of, for each respective client in the set of clients:

placing a state event in the client event queue associated with the respective client, the state event indicating the current state of the particular data object; and

subsequently transmitting a client event derived from the queued event to the respective client.

17. The method of claim 16, further comprising the steps of:

identifying pending state events associated with a respective client which are related to a common data object; and

aggregating the identified state events to thereby reduce the number of pending state events.

18. The method of claim 17, where the identified state events are aggregated into at most one state event.

19. The method of claim 1, further comprising the steps of:

monitoring the performance of communication with each connected client; and

dynamically adjusting a rate at which the current state of the specific data is transmitted to each respective client in response to the monitored performance.

20. The method of claim 19, wherein the step of monitoring the performance of communication with each connected client comprises determining network transmission time and a client processing time for received transmissions.

21. A system for processing information provided from at least one content provider about the state of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the system comprising:

an information manager comprising at least one raw data stream as input, an object pool configured to store formatted data objects, and at least one broadcast data stream as output, each raw data stream carrying a plurality of raw data objects;

the information manager configured to;

generate a formatted data object from a received raw data object;

store a current state of the formatted data object in the object storage pool; and

broadcast the current state of the formatted data object on a particular broadcast data stream;

a client manager receiving at least one broadcast data stream as input, comprising an object pool cache, and connectable to a plurality of clients;

the client manager configured to;

establish communication sessions with a plurality of clients;

connect to at least one broadcast data stream;

receive on a connected broadcast data stream a current state for a specific data object;

update an object pool cache to reflect the current state of the specific data object; and

transmit the current state of the specific data object to a set of clients selected from the plurality of clients.

22. The system of claim 21, wherein the information manager is configured to:

determine if a prior version of the formatted data object was present in the object storage pool;

responsive to the determination that a prior version of the formatted data object was present, determine a data differential between the prior version and the current state of the formatted data object and broadcast the data differential on the particular broadcast data stream;

otherwise, broadcast the current state of the formatted data object on the particular broadcast data stream;

23. The system of claim 21, wherein the client manager further comprises a client profile database containing a plurality of client profiles therein, each client profile comprising data indicating data stream subscriptions and at least one object rule associated with the subscribed data streams;

the client manager being further configured to, for each respective client subscribed to the particular input data stream, evaluate from the client profile associated with the respective client the object rules associated with the particular input data stream against the specific data object to identify the set of clients.

24. The system of claim 21, wherein the client manager receives a first broadcast data stream from a first information manager and a second broadcast data stream from a second information manager.

25. The system of claim 21, wherein the information manager further comprises an offer processor configured to determine an object type of the raw data object and apply a set of formatting rules to the received raw data object in accordance with the object type to generate the formatted data object.

26. The system of claim 25, wherein the information manager further comprises a processing database having object typing and formatting rules stored therein.

27. The system of claim 25, wherein the information manager further comprises a translator receiving the raw data stream as input and configured to translate the raw data object into a raw event comprising at least one name-value pair prior and provide the raw event as output;

the offer processor receiving the raw event as input.

28. The system of claim 25, wherein the client manager is configured to select the particular broadcast data stream from a plurality of broadcast data streams according to the determined object type.

29. The system of claim 21, wherein the client manager is further configured to:

validate the contents of the raw data object; and

upon a failed validation, prevent subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

30. The system of claim 21, wherein the raw data object comprises information related to a financial product offering.

31. The system of claim 21, wherein the client manager is further configured to:

in response to a detection that a particular client has subscribed to a new broadcast data stream not in a set of connected broadcast data streams, connecting to the new broadcast data stream.

32. The system of claim 21, wherein the client manager further comprises a delivery manager comprising a client event queue associated with each client;

the delivery manager configured to;

queue state events directed to a particular client in the client event queue associated with the particular client, the state events indicating the current state of specific data objects; and

transmit a client events derived from queued state events to the respective client.

33. The system of claim 32, wherein the delivery manager is further configured to:

identify pending state events associated with a respective client which are related to a common data object; and

aggregate the identified state events to thereby reduce the number of pending state events.

34. The system of claim 21, wherein the client manager is further configured to:

monitor the performance of communication with each connected client; and

dynamically adjust a rate at which the current state of the specific data is transmitted to each respective client in response to the monitored performance.

36. The system of claim 35, wherein the offer pool manager is configured to select the broadcast data stream from a plurality of broadcast channels in accordance with the object type.

37. The system of claim 35, wherein the offer processor is configured to apply specific formatting rules in accordance with the determined object type.

38. The system of claim 35, the processing database further having object validation conditions stored therein;

the offer processor being further configured to validate the contents of the raw data object in accordance with selected validation conditions.

39. The system of claim 35, wherein the object typing and formatting rules are stored in a tree format;

the tree having a root node and at least one descendant stream nodes, each stream node being associated with a specific raw data stream;

each stream node further having at least one type leaf node descending therefrom, each type leaf node being associated with a specific object type carried by the raw data stream associated with the respective stream node;

each type leaf node having at least one associated object typing rule for identifying objects of the type associated with the respective type leaf node.

40. The system of claim 39, wherein each type leaf node further has at least one associated formatting rule.

41. The system of claim 35, wherein each delivery manager comprises:

a client event queue for containing events to be transmitted to the respective client;

a queue manager configured to place events on the queue in accordance with received state events; and

a push module configured to retrieve events from the client event queue and send appropriate client events to the respective client.

42. The system of claim 41, wherein at least one of the client session and the push module are configured to monitor the performance characteristics for communications with the respective client and dynamically determine a rate at which client events should be transmitted in response to the monitored characteristics;

the push module being configured to send client events to the respective client at the dynamically determined rate.

43. The system of claim 42 wherein the performance characteristics comprise network transmission time and a client processing speed time for received client events.

44. The system of claim 31, wherein the queue manager is further configured to:

identify a queued event related to a data object common to a received state event; and

initiate an aggregation of the state event and identified queued event.

45. The system of claim 44, wherein:

each state event received by the queue manager has associated aggregation functionality; and

the queue manager is configured to initiate aggregation by executing the aggregation functionality associated with the received state event.

46. The system of claim 35, further comprising:

a state dispatch module in communication with the information manager and the object caches associated with the object state managers and configured to receive a state data request as input from a requester, retrieve the current states of particular product offerings in accordance with the request, and return the current states to the requestor.

47. The system of claim 46, wherein the object state manager is further configured to, upon first connecting to the respective broadcast data stream, send a request to the state dispatch module to obtain the current states of product offerings carried on the respective broadcast data stream, the obtained currents states being used to initialize the associated object cache.

48. The system of claim 46, wherein the client manager is further configured to send a request to the state dispatch module to obtain the current states of product offerings carried on broadcast data streams to which the new client is subscribed and initiate the return of at least a portion of the obtained current states to the new client.

49. The system of claim 35, wherein the client manager is further configured to:

identify an unconnected broadcast data stream specified in the client profile associated with the new client; and

initiate a connection to the unconnected data stream from a new object state manager.

35. A system for processing raw data streams carrying a plurality of raw data objects containing information related to financial product offerings and subject to periodic updates, and for delivering substantially real-time offer updates to a plurality of clients via a data communication network, the system comprising:

at least one information manager connected to the raw data streams and providing at least one broadcast data stream as output; and

at least one client manager receiving at least one broadcast data stream as input and transmitting the offer updates to the plurality of clients;

the information manager comprising;

(a) a translator configured to receive the raw data objects as input and generate raw events comprising a set of name-value pairs derived from data in the raw data objects;

(b) a processing database having object typing and formatting rules stored therein;

(c) an offer processor connected to the processing database and configured to, in response to the receipt of a raw event;

determine an object type associated with the raw event; and

generate an external event containing a formatted data object derived from the contents of the received raw event in accordance with the formatting rules;

an object storage pool configured to store a plurality of formatted data objects therein;

(d) an offer pool manager connected to the object storage pool and configured to, upon receipt of an external event;

determine if a prior version of the formatted data object is present in an object storage pool;

if a prior version is present in the object storage pool;

determine a data differential between the formatted data object and the prior version,

update the prior version of the formatted data object to correspond with the generated formatted data object, and

broadcast the data differential on a broadcast data stream;

otherwise, if no prior version is present in the object storage pool;

provide the formatted data object to the connected client manager, and

store the formatted data object in the object storage pool;

the client manager comprising;

(a) at least one object state manager, each object state manager connected to a respective broadcast data stream, comprising an associated object cache and subscriber data indicating subscribers to the respective data input stream, and configured to;

receive a data object on the connected input data stream related to a specific product offering,

update the state of the specific product offering in associated object pool cache in accordance with the state changed indicated in the received data object, and

generate an object event directed to subscribers of the respective data input stream indicating the state change for the specific product offering in accordance with the subscriber data;

(b) a plurality of client session modules, each client session module being in communication with a respective client and configured to;

receive object events generated by object state managers connected to an broadcast data stream to which the respective client has subscribed,

upon receiving an object event, evaluate client object rules against the object event, the object rules selected in accordance with the particular broadcast data stream associated with the object event, and

generate a state event from the object event indicating the changed state of the specific data object to the respective client in response to a positive evaluation;

(c) a plurality of delivery managers, each delivery manager being associated with a respective client session module and configured to;

receive state events generated by the associated client session module, and

send client events derived from the received state events to the respective client; and

(d) a client session manager having access to a plurality of client profiles, each client profile containing data stream subscription information and related client object rules, the client manager configured to;

receive an initial communication from a new client;

associate the new client with a new client session module;

retrieve the client profile associated with the new client;

identify a set of data stream subscriptions indicated in the client profile; and

add the new client session module to the subscription data for each object state manager connected to an input data stream to which the new client is subscribed.