Method and system for processing raw financial data streams to produce and distribute structured and validated product offering data to subscribing clients

• US 7,958,251 B2
• Filed: 05/31/2001
• Issued: 06/07/2011
• Est. Priority Date: 08/04/2000
• Status: Active Grant

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Smith, Andrew J. R., Thomas, Deane, Cowan, Randy
• Primary Examiner(s): Bates, Kevin
• Assistant Examiner(s): Survillo, Oleg
• Application Number:
  US09/871,176
• Publication Number:
  US 20020016839A1
• Time in Patent Office: 3,659 Days
• Field of Search: 709/231, 709/246, 709/250, 705/35, 707/6
• US Class Current: 709/231
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/06 Asset management; Financial planning or analysis

Abstract

A method and system is disclosed for processing information provided from various content providers in the form objects having states subject to periodic updates. Formatted information indicating a current state of the objects are delivered to subscribing clients in substantially real-time. An information manager module is configured to receiving raw data objects from the content providers, format the information, and broadcast current state information on various broadcast data stream. A client manager module supports multiple client communication sessions and connects to at least one broadcast data stream. Current state information received on a broadcast data stream is processed and transmitted to specific clients in accordance with processing rules specified in the client profiles.

Claims

1. A processor-implemented method for processing information provided from at least one content provider and for delivering formatted information to a plurality of clients via a data communication network in substantially real-time, the method comprising:

in each of at least a first and a second information manager;

receiving raw data objects on at least one raw data stream input;

generating via a processor a formatted data object from a received raw data object;

storing a current state of the formatted data object in an object storage pool; and

broadcasting the current state of the formatted data object on a particular broadcast data stream;

in a client manager;

establishing communication sessions with a plurality of clients;

connecting to at least two broadcast data streams,

receiving on each of the connected particular broadcast data stream the current state of the formatted data object;

updating an object pool cache to reflect the current state of each of the formatted data objects; and

transmitting the current state of at least one of the formatted data objects to a set of clients from the plurality of clients;

placing at least one state event in a client event queue associated with the respective client, the at least one state event containing the current state of the corresponding formatted data objects and embedded functions that encompass a basic set of aggregation and combination rules for state events;

deriving a client event from the at least one state event prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmitting the client event derived from the at least one state event in the client event queue to the respective client.

2. The method of claim 1, wherein the connecting to at least two broadcast data streams further comprises:

connecting to the particular broadcast data stream from the first information manager of the plurality of information managers; and

connecting to the particular broadcast data stream from the second information manager of the plurality of information managers.

3. The method of claim 1, further comprising:

validating the contents of the raw data object; and

upon a failed validation, preventing subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

5. The method of claim 1, wherein the broadcasting the current state of the formatted data object comprises:

determining if a prior version of the formatted data object was present in the object storage pool;

if a prior version of the formatted data object was present, determining a data differential between the prior version and the current state of the formatted data object and broadcasting the data differential on the particular broadcast data stream;

otherwise, broadcasting the current state of the formatted data object on the particular broadcast data stream.

6. The method of claim 1, wherein each client has an associated profile comprising data indicating data stream subscriptions and at least one object rule associated with the subscribed data streams;

wherein, the transmitting the current state of the at least one of the formatted data objects further comprises;

for each respective client subscribed to particular broadcast data stream, evaluating from the client profile associated with the respective client the object rules associated with the particular broadcast data stream against the formatted data object; and

transmitting the current state of the formatted data object to the respective client in response to a positive evaluation.

12. The method of claim 1, wherein the raw data object comprises information related to a financial product offering.

4. A processor-implemented method for processing information provided from at least one content provider about states of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the method comprising:

in each of at least a first and a second information manager;

receiving raw data objects on at least one raw data stream input;

generating via a processor a formatted data object from a received raw data object;

storing a current state of the formatted data object in an object storage pool; and

broadcasting the current state of the formatted data object on a particular broadcast data stream;

in a client manager;

establishing communication sessions with a plurality of clients;

connecting to at least two broadcast data streams, wherein the connecting to at least two broadcast data streams comprises;

connecting to the particular broadcast data stream from the first information manager of the plurality of information managers; and

connecting to the particular broadcast data stream from the second information manager of the plurality of information managers;

receiving on each of the connected particular broadcast data stream the current state of the formatted data object;

updating an object pool cache to reflect the current state of each of the formatted data objects; and

transmitting the current state of at least one of the formatted data objects to a set of clients from the plurality of clients;

wherein each connected client has a respective client event queue, the transmitting the current state of at least one of the formatted data objects to the set of clients comprises, for each respective client in the set of clients and for each of the at least one of the formatted data objects;

placing at least one state event in the client event queue associated with the respective client, the at least one state event containing the current state of the corresponding formatted data object and embedded functions that encompass a basic set of aggregation and combination rules for state events;

deriving a client event from the at least one state event prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmitting the client event derived from the at least one state event in the client event queue to the respective client.

7. The method of claim 4, wherein the broadcasting the current state comprises broadcasting a corresponding sequence number associated with the current state.

8. The method of claim 4, further comprising determining an object type of the raw data object;

wherein, the generating the formatted data object further comprises applying a set of formatting rules to the received raw data object in accordance with the object type.

9. The method of claim 8, further comprising translating the raw data object into a raw event comprising at least one name-value pair prior to determining an object type of the raw data object and generating a formatted data object.

10. The method of claim 4, further comprising determining an object type of the raw data object;

the particular broadcast data stream being selected from a plurality of broadcast data streams according to the object type.

11. The method of claim 4, further comprising:

validating the contents of the raw data object; and

upon a failed validation, preventing subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

13. The method of claim 4, further comprising, in the client manager:

after connecting to a particular broadcast data stream, initializing the object pool cache with an initial state of data objects carried on the particular broadcast data stream.

14. The method of claim 4, further comprising obtaining an initial state of data objects from the information manager generating the particular broadcast data stream.

15. The method of claim 13, further comprising, after establishing a communication session with a particular client, delivering to the particular client a snapshot of a set of data objects in the object pool cache which are carried on broadcast data streams to which the particular client is subscribed.

16. The method of claim 4, further comprising, in the client manager:

in response to a detection that a particular client has subscribed to a new broadcast data stream not in set of connected broadcast data streams, connecting to the new broadcast data stream.

17. The method of claim 16, further comprising, in the client manager:

initializing the object pool cache with an initial state of data objects carried on the new broadcast data stream; and

delivering to the particular client a snapshot of a set of the data objects in the object pool cache associated with the new data stream.

18. The method of claim 4, further:

identifying pending state events associated with a respective client which are related to a common data object; and

aggregating the identified state events to thereby reduce the number of pending state events.

19. The method of claim 18, where the identified state events are aggregated into a single state event.

20. The method of claim 4, further comprising:

monitoring the performance of communication with each connected client; and

dynamically adjusting a rate at which the current state of the formatted data object is transmitted to each respective client in response to the monitored performance.

21. The method of claim 20, wherein the monitoring the performance of communication with each connected client comprises determining network transmission time and a client processing time for received transmissions.

22. A system including at least one processor for processing information provided from at least one content provider about states of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the system comprising:

at least a first and a second information manager, each comprising at least one raw data stream as input, an object storage pool configured to store formatted data objects, and at least one broadcast data stream as output, each raw data stream carrying a plurality of raw data objects;

each of the at least the first and the second information manager configured to;

generate a formatted data object from a received raw data object;

store a current state of the formatted data object in the object storage pool; and

broadcast the current state of the formatted data object on a particular broadcast data stream;

a client manager receiving at least one broadcast data stream as input, comprising an object pool cache, and connectable to a plurality of clients;

the client manager configured to;

establish communication sessions with a plurality of clients;

connect to at least two broadcast data streams, wherein the client manager receives the particular broadcast data stream from the first information manager of the plurality of information managers and the particular broadcast data stream from the second information manager of the plurality of information managers;

receive on each of the connected particular broadcast data stream the current state of the formatted data object;

update the object pool cache to reflect the current state of each of the formatted data objects; and

transmit the current state of at least one of the formatted data objects to a set of clients from the plurality of clients;

wherein the client manager further comprises a delivery manager comprising a client event queue associated with each client;

the delivery manager configured to;

queue state events directed to a particular client in the client event queue associated with the particular client, the state events containing the current state of formatted data objects and embedded functions that encompass a basic set of aggregation and combination rules for state events;

derive a client event from at least one of the queued state events prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmit the client event derived from the at least one queued state event to the respective client.

23. The system of claim 22, wherein each of the at least the first and the second information manager is configured to:

determine if a prior version of the formatted data object was present in the object storage pool;

responsive to the determination that a prior version of the formatted data object was present, determine a data differential between the prior version and the current state of the formatted data object and broadcast the data differential on the particular broadcast data stream;

otherwise, broadcast the current state of the formatted data object on the particular broadcast data stream.

24. The system of claim 22, wherein the client manager further comprises a client profile database containing a plurality of client profiles therein, each client profile comprising data indicating data stream subscriptions and at least one object rule associated with the subscribed data streams;

the client manager being further configured to, for each respective client subscribed to a particular broadcast data stream, evaluate from the client profile associated with the respective client the object rules associated with the particular broadcast data stream against the formatted data object to identify the set of clients.

25. The system of claim 22, wherein each of the at least the first and the second information manager further comprises an offer processor configured to determine an object type of the raw data object and apply a set of formatting rules to the received raw data object in accordance with the object type to generate the formatted data object.

26. The system of claim 25, wherein each of the at least the first and the second information manager comprises a processing database having object typing and formatting rules stored therein.

27. The system of claim 25, wherein each of the at least the first and the second information manager further comprises a translator receiving the raw data stream as input and configured to translate the raw data object into a raw event comprising at least one name-value pair and provide the raw event as output;

the offer processor receiving the raw event as input.

28. The system of claim 25, wherein the client manager is configured to select the particular broadcast data stream from a plurality of broadcast data streams according to the determined object type.

29. The system of claim 22, wherein the client manager is further configured to:

validate the contents of the raw data object; and

upon a failed validation, prevent subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

30. The system of claim 22, wherein the raw data object comprises information related to a financial product offering.

31. The system of claim 22, wherein the client manager is further configured to:

in response to a detection that a particular client has subscribed to a new broadcast data stream not in set of connected broadcast data streams, connecting to the new broadcast data stream.

32. The system of claim 22, wherein the delivery manager is further configured to:

identify pending state events associated with a respective client which are related to a common data object; and

aggregate the identified state events to thereby reduce the number of pending state events.

33. The system of claim 22, wherein the client manager is further configured to:

monitor the performance of communication with each connected client; and

dynamically adjust a rate at which the current state of the formatted data object is transmitted to each respective client in response to the monitored performance.

34. A processor-implemented method for processing information provided from at least one content provider about states of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the method comprising:

in each of at least a first and a second information manager;

receiving raw data objects on at least one raw data stream input;

translating a received raw data object into a raw event;

determining via a processor an object type of the raw data objects;

applying a set of formatting rules to the received raw data object in accordance with the object type to generate a formatted data object from the received raw data object;

storing a current state of the formatted data object in an object storage pool; and

broadcasting the current state of the formatted data object on a particular broadcast data stream;

in a client manager;

establishing communication sessions with a plurality of clients;

connecting to at least two broadcast data streams, wherein the connecting to at least two broadcast data streams comprises;

connecting to the particular broadcast data stream from the first information manager of the plurality of information managers; and

connecting to the particular broadcast data stream from the second information manager of the plurality of information managers;

receiving on each of the connected particular broadcast data stream the current state of the formatted data object;

updating an object pool cache to reflect the current state of each of the formatted data objects; and

transmitting the current state of at least one of the formatted data objects to a set of clients from the plurality of clients;

wherein each connected client has a respective client event queue, transmitting the current state of at least one of the formatted data objects to the set of clients comprises for each respective client in the set of clients and for each of the at least one of the formatted data objects;

placing at least one state event in the client event queue associated with the respective client, the at least one state event containing the current state of the corresponding formatted data object and embedded functions that encompass a basic set of aggregation and combination rules for state events;

deriving a client event from the at least one state event prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmitting the client event derived from the at least one state event in the client event queue to the respective client.

35. The method of claim 34, wherein broadcasting the current state of the formatted data object comprises:

determining if a prior version of the formatted data object was present in the object storage pool;

if a prior version of the formatted data object was present, determining a data differential between the prior version and the current state of the formatted data object and broadcasting the data differential on the particular broadcast data stream;

otherwise, broadcasting the current state of the formatted data object on the particular broadcast data stream.

36. The method of claim 34, wherein each client has an associated profile comprising data indicating data stream subscriptions and at least one object rule associated with the subscribed data streams;

transmitting the current state of the at least one of the formatted data object objects comprising;

for each respective client subscribed to particular broadcast data stream, evaluating from the client profile associated with the respective client the object rules associated with the particular broadcast data stream against the formatted data object; and

transmitting the current state of the formatted data object to the respective client in response to a positive evaluation.

37. The method of claim 34, wherein broadcasting the current state comprises broadcasting a corresponding sequence number associated with the current state.

38. The method of claim 34, further comprising:

validating the contents of the raw data object; and

upon a failed validation, preventing subsequent broadcast of the current state of the formatted data object data derived from the raw data object.

39. A system for processing information provided from at least one content provider about states of a plurality of objects, the states being subject to periodic updates, and for delivering formatted information indicating a current state of at least a portion of the plurality of objects to a plurality of clients via a data communication network in substantially real-time, the system comprising:

a memory;

a processor disposed in communication with the memory, and configured to issue a plurality of processing instructions stored in the memory, wherein the processor issues instructions to;

each of at least a first and a second information manager to;

receive raw data objects on at least one raw data stream input;

generate a formatted data object from a received raw data object;

store a current state of the formatted data object in an object storage pool; and

broadcast the current state of the formatted data object on a particular broadcast data stream;

a client manager to;

establish communication sessions with a plurality of clients;

connect to at least two broadcast data streams, wherein the at least two broadcast data streams are further configured to;

connect to the particular broadcast data stream from the first information manager of the plurality of information managers; and

connect to the particular broadcast data stream from the second information manager of the plurality of information managers;

receive on each of the connected particular broadcast data stream the current state of the formatted data object;

update an object pool cache to reflect the current state of each of the formatted data objects; and

transmit the current state of at least one of the formatted data objects to a set of clients from the plurality of clients;

wherein each connected client has a respective client event queue, the transmission of the current state of at least one of the formatted data objects to the set of clients further comprise, for each respective client in the set of clients and for each of the at least one of the formatted data objects instructions to;

place at least one state event in the client event queue associated with the respective client, the at least one state event containing the current state of the corresponding formatted data object and embedded functions that encompass a basic set of aggregation and combination rules for state events;

derive a client event from the at least one state event prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmit the client event derived from the at least one state event in the client event queue to the respective client.

40. The system of claim 39, wherein the client manager is further configured to, after establishing a communication session with a particular client, deliver to the particular client a snapshot of a set of data objects in the object pool cache which are carried on broadcast data streams to which the particular client is subscribed.

41. The system of claim 40, wherein the client manager is further configured to, after connecting to a particular broadcast data stream, initialize the object pool cache with an initial state of data objects carried on the particular broadcast data stream.

42. A processor-implemented method, comprising:

in each of at least a first and a second information manager;

receiving raw data objects on at least one raw data stream input;

determining via a processor an object type of a received raw data object;

generating via the processor a formatted data object from the received raw data object, wherein the generating the formatted data object further comprises applying a set of formatting rules to the received raw data object in accordance with the object type;

storing a current state of the formatted data object in an object storage pool; and

broadcasting the current state of the formatted data object on a particular broadcast data stream;

in a client manager;

establishing communication sessions with a plurality of clients;

connecting to at least two broadcast data streams, wherein the connecting to at least two broadcast data streams comprises;

connecting to the particular broadcast data stream from the first information manager of the plurality of information managers; and

connecting to the particular broadcast data stream from the second information manager of the plurality of information managers;

receiving on each of the connected particular broadcast data stream the current state of the formatted data object;

updating an object pool cache to reflect the current state of each of the formatted data objects; and

transmitting the current state of at least one of the formatted data objects to a set of clients from the plurality of clients, wherein the transmitting includes;

monitoring the performance of communication with each connected client; and

dynamically adjusting a rate at which the current state of the at least one of the formatted data objects is transmitted to each respective client in response to the monitored performance;

wherein each connected client has a respective client event queue, the transmitting the current state of the at least one of the formatted data objects to the set of clients comprises, for each respective client in the set of clients and for each of the at least one of the formatted data objects;

placing at least one state event in the client event queue associated with the respective client, the at least one state event containing the current state of the corresponding formatted data object and embedded functions that encompass a basic set of aggregation and combination rules for state events;

deriving a client event from the at least one state event prior to transmission of the client event to the respective client, wherein the deriving of the client event occurs upon placement of the at least one state event in the client event queue or upon removal of the at least one state event from the client event queue, and further wherein said embedded functions that encompass a basic set of aggregation and combination rules for state events are removed from the client event; and

subsequently transmitting the client event derived from the at least one state event in the client event queue to the respective client.