Method and system for processing raw financial data streams to produce and distribute structured and validated product offering objects

• US 8,671,212 B2
• Filed: 04/20/2011
• Issued: 03/11/2014
• Est. Priority Date: 08/04/2000
• Status: Expired due to Fees

• Current Assignee: Goldman Sachs & Co. LLC (Goldman Sachs Group Incorporated)
• Original Assignee: Goldman Sachs & Company (Goldman Sachs Group Incorporated)
• Inventors: Smith, Andrew J. R., Thomas, Deane, Cowan, Randy
• Primary Examiner(s): Survillo, Oleg
• Application Number:
  US13/090,793
• Publication Number:
  US 20110196839A1
• Time in Patent Office: 1,056 Days
• Field of Search: 709/231, 709/219, 709/246, 709/250, 705/36.R, 707/790
• US Class Current: 709/231
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/06 Asset management; Financial planning or analysis
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 67/55 Push-based network services

Abstract

A real time information manager which processes raw data from data providers, such as feeds of current information about financial product offerings, is provided. Raw data objects received from various data providers are analyzed, formatted, and a current version of each unique data object is stored in an object storage pool. Information about the state of each objects is transmitted on a particular output data channel in accordance with the type of object. A subscriber process can access one or more of the data channels and utilize the broadcast data. In one configuration, one or more client managers are provided which subscribe to particular data channels and serve as an intermediary between the real time information manager and sets of clients.

Claims

1. A method for processing raw data streams carrying a plurality of raw data objects containing information which is subject to periodic updates, the method comprising the steps of:

receiving a raw data object carried on a raw data stream input;

processing the received raw data object to determine an object type of the raw data object;

generating a formatted data object based on contents of the raw data object by applying a set of formatting rules;

receiving a request from a client manager for a snapshot of a current state of the formatted data object;

transmitting the snapshot to the client manager on a channel;

determining whether a prior version of the formatted data object is present in an object storage pool;

upon determining that a prior version is present in the object storage pool;

determining a data differential between the formatted data object and the prior version;

updating the prior version of the formatted data object to correspond with the formatted data object; and

broadcasting the data differential on an output broadcast data channel, the data differential including differential pricing data, wherein the output broadcast data channel transmitting the data differential is separate from the channel transmitting the snapshot;

wherein the client manager;

 

receives the data differential;

 

uses a unique profile key to load a profile of a subscriber associated with the unique profile key, the profile including subscription selections; and

 

sends the data differential to the subscriber according to the subscription selections in the profile;

otherwise, upon determining that no prior version is present in the object storage pool;

providing the formatted data object to subscriber processes; and

storing the formatted data object in the object storage pool.

2. The method of claim 1, wherein the formatted data object stored in the object storage pool has an associated sequence number, wherein the step of updating includes incrementing the sequence number of the formatted data object, and wherein broadcasting the data differential includes broadcasting the sequence number associated with the formatted data object.

3. The method of claim 1, wherein the step of providing the formatted data object comprises broadcasting the formatted data object on the output broadcast channel.

4. The method of claim 1, wherein the broadcast channel is selected from a plurality of broadcast channels according to the object type.

5. The method of claim 1, wherein the applied formatting rules are selected in accordance with the determined object type.

6. The method of claim 1, further comprising translating the raw data object into a raw event comprising at least one name-value pair prior to determining an object type of the raw data object and generating a formatted data object.

7. The method of claim 1, further comprising the steps of:

validating the contents of the raw data object; and

upon a failed validation, preventing subsequent broadcast of the data differential or formatted data object derived from the respective raw data object.

8. The method of claim 1, further comprising:

receiving a particular formatted data object at a subscriber process;

storing the received formatted data object in a database associated with the subscriber process;

receiving at the subscriber process a data differential for the particular formatted object via the broadcast data channel; and

updating the stored formatted data object in accordance with the received data differential.

9. The method of claim 8, wherein the broadcast data channel is received by a plurality of subscriber processes.

10. The method of claim 8, wherein each of the subscriber processes receives a plurality of broadcast channels.

11. The method of claim 8, further comprising communicating information about the formatted data object stored in the database associated with the subscriber process to a remote location.

12. The method of claim 1, wherein the raw data object comprises information related to a financial product offering.

13. A system for processing information comprising:

at least one computer processor;

an information manager, using the at least one computer processor, configured to;

receive a raw data object carried on a raw data stream input,

process the received raw data object to determine an object type of the raw data object,

generate a formatted data object based on contents of the raw data object by applying a set of formatting rules,

receive a request from a client manager for a snapshot of a current state of the formatted data object,

transmit the snapshot to the client manager on a channel,

determine whether a prior version of the formatted data object is present in an object storage pool, upon determining that a prior version is present in the object storage pool;

determine a data differential between the formatted data object and the prior version,

update the prior version of the formatted data object to correspond with the formatted data object, and

broadcast the data differential on an output broadcast data channel, the data differential including differential pricing data, wherein the output broadcast data channel transmitting the data differential is separate from the channel transmitting the snapshot,

upon determining that no prior version is present in the object storage pool;

provide the formatted data object to subscriber processes, and

store the formatted data object in the object storage pool; and

the client manager configured to;

receive the data differential,

use a unique profile key to load a profile of a subscriber associated with the unique profile key, the profile including subscription selections, and

send the data differential to the subscriber according to the subscription selections in the profile.

14. The system of claim 13, wherein the formatted data object stored in the object storage pool has an associated sequence number, wherein the information manager is further configured to:

increment the sequence number of the formatted data object, and

broadcast the sequence number associated with the formatted data object.

15. The system of claim 13, wherein the information manager is further configured to broadcast the formatted data object on the output broadcast channel.

16. The system of claim 13, wherein the broadcast channel is selected from a plurality of broadcast channels according to the object type.

17. The system of claim 13, wherein the applied formatting rules are selected in accordance with the determined object type.

18. The system of claim 13, wherein the information manager is further configured to translate the raw data object into a raw event comprising at least one name-value pair prior to determining an object type of the raw data object and generating a formatted data object.

19. The system of claim 13, wherein the information manager is further configured to:

validate the contents of the raw data object, and

upon a failed validation, prevent subsequent broadcast of the data differential or formatted data object derived from the respective raw data object.

20. The system of claim 13, wherein the raw data object comprises information related to a financial product offering.