Handling and processing of massive numbers of processing instructions in real time

• US 8,612,299 B2
• Filed: 02/27/2009
• Issued: 12/17/2013
• Est. Priority Date: 02/29/2008
• Status: Active Grant

• Current Assignee: Euroclear SA/NV (Euroclear Ltd.)
• Original Assignee: Euroclear SA/NV (Euroclear Ltd.)
• Inventors: Petit, Henri, Collin, Jean-Francois, Marechal, Nicolas, Deloge, Christine
• Primary Examiner(s): ZURITA, JAMES H
• Application Number:
  US12/919,838
• Publication Number:
  US 20110004788A1
• Time in Patent Office: 1,754 Days
• Field of Search: 705/26.1, 705/27.1, 705/37, 705/35
• US Class Current: 705/26.1
• CPC Class Codes: G06F 16/2315 Optimistic concurrency control
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange

Abstract

A system is designed for processing instructions in real time during a session. This system comprises: a preloader for obtaining reference data relating to the instructions, the reference data indicating the current values of each specified resource account data file, and the preloader being arranged to read the reference data for a plurality of received instructions in parallel from a master database; an enriched instruction queue for queuing the instructions together with their respective preloaded reference data; an execution engine for determining sequentially whether each received instruction can be executed under the present values of the relevant resource account files and for each executable instruction to generate an updating command; and an updater, responsive to the updating command from the execution engine (for updating the master database with the results of each executable instruction, the operation of the plurality of updaters being decoupled from the operation of the execution engine.

Claims

1. A system for processing and handling a plurality of processing instructions in real time during a processing session, each processing instruction specifying resource account data files relating to two different entities and the quantity and type of resources to be exchanged between those files, the system comprising:

a non-transitory, computer-readable medium storing executable instructions; and

one or more processors in communication with the non-transitory, computer-readable medium that, having executed the executable instructions, are programmed to;

instantiate a plurality of preloaders, each preloader being arranged to obtain reference data relating to the instructions, the reference data indicating the current values of each of the specified resource account data files, and the plurality of preloaders being arranged to operate in parallel to read the reference data from a master database for a plurality of respective received instructions;

establish an enriched instruction queue for queuing a plurality of the processing instructions together with their respective preloaded reference data;

instantiate an execution engine arranged to determine sequentially using the received reference data, whether each received instruction can be executed under the present values of the relevant resource account files and for each executable instruction to generate an updating command; and

instantiate a plurality of updaters, responsive to the updating command from the execution engine for updating the master database with the results of each executable instruction, the operation of the plurality of updaters being decoupled from the operation of the execution engine.

2. The system according to claim 1, wherein the one or more processors are further programmed to:

establish an initial input queue arranged to receive real-time instructions and batch instructions from a plurality of different sources and to queue the instructions for input to the preloaders.

3. The system according to claim 2, wherein the initial input queue comprises a plurality of initial input queues each one of the plurality being arranged to provide a dedicated input to a corresponding one of the plurality of preloaders.

4. The system according to claim 2, wherein the initial input queue is arranged to assign a priority to each incoming instruction.

5. The system according to claim 1, wherein the one or more processors are further programmed to:

implement a current condition table stored at the execution engine and arranged to be updated with the results of the executed instructions on the resource account files such that representation of an updated real-time value of each of the updated resource account files is obtained, wherein the execution engine is arranged, for the subsequent sequential processing of instructions, to use the information in the current condition table in preference to reference information from the preloader for a specific resource account data file.

6. The system according to claim 1, wherein the one or more processors are further programmed to:

implement a master data collector for each preloader, each master data collector being arranged to read from the master database the values of the resource account files identified in the instruction and aggregated resource values, and to compile and output an enriched processing instruction including the data read from the master database.

7. The system according to claim 1, wherein the execution engine comprises a high-speed local data memory for storing the current condition table.

8. The system according to claim 1, wherein the execution engine is arranged to determine whether the current instruction can be executed under the present values of the relevant resource account files by referring to a stored set of predetermined rules.

9. The system according to claim 1, wherein the one or more processors are further programmed to:

instantiate a time-stamping module arranged to determine the time of receipt of one of the processing instructions and to create a timestamp and apply the same to the processing instruction.

10. The system according to claim 1, wherein the one or more processors are further programmed to:

implement a recycling module for storing a list of failed processing instructions in a high-speed local data store and for carrying out a recycling procedure where the failed processing instructions are presented for re-execution after an update, in the resource accounts files identified by that failed processing instruction, has occurred.

11. The system according to claim 10, wherein the recycling module is arranged to store the failed instructions in order of their priority and to present the highest priority instructions first for re-execution during the recycling procedure.

12. The system according to claim 10, wherein the recycling module is arranged to reserve the current value of a resource account file for an instruction in the list and to use this reserved amount in fulfilling the requirements of the instruction during a recycling procedure.

13. The system according to claim 10, wherein the recycling module is arranged to present failed processing instructions for re-execution a predetermined maximum number of times, and if the processing instruction has still not been executed, the recycling module is arranged to cancel the failed instruction.

14. The system according to claim 1, wherein the one or more processors are further programmed to:

instantiate a reporting module for reporting the outcome of the execution of one of the received instructions, the reporting module being arranged to output the updating command for each instruction which it has been determined can be executed and a notification instruction for each failed instruction.

15. The system according to claim 14, wherein the reporting module is arranged to store temporarily a plurality of updating commands until a commit event occurs, and to output the plurality of stored updating commands on occurrence of the commit event, wherein the commit event represents the completion of one physical unit of work by the execution engine.

16. The system according to claim 14, wherein the one or more processors are further programmed to:

establish a finality queue arranged to receive and queue the updating command and the notification instruction from the reporting module.

17. The system according to claim 1, wherein each updater comprises a plurality of updating instances operating in parallel, each instance operating on a unique portion of the master database.

18. The system according to claim 17, wherein the one or more processors are further programmed to:

implement a segregation input queue for each updating instance, each segregation input queue being arranged to receive a plurality of updating commands pertaining to the specific portion of the master database which is updated by the corresponding updating instance.

19. The system according to claim 18, wherein each segregation input queue is arranged to effect netting of updating commands where the updating commands relate to the same reference account files.

20. The system according to claim 18, wherein the one or more processors are further programmed to:

instantiate a dispatcher module arranged to determine the routing of each updating command to a specific segregation input queue of the plurality of updating instances and to create reporting messages for the different entities in which the failure or success of an execution instruction is described.

21. The system according to claim 18, wherein:

the initial input queue comprises a plurality of initial input queues each one of the plurality of initial input queues being arranged to provide a dedicated input to a corresponding one of the plurality of preloaders; and

the one or more processors are further programmed to;

implement a routing framework, the routing framework being arranged to distribute an instruction to the plurality of initial input queues or segregation input queues operating concurrently.

22. The system according to claim 21, wherein the routing framework is arranged to assign to an instruction an instruction name describing the type of information the instruction relates to and a workload balancer key which provides the unique key identifying the instruction.

23. The system according to claim 22, wherein the one or more processors are further programmed to:

instantiate a workload balancer arranged in combination with the routing framework to select one of the initial input queues or segregation input queues by use of the received instruction name and workload balancer key.

24. The system according to claim 23, wherein the workload balancer is arranged to link a given workload balancer key to the same specific destination to ensure that all instructions relating to a specific resource account data file are always routed to the same one of the plurality of queues.

25. The system according to claim 1, wherein the one or more processors are further programmed to:

implement a delimiter framework arranged, in response to the closing of the processing session at a given time, to send a message to the input of a queue provided at the output of the execution engine and to await its response, and to send a further message to an input queue of one of the plurality of updaters and await its response, receipt of the sequential responses indicating updating of the master database with the results of all of the processing instructions executed before the close of the processing session.

26. The system according to claim 25, wherein the one or more processors are further programmed to:

instantiate a dispatcher module arranged to determine the routing of each updating command to a specific segregation input queue and to create reporting messages for the different entities in which the failure or success of an execution instruction is described; and

wherein

the delimiter framework is also arranged to send a message to an input queue of the dispatcher module and to await respective feedback to confirm that reporting messages for the processing instructions executed before the close of the processing session, have all been sent.

27. The system according to claim 1, wherein the one or more processors are further programmed to:

instantiate a pacing module for determining the processing instruction throughput of the execution engine and applying wait states to any queue to slow down the speed of loading of processing instructions to less than the throughput of the execution engine.

28. A resource transfer system for processing and handling a plurality of resource transfer instructions in real time during a resource transfer processing session, the system comprising:

a non-transitory, computer-readable medium storing executable instructions; and

one or more processors in communication with the non-transitory, computer-readable medium that, having executed the executable instructions, are programmed to;

instantiate a plurality of preloaders, each preloader being arranged to obtain reference data relating to the instructions, the reference data indicating the current values of each of the specified resource account data files, and the plurality of preloaders being arranged to operate in parallel to read the reference data from a master database for a plurality of respective received instructions;

each processing instruction specifying resource account data files relating to two different entities and the quantity and type of resources to be exchanged between those account data files,

establish an enriched instruction queue for queuing a plurality of the processing instructions together with their respective preloaded reference data;

establish an execution engine arranged to determine sequentially using the received reference data, whether each received instruction can be executed under the present values of the relevant resource account data files and for each executable instruction to generate an updating command; and

instantiate a plurality of updaters, responsive to the updating command from the execution engine for updating the master database with the results of each executable instruction, the operation of the plurality of updaters being decoupled from the operation of the execution engine.

29. The resource transfer system according to claim 28, wherein each resource transfer instruction comprises the following information:

a) an identification of the resource to be exchanged;

b) a quantity of resource to be exchanged;

c) a cash currency to be used in the payment; and

d) a cash amount.

30. The resource transfer system according to claim 28, wherein the one or more processors are further programmed to:

establish a securities settlement system and the resource transfer instructions each comprise a security settlement processing instruction.

31. A computer implemented method of processing and handling plurality of processing instructions in real time during a processing session, each processing instruction specifying resource account data files relating to two different entities and the quantity and type of resources to be exchanged between those files, the method comprising:

obtaining, with one or more processors, reference data relating to instructions from a plurality of preloaders stored in a non-transitory computer-readable memory, each preloader being arranged to obtain reference data relating to the instructions, the reference data indicating the current values of each of the specified resource account data files, the obtaining step comprising operating the plurality of preloaders in parallel to read the reference data from a master database for a plurality of respective received instructions;

queuing, with the one or more processors, a plurality of the processing instruction together with their respective preloaded reference data in an enriched instruction queue;

using an execution engine being executed by the one or more processors to determine sequentially using the received reference data, whether each received instruction can be executed under the present values of the relevant resource account files and generating an updating command for each executable instruction; and

updating, with the one or more processors, the master database with the results of each executable instruction using a plurality of updaters which are responsive to the updating command from the execution engine, the updating step being decoupled from the operation of the execution engine.