AGGREGATING DATA IN A MEDIATION SYSTEM

• US 20140040213A1
• Filed: 08/02/2012
• Published: 02/06/2014
• Est. Priority Date: 08/02/2012
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Software Corporation
• Inventors: Rossi, Larry Paul
• Granted Patent: US 9,185,235 B2
• Time in Patent Office: Days
• US Class Current: 707/693
• CPC Class Codes: G06F 11/3082 the data filtering being achieved by aggregating or compressing the monitored data
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: H04M 15/00 Arrangements for metering, time-control or time indication ; Metering, charging or billing arrangements for voice wireline or wireless communications, e.g. VoIP
• CPC Class Codes: H04M 15/41 Billing record details, i.e. parameters, identifiers, structure of call data record [CDR]
• CPC Class Codes: H04M 15/43 Billing software details
• CPC Class Codes: H04M 15/44 Augmented, consolidated or itemized billing statement or bill presentation
• CPC Class Codes: H04M 15/53 using mediation
• CPC Class Codes: H04M 15/62 based on trigger specification

Abstract

Records received from one or more sources in a network are processed. For each of multiple intervals of time, a matching procedure is attempted on sets of one or more records, including comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets. The processing also includes, for at least some of the intervals of time, processing at least one complete set, consisting of one or more of the received records on which the matching procedure is first attempted during the interval of time and one or more records stored in a data store before the interval of time, and for at least some of the intervals of time, processing at least one incomplete set, consisting of one or more records stored in the data store before the interval of time.

Claims

1. A method for processing data received at a node in a network, the method including:

receiving records from one or more sources in the network, at least some of the records each being associated with an identifier;

for each of multiple intervals of time, attempting a matching procedure on sets of one or more records, the matching procedure including comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets;

for at least some of the intervals of time, processing at least one complete set of records, for which the completeness criterion was satisfied during that attempt, consisting of one or more of the received records on which the matching procedure is first attempted during the interval of time and one or more records stored in a data store before the interval of time; and

for at least some of the intervals of time, processing at least one incomplete set of records, for which the completeness criterion was not satisfied during that attempt, consisting of one or more records stored in the data store before the interval of time.

2. The method of claim 1, further including:

after at least some attempts of the matching procedure, storing in the data store at least some incomplete sets of records for which the completeness criterion was not satisfied during that attempt, and storing keys corresponding to the incomplete sets of records in a data structure separate from the data store; and

retrieving stored keys from the data structure to attempt the matching procedure on corresponding records during a subsequent interval of time.

3. The method of claim 2, wherein stored keys are retrieved from the data structure asynchronously with the intervals of time.

4. The method of claim 1, further including, for at least some of the intervals of time, processing at least one complete set of records, and storing in the data store information identifying the processed complete set of records as having been processed.

5. The method of claim 1, further including, for at least some of the intervals of time, processing at least one incomplete set of records, and storing in the data store information identifying the processed incomplete set of records as having been processed.

6. The method of claim 1, wherein an incomplete set of records is processed after at least two attempts of the matching procedure in response to comparison of a time associated with at least one record in the incomplete set of records with an expiration criterion.

7. The method of claim 6, wherein the expiration criterion depends on a degree of incompleteness of the incomplete set of records.

8. The method of claim 2, wherein comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets of records includes:

determining distinct identifiers associated with the received records,

for each distinct identifier, retrieving any records stored in the data store associated with that distinct identifier, and aggregating a set of records associated with that identifier that includes one or more of the received records and any retrieved records, and

determining whether or not the completeness criterion is satisfied for each aggregated set of records.

9. The method of claim 8, wherein retrieving any records stored in the data store associated with that distinct identifier includes:

looking up the distinct identifier in an index associated with the data store.

10. The method of claim 9, wherein retrieving any records stored in the data store associated with that distinct identifier includes:

decompressing a portion of the data store that includes any records associated with the distinct identifier.

11. The method of claim 10, wherein retrieving any records stored in the data store associated with that distinct identifier includes:

scanning the decompressed portion of the data store to locate any records associated with the distinct identifier.

12. The method of claim 8, wherein retrieving any records stored in the data store associated with that distinct identifier includes:

determining whether or not a stored set of records associated with that distinct identifier has been identified as having been processed.

13. The method of claim 8, wherein retrieving any records stored in the data store associated with that distinct identifier includes:

determining a hash value based on the distinct identifier.

14. The method of claim 2, wherein receiving records from one or more sources in the network includes storing the received records in an input buffer.

15. The method of claim 14, wherein retrieving stored keys from the data structure to attempt the matching procedure on corresponding records during a subsequent interval of time includes:

removing a key from the data structure, and adding the removed key or a record corresponding to the removed key to the input buffer.

16. The method of claim 1, wherein the key associated with a set of records includes the identifier associated with the set of records.

17. The method of claim 1, wherein the intervals of time are determined based on a clock at the node.

18. The method of claim 1, wherein the intervals of time are determined based on a number of records received.

19. The method of claim 1, wherein one or more of the intervals of time are included within a checkpoint interval in which data associated with the matching procedure that has been received or generated since a previous checkpoint interval is persistently stored.

20. The method of claim 1, wherein the data store enables random access of records stored in the data store.

21. The method of claim 1, wherein processing a set of records includes sending information in the records in the set to another node in the network.

22. A computer-readable storage medium storing a computer program for processing data received at a node in a network, the computer program including instructions for causing a computing system to:

receive records from one or more sources in the network, at least some of the records each being associated with an identifier;

for each of multiple intervals of time, attempt a matching procedure on sets of one or more records, the matching procedure including comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets;

for at least some of the intervals of time, process at least one complete set of records, for which the completeness criterion was satisfied during that attempt, consisting of one or more of the received records on which the matching procedure is first attempted during the interval of time and one or more records stored in a data store before the interval of time; and

for at least some of the intervals of time, process at least one incomplete set of records, for which the completeness criterion was not satisfied during that attempt, consisting of one or more records stored in the data store before the interval of time.

23. A network node, including:

a network interface configured to receive records from one or more sources in the network, at least some of the records each being associated with an identifier; and

at least one processor configured to process sets of records, the processing including;

for each of multiple intervals of time, attempting a matching procedure on sets of one or more records, the matching procedure including comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets;

for at least some of the intervals of time, processing at least one complete set of records, for which the completeness criterion was satisfied during that attempt, consisting of one or more of the received records on which the matching procedure is first attempted during the interval of time and one or more records stored in a data store before the interval of time; and

for at least some of the intervals of time, processing at least one incomplete set of records, for which the completeness criterion was not satisfied during that attempt, consisting of one or more records stored in the data store before the interval of time.

24. A network node, including:

means for receiving records from one or more sources in the network, at least some of the records each being associated with an identifier; and

means for processing sets of records, the processing including;

for each of multiple intervals of time, attempting a matching procedure on sets of one or more records, the matching procedure including comparing identifiers associated with different records to generate the sets and determining whether or not a completeness criterion is satisfied for one or more of the sets;

for at least some of the intervals of time, processing at least one complete set of records, for which the completeness criterion was satisfied during that attempt, consisting of one or more of the received records on which the matching procedure is first attempted during the interval of time and one or more records stored in a data store before the interval of time; and

for at least some of the intervals of time, processing at least one incomplete set of records, for which the completeness criterion was not satisfied during that attempt, consisting of one or more records stored in the data store before the interval of time.