SYSTEM AND METHOD FOR PROCESSING DATA ASSOCIATED WITH A TRANSMISSION IN A DATA COMMUNICATION SYSTEM

• US 20080052341A1
• Filed: 08/24/2006
• Published: 02/28/2008
• Est. Priority Date: 08/24/2006
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: Goggin, Sean A.
• Granted Patent: US 7,788,330 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

A method and system are provided for processing large amounts of data associated with messages and other transmissions that are routed through a data communications system. A distributed system is used to process the data in parallel. The system includes a master processor and at least one additional processor. The master processor is responsible for obtaining the data and routine the data to the additional processors. The additional processors filter the data for information pertaining to the data and the filter results are combined. An on-demand filter can be run in parallel from a web server which allows dynamic filtering of information that is of interest at a particular time.

Claims

1. A method for processing operational data associated with a transmission in a data communication system to obtain information pertaining to said transmission, said method comprising

assigning a master service for distributing said data to a respective one of a plurality of processor services in a distributed system for processing said data;

said master service obtaining said data from said transmission and routing said data to said one of said plurality of processor services according to availability of said plurality of processor services;

said one of said plurality of processor services processing said data in a predetermined manner to obtain said information and venerate one or more result list containing said information;

merging said one or more result list with other corresponding result lists generated by others of said plurality of processor services to generate one or more merged result list; and

storing said one or more merged result list for later analysis.

2. A method according to claim 1 wherein said processing comprises running one or more filters to obtain said information, said filters being chosen according to predetermined criteria.

3. A method according to claim 2 wherein said criteria comprise any one or any combination of data type, content pattern and/or time range.

4. A method according to claim 1 further comprises requesting an on-demand filter to obtain selected information, adding said on-demand filter to one or more of said plurality of processor services, running said on-demand filter to obtain said selected information, and merging said selected information from all of said plurality of processor services running said on-demand filter for later analysis.

5. A method according to claim 4 further comprising revising said on-demand filter and recursively running said on-demand filter according to a revised set of selected information.

6. A method according to claim 1 further comprising removing said data once said one or more merged result list has been stored.

7. A method according to claim 1, wherein upon failure of said master service, said method further comprises re-assigning, a new master service chosen from said distributed system.

8. A method according to claim 1 wherein if a duplicate master service exists, said master service and said duplicate master service are de-allocated and a new master service is assigned.

9. A method according to claim 1 further comprising determining if a low memory state exists for any one of said plurality of processor services and redistributing said data to balance memory resources.

10. A system for processing operational data associated with a transmission in a data communication system to obtain information pertaining to said transmission, said system comprising:

a distributed system of processors connected to said data communication system and capable of obtaining said data from said transmission, said distributed system comprising an assigned master processor and a plurality of additional processors, wherein said master processor is responsible for routing said data to a respective one of said plurality of additional processors according to availability of said plurality of additional processors; and

a data storage device;

wherein said one of said plurality of additional processors is configured to process said data in a predetermined manner to obtain said information and to generate one or more result list containing said information, and the system is configured to merge said one or more result list with other corresponding result lists generated by others of said plurality of additional processors to generate one or more merged result list, and to store said one or more merged result list on said data storage device for later analysis.

11. A system according to claim 10 further comprising a set of filters for obtaining said information, wherein said one of said plurality of additional processors is configured to choose one or more of said filters according to predetermined criteria.

12. A system according to claim 10 further comprising a web server for requesting an on-demand filter to obtain selected information, one or more of said plurality of additional processors being configured to run said on-demand filter to obtain said selected information, the web server being configured to merge and store said selected information from all of said plurality of processor services running said on-demand filter for later analysis.

13. A system according to claim 12 wherein said web server is configured to revise said on-demand filter and to recursively request said on-demand filter according to a revised set of selected information.

14. A system according to claim 10 wherein said system is configured to remove said data once said one or more merged result list has been stored.

15. A system according to claim 10 wherein said distributed system is configured to re-assign a new master processor chosen from said plurality of additional processors upon failure of said master processor.

16. A system according to claim 10 wherein said distributed system is configured to detect a duplicate master processor, wherein if said duplicate master processor is found, said master processor and said duplicate master processor are de-allocated and a new master processor is assigned.

17. A system according to claim 10 wherein said master processor is configured to determine if a low memory state exists for any one of said additional processors and to redistribute said data to balance memory resources.

18. A system according to claim 10 wherein said distributed system is a cluster of server machines connected to each other in a network.

19. A system according to claim 18 wherein said cluster utilizes Jini technology and said master processor and said additional processors are configured to operate in a Java environment.