System and method for processing data associated with a transmission in a data communication system

• US 7,788,330 B2
• Filed: 08/24/2006
• Issued: 08/31/2010
• Est. Priority Date: 08/24/2006
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Research In Motion Limited (Blackberry Limited)
• Inventors: Goggin, Sean A.
• Primary Examiner(s): Dollinger; Tonia L M
• Assistant Examiner(s): Shin; Kyung H
• Application Number:
  US11/466,821
• Publication Number:
  US 20080052341A1
• Time in Patent Office: 1,468 Days
• Field of Search: 709/201, 709/206, 709/208, 709/224, 370/235
• US Class Current: 709/206
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

A method and system are provided for processing large amounts of data associated with messages and other transmissions that are routed through a data communications system. A distributed system is used to process the data in parallel. The system includes a master processor and at least one additional processor. The master processor is responsible for obtaining the data and routine the data to the additional processors. The additional processors filter the data for information pertaining to the data and the filter results are combined. An on-demand filter can be run in parallel from a web server which allows dynamic filtering of information that is of interest at a particular time.

Claims

1. A method for processing operational data associated with transmissions in a data communication system to obtain information pertaining to said transmissions, said method comprising:

enabling access to a server in said data communication system, wherein all transmissions in said data communication system pass through said server;

obtaining operational data associated with each of a plurality of transmissions as said transmissions pass through said server and forwarding that operational data to a master service in a distributed system comprising a plurality of processor services, wherein said master service has been assigned from one of said plurality of processor services based upon availability, wherein any available one of said plurality of processor services can be assigned as said master service, said master service for distributing said data amongst the others of said plurality of processor services for processing said data;

said master service routing said data to respective ones of said others of said plurality of processor services according to availability of said others of said plurality of processor services;

each of said respective ones of said others of said plurality of processor services processing, in parallel, respective data routed thereto in a predetermined manner until said processing has been completed, to obtain said information pertaining to said transmissions, and to generate one or more result lists containing said information;

merging said one or more result lists with other corresponding result lists generated by the others of the respective ones of said others of said plurality of processor services to generate one or more merged result lists; and

storing said one or more merged result lists for later analysis.

2. A method according to claim 1 wherein said processing comprises running one or more filters to obtain said information, said filters being chosen according to predetermined criteria.

3. A method according to claim 2 wherein said criteria comprise one or more of data type, content pattern, and time range.

4. A method according to claim 1 further comprises requesting an on-demand filter to obtain selected information, adding said on-demand filter to one or more of said others of said plurality of processor services, running said on-demand filter to obtain said selected information, and merging said selected information from all of said plurality of processor services running said on-demand filter for later analysis.

5. A method according to claim 4 further comprising revising said on-demand filter and recursively running said on-demand filter according to a revised set of selected information.

6. A method according to claim 1 further comprising removing said data once said one or more merged result lists have been stored.

7. A method according to claim 1, wherein upon failure of said master service, said method further comprises re-assigning a new master service chosen from said distributed system.

8. A method according to claim 1 wherein if a duplicate master service exists, said master service and said duplicate master service are de-allocated and a new master service is assigned.

9. A method according to claim 1 further comprising determining if a low memory state exists for any one of said others of said plurality of processor services and redistributing said data to balance memory resources.

10. A system for processing operational data associated with transmissions in a data communication system to obtain information pertaining to said transmissions, said system comprising:

a distributed system connectable to a server in said data communication system, wherein all transmissions in said data communication system pass through said server, said distributed system capable of obtaining operational data associated with a plurality of transmissions as said transmissions pass through said server and forwarding that operational data to a master processor in said distributed system, said distributed system comprising a plurality of processors and being configured for assigning one of said plurality of processors as said master processor based upon availability, wherein any available one of said plurality of processors can be assigned as said master processor, said master processor being responsible for routing said data to respective ones of said others of said plurality of processors according to availability of said others of said plurality of additional processors, each of said respective ones of said others of said plurality of processors being configured for processing, in parallel, respective data routed thereto in a predetermined manner until said processing has been completed, to obtain said information pertaining to said transmissions, and for generating one or more result lists containing said information, the distributed system being configured for merging said one or more result lists with other corresponding result lists generated by the others of respective ones of said others of said plurality of processors to generate one or more merged result lists; and

a data storage device configured for storing said one or more merged result lists for later analysis.

11. A system according to claim 10 further comprising a set of filters for obtaining said information, wherein each of said respective ones of said others of said plurality of processors is configured for choosing one or more of said filters according to predetermined criteria.

12. A system according to claim 10 further comprising a web server configured for requesting an on-demand filter to obtain selected information, said system being configured for adding said on-demand filter to one or more of said others of said plurality of processors, said one or more of said others of said plurality of processors being configured for running said on-demand filter to obtain said selected information, the web server also being configured for merging and storing said selected information from all of said plurality of processor services running said on-demand filter for later analysis.

13. A system according to claim 12 wherein said web server is capable of revising said on-demand filter and recursively requesting said on-demand filter according to a revised set of selected information.

14. A system according to claim 10 wherein said system is configured to remove said data from said system once said one or more merged result lists have been stored.

15. A system according to claim 10 wherein said distributed system is capable of reassigning a new master processor chosen from said others of said plurality of processors upon failure of said master processor.

16. A system according to claim 10 wherein said distributed system is capable of detecting a duplicate master processor, and initiating the de-allocation of said master processor and said duplicate master processor and the assignment of a new master processor if said duplicate master processor is found.

17. A system according to claim 10 wherein said master processor is capable of determining if a low memory state exists for any one of said others of said plurality of processors and redistributing said data to balance memory resources.

18. A system according to claim 10 wherein said distributed system is a cluster of server machines connected to each other in a network.

19. A system according to claim 18 wherein said cluster is configured to utilize Jini technology and said master processor and said others of said plurality of processors are configured to operate in a Java environment.