Securely and efficiently extending data processing pipeline functionality

• US 8,037,123 B2
• Filed: 08/27/2004
• Issued: 10/11/2011
• Est. Priority Date: 08/27/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Sharma, Vivek, Wei, Min
• Primary Examiner(s): Nawaz; Asad
• Assistant Examiner(s): DRABIK, SARAH E
• Application Number:
  US10/928,578
• Publication Number:
  US 20060069713A1
• Time in Patent Office: 2,601 Days
• Field of Search: 709201-203, 709217-219, 709223-224, 707 3- 4, 707 9- 10, 719/318, 719/321, 719328-329
• US Class Current: 709/202
• CPC Class Codes: G06F 9/4862 the task being a mobile agent, i.e. specifically designed to migrate

Abstract

The present invention extends to methods, systems, and computer program products for securely and efficiently extending data processing pipeline functionality. Data processing agents register, potentially based solely on configuration data contained in the data processing agents, to manipulate data of a data processing pipeline. Configuration data can identify a specified event (e.g., an SMTP verb) the data processing agent is to hook into. Upon detection of a specified event, one or more agents that have registered for the specified event are loaded into secure zones. Process boundaries separate agents from one another and from other processes such that execution of each agent is isolated from other processes. Pipeline related data (e.g., electronic message data) is transferred to each loaded agent. Application Program Interfaces (“APIs”) can be used to transfer data between modules that implement embodiments of the present invention.

Claims

1. At a computer system, the computer system including a processor and system memory, the computer system also including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequentially processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, a method for extending the functionality of the data processing pipeline, the method comprising:

an act of an agent engine that is external to the data pipeline detecting an occurrence of an event within the data pipeline;

an act of mapping the detected event to a specified topic string representing an electronic messaging command;

an act of matching the specified topic string to the topic string for a plurality of data processing agents previously stored at a specified known location at the computer system so as to determine that the plurality of data processing agents have registered for the detected event, the plurality of data processing agents being external to the data pipeline;

an act of the processor loading each of the plurality of data processing agents into a secure zone such that execution of each data processing agent in the plurality of data processing agents is isolated from execution of all the other data processing agents in the plurality of data processing agents;

for each of the plurality of data processing agents;

an act of the agent engine external to the data pipeline diverting pipeline data related to the represented electronic messaging command out of the data pipeline from a first pipeline module and to the data processing agent to extend the functionality of the data pipeline subsequent to loading the data processing agent, wherein the pipeline data is diverted by the external agent engine when the pipeline data is directed from the first pipeline module to a second pipeline module, such that the external agent engine at least temporarily prevents the pipeline data from reaching its intended destination, namely the second pipeline module, the second pipeline module being a next sequential pipeline module following the first pipeline module; and

an act of the agent engine external to the data pipeline injecting results from the data processing agent back into the data pipeline for further processing, the results being injected at the second pipeline module such that sequential processing continues at the next sequential pipeline module following the first pipeline module, namely the second pipeline module to which the pipeline data was directed prior to such pipeline data being diverted by the agent engine;

the agent engine accessing a plurality of application programming interfaces (API), including at least;

an Initialize API configured to transfer a server value, configuration settings values, and an agent group value for initializing the agent engine;

an Invoke API configured to transfer an execution context value, an event identifier value, and a list of other context values for invoking the plurality of data processing agents;

an Asynchronous Invoke API configured to transfer the execution context value, the event identifier value, a time out value, a callback value, a state value, and the list of other context values for asynchronously invoking the plurality of data processing agents; and

an Event Handler API configured to transfer an event source value and event argument values for executing a specified agent handler within one or more of the plurality of data processing agents; and

the specified known location at the computer system storing data processing agents mapped to each of the following topic strings and Extended Simple Mail Transfer Protocol keywords and verbs;

OnDataCommand, OnEhloCommand, OnEndOfData, OnEndOfHeaders, OnHeloCommand, OnHelpCommand, OnMailCommand, OnNoopCommand, OnRcptCommand, OnReject, OnRsetCommand, OnSessionStopEvent, OnCategorization, OnDeleteMailItem, OnPostCategorization, OnSubmit, OnPerRecipient, OnAuthCommand, SMTP HELLO, and SMTP AUTH.

2. The method as recited in claim 1, wherein the act of detecting an occurrence of an event comprises an act of detecting the occurrence of an electronic mail messaging event.

3. The method as recite in claim 2, wherein the act of detecting occurrence of an electronic mail messaging event comprises an act of detecting an event corresponding to an Simple Mail Transfer Protocol (SMTP) verb.

4. The method as recited in claim 1, wherein the act of determining that the plurality of data processing agents have registered for the detected event comprises an act of determining that the plurality of data processing agents contain a topic string that represents the detected event.

5. The method as recited in claim 1, wherein the act of determining that the plurality of data processing agents have registered for the detected event comprises an act of determining that the plurality of data processing agents include a handler that corresponds to the detected event.

6. The method as recited in claim 1, wherein the act of loading each of the plurality of data processing agents into a secure zone comprises an act of loading each of the plurality of data processing agents into a separate agent container.

7. The method as recited in claim 1, wherein the act of loading each of the plurality of data processing agents into a secure zone comprises an act of establishing a process boundary between each of the plurality of data processing agents and other processes at the computer system.

8. The method as recited in claim 1, wherein the act of loading each of the plurality of data processing agents into a secure zone comprises an act of allocating resources to each of the plurality of data processing agents as dictated by a control module.

9. The method as recited in claim 8, wherein act of allocating resources to each of the plurality of data processing agents as dictated by a control module comprises an act of the control module securely allocating memory resources and processing resources to each of the plurality of data processing agents.

10. A computer program product for use at a computer system, the computer system including a processor and system memory, the computer system also including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequentially processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, the computer program product for implementing a method for extending the functionality of the data processing pipeline, the computer program product comprising one or more physical computer storage devices having stored thereon computer-executable instructions that, when executed by a processor, cause the computer system to perform the following:

at an agent engine that is external to the data pipeline, detect an occurrence of an event within the data pipeline;

map the detected event to a specified topic string representing an electronic messaging command

match the specified topic string to the topic string for a plurality of data processing agents previously stored at a specified known location at the computer system so as to determine that a plurality of data processing agents have registered for the detected event, the plurality of data processing agents being external to the data pipeline;

load each of the plurality of data processing agents into a secure zone such that execution of each data processing agent in the plurality of data processing agents is isolated from execution all other data processing agents in the plurality of data processing agents;

for each of the plurality of data processing agents;

at the agent engine external to the data pipeline, and acting independent of directives from the data pipeline, divert pipeline data related to the represented electronic messaging command out of the data pipeline from a first pipeline module to the data processing agent to extend the functionality of the data pipeline subsequent to loading the data processing agent, wherein the pipeline data is diverted by the external agent engine when the pipeline data is directed from the first pipeline module to a second pipeline module, such that the external agent engine at least temporarily prevents the pipeline data from reaching its intended destination, namely the second pipeline module, the first pipeline module being configured to maintain the pipeline data within the data pipeline by directing the pipeline data to the second pipeline module and the second pipeline module being a next sequential pipeline module following the first pipeline module; and

at the agent engine external to the data pipeline, inject results from the data processing agent back into the data pipeline for further processing, the results being injected at the second pipeline module such that sequential processing continues at the next sequential pipeline module following the first pipeline module, namely the second pipeline module to which the pipeline data was directed prior to such pipeline data being diverted by the agent engine;

the agent engine accesses a plurality of application programming interfaces (API), including at least;

an Initialize API configured to transfer a server value, configuration settings values, and an agent group value for initializing the agent engine;

an Invoke API configured to transfer an execution context value, an event identifier value, and a list of other context values for invoking the plurality of data processing agents;

an Asynchronous Invoke API configured to transfer the execution context value, the event identifier value, a time out value, a callback value, a state value, and the list of other context values for asynchronously invoking the plurality of data processing agents; and

an Event Handler API configured to transfer an event source value and event argument values for executing a specified agent handler within one or more of the plurality of data processing agents; and

the specified known location at the computer system stores data processing agents mapped to each of the following topic strings and Extended Simple Mail Transfer Protocol keywords and verbs;

OnDataCommand, OnEhloCommand, OnEndOfData, OnEndOfHeaders, OnHeloCommand, OnHelpCommand, OnMailCommand, OnNoopCommand, OnRcptCommand, OnReject, OnRsetCommand, OnSessionStopEvent, OnCategorization, OnDeleteMailItem, OnPostCategorization, OnSubmit, OnPerRecipient, OnAuthCommand, SMTP HELO, and SMTP AUTH.

11. The computer program product as recited in claim 10, wherein computer-executable instructions that, when executed by a processor cause the computer system to load each of the plurality of data processing agents into a secure zone comprise computer-executable instructions that, when executed by a processor, cause the computer system to load each of the plurality of data processing agents into an separate agent container.

12. The computer program product as recited in claim 10, wherein computer-executable instructions that, when executed by a processor cause the computer system to load each of the plurality of data processing agents into a secure zone comprise computer-executable instructions that, when executed by a processor, cause the computer system to establish a process boundary between each of the plurality of data processing agents and other processes at the computer system.

13. The computer program product as recited in claim 10, wherein computer-executable instructions that, when executed by a processor cause the computer system to load each of the plurality of data processing agents into a secure zone comprise computer-executable instructions that, when executed by a processor, cause the computer system to allocate resources to each of the plurality of data processing agents as dictated by a control module.

14. The computer program product as recited in claim 13, wherein computer-executable instructions that, when executed by a processor cause the computer system to allocate resources to each of the plurality of data processes agents as dictated by a control module comprise computer-executable instructions that, when executed by a processor, cause the control module to securely allocate memory resources and processing resources to each of the plurality of data processing agents.