Securely extending data processing pipeline functionality

• US 7,627,636 B2
• Filed: 10/27/2004
• Issued: 12/01/2009
• Est. Priority Date: 08/27/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Sharma, Vivek, Wei, Min
• Primary Examiner(s): LAZARO, DAVID R
• Application Number:
  US10/974,135
• Publication Number:
  US 20060075045A1
• Time in Patent Office: 1,861 Days
• Field of Search: 709/201, 709/202, 709/206, 709/207, 709/223, 709/224, 719313-318, 719/328
• US Class Current: 709/206
• CPC Class Codes: G06F 9/4862 the task being a mobile agent, i.e. specifically designed to migrate

Abstract

The present invention extends to methods, systems, and computer program products for securely and efficiently extending data processing pipeline functionality. Data processing agents register, potentially based solely on configuration data contained in the data processing agents, to manipulate data of a data processing pipeline. Configuration data can identify a specified event (e.g., an SMTP verb) the data processing agent is to hook into. Upon detection of a specified event, one or more agents that have registered for the specified event are loaded into secure zones. Process boundaries separate agents from one another and from other processes such that execution of each agent is isolated from other processes. Pipeline related data (e.g., electronic message data) is transferred to each loaded agent. Application Program Interfaces (“APIs”) can be used to transfer data between modules that implement embodiments of the present invention.

Claims

1. A computer program product for use at a computer system, the computer system including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequential processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, the computer program product for implementing a method for invoking an agent runtime to customize the functionality of a data pipeline, the computer program product comprising computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the computer system to perform the method, including the following:

receive electronic message related data at a pipeline module in the sequential arrangement of pipeline modules;

detect the occurrence of an event at the pipeline module in response to processing the electronic message related data at the pipeline module;

mapping the detected event to a specified topic string representing an electronic messaging command;

scan configuration data for stored agents to attempt to identify any stored agents having topic strings corresponding to the specified topic string;

indentify an agent having a topic string corresponding to the specified topic string, the agent having the corresponding topic string indicating that the agent is self-configured to hook into the detected event;

loading the identified agent such that the identified agent can be executed to manipulate messaging data associated with the detected event in accordance with the configuration data for the identified agent, loading the identified agent including supplying appropriate values to an invoke API to invoke the identified agent, supplying appropriate values including supplying values for;

an execution context field for storing an execution context value that identifies an execution context object for an agent runtime;

an event topic field for storing an event topic value that represents a type of event that is to be processed by the agent runtime in the execution context object identified by the execution context value stored in the execution context field;

a contexts field for storing a list of context values that represent a corresponding list of objects from a data processing pipeline that can be accessed from the execution context object identified by the execution context value stored in the execution context field;

divert pipeline data related to the represented messaging command out of the data pipeline to the identified agent to extend the functionality of the data pipeline subsequent to loading the agent; and

inject results from the identified agent back into the data pipeline for further processing.

2. The computer program product as recited in claim 1, wherein the execution context field for storing an execution context value comprises an execution context field for storing per session information related to agents.

3. The computer program product as recited in claim 1, wherein the event topic field for storing an event topic value comprises an event topic field for storing an event topic string that identifies a type of event.

4. The computer program product as recited in claim 3, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an SMTP verb.

5. The computer program product as recited in claim 3, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an SMTP keyword.

6. The computer program product as recited in claim 3, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an event from the data processing pipeline.

7. The computer program product as recited in claim 3, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an user-defined event for performing a specified messaging operation.

8. The computer program product as recited in claim 1, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP verbs.

9. The computer program product as recited in claim 1, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP keywords.

10. The computer program product as recited in claim 1, wherein the contexts field for storing a list of context values comprises a contexts field for storing an object corresponding to a user-defined event.

11. A computer program product for use at a computer system, the computer system including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequential processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, the computer program product for implementing a method for invoking an agent runtime to customize the functionality of a data pipeline, the computer program product comprising computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the computer system to perform the method, including the following:

receive electronic message related data at a pipeline module in the sequential arrangement of pipeline modules;

detect the occurrence of an event at the pipeline module in response to processing the electronic message related data at the pipeline module;

mapping the detected event to a specified topic string representing an electronic messaging command;

scan configuration data for stored agents to attempt to identify any stored agents having topic strings corresponding to the specified topic string;

indentify an agent having a topic string corresponding to the specified topic string, the agent having the corresponding topic string indicating that the agent is self-configured to hook into the detected event;

asynchronously invoke the identified agent such that the identified agent can be executed to manipulate data related to the detected event in accordance with the configuration data for the identified agent, asynchronously invoking the identified agent including supplying appropriate values to an invoke API for the identified agent, supplying appropriate values including supplying values for;

an execution context field for storing an execution context value that identifies an execution context object for an agent runtime;

an event topic field for storing an event topic value that represents a type of event that is to be processed by the agent runtime in the execution context object identified by the execution context value stored in the execution context field;

a time out field for storing a time out value that represents an amount of time after which the attempt to asynchronously invoke an agent runtime will time out;

a callback field for storing a callback value that represents an object to be called when the asynchronous invocation of the agent runtime is complete;

a state field for storing a state value that distinguishes the asynchronous invocation of the agent runtime from other invocations of the agent runtime; and

a contexts field for storing a list of context values that represent a corresponding list of objects from a data processing pipeline that can be accessed from the execution context object identified by the execution context value stored in the execution context field;

divert pipeline data related to the represented messaging command out of the data pipeline to the identified agent to extend the functionality of the data pipeline subsequent to loading the agent; and

inject results from the identified agent back into the data pipeline for further processing.

12. The computer program product as recited in claim 11, wherein the execution context field for storing an execution context value comprises an execution context field for storing per session information related to agents.

13. The computer program product as recited in claim 11, wherein the event topic field for storing an event topic value comprises an event topic field for storing an event topic string that identifies a type of event.

14. The computer program product as recited in claim 11, wherein the time out field for storing a time out value comprises a time out field for storing a time out value in milliseconds.

15. The computer program product as recited in claim 11, wherein state field for storing a state value comprises a state field for storing a user-defined object.

16. The interface computer program product as recited in claim 11, where the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP verbs.

17. The computer program product as recited in claim 11, where the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP keywords.

18. The computer program product as recited in claim 11, where the contexts field for storing a list of context values comprises a contexts field for storing an object corresponding to a user-defined event.

19. At a computer system, the computer system including a processor and system memory, the computer system also including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequential processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, a method for invoking an agent runtime to customize the functionality of a data pipeline, the method comprising:

receiving electronic message related data at a pipeline module in the sequential arrangement of pipeline modules;

detecting the occurrence of an event at the pipeline module in response to processing the electronic message related data at the pipeline module;

mapping the detected event to a specified topic string representing an electronic messaging command;

scanning configuration data for stored agents to attempt to identify any stored agents having topic strings corresponding to the specified topic string;

indentifying an agent having a topic string corresponding to the specified topic string, the agent having the corresponding topic string indicating that the agent is self-configured to hook into the detected event;

the processor loading the identified agent such that the identified agent can be executed to manipulate messaging data associated with the detected event in accordance with the configuration data for the identified agent, loading the identified agent including supplying appropriate values to an invoke API to invoke the identified agent, supplying appropriate values including supplying values for;

an execution context field for storing an execution context value that identifies an execution context object for an agent runtime;

an event topic field for storing an event topic value that represents a type of event that is to be processed by the agent runtime in the execution context object identified by the execution context value stored in the execution context field;

a contexts field for storing a list of context values that represent a corresponding list of objects from a data processing pipeline that can be accessed from the execution context object identified by the execution context value stored in the execution context field;

diverting pipeline data related to the represented messaging command out of the data pipeline to the identified agent to extend the functionality of the data pipeline subsequent to loading the agent; and

injecting results from the identified agent back into the data pipeline for further processing.

20. The method as recited in claim 19, wherein the execution context field for storing an execution context value comprises an execution context field for storing per session information related to agents.

21. The method as recited in claim 19, wherein the event topic field for storing an event topic value comprises an event topic field for storing an event topic string that identifies a type of event.

22. The method as recited in claim 21, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an SMTP verb.

23. The method as recited in claim 21, wherein the event topic field for storing an event topic string that identifies a type of event comprises an event topic field for storing an event topic string that identifies an SMTP keyword.

24. The method as recited in claim 19, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP verbs.

25. The method as recited in claim 19, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP keywords.

26. At a computer system, the computer system including a processor and system memory, the computer system also including a data pipeline comprising a sequential arrangement of a plurality of pipeline modules, an electronic message received at the data pipeline being sequential processed by pipeline modules such that the output of at least one pipeline module is the input to the next sequential pipeline module, a method for invoking an agent runtime to customize the functionality of a data pipeline, the method comprising:

receiving electronic message related data at a pipeline module in the sequential arrangement of pipeline modules;

detecting the occurrence of an event at the pipeline module in response to processing the electronic message related data at the pipeline module;

mapping the detected event to a specified topic string representing an electronic messaging command;

scanning configuration data for stored agents to attempt to identify any stored agents having topic strings corresponding to the specified topic string;

indentifying an agent having a topic string corresponding to the specified topic string, the agent having the corresponding topic string indicating that the agent is self-configured to hook into the detected event;

the processor asynchronously invoking the identified agent such that the identified agent can be executed to manipulate data related to the detected event in accordance with the configuration data for the identified agent, asynchronously invoking the identified agent including supplying appropriate values to an invoke API for the identified agent, supplying appropriate values including supplying values for;

an execution context field for storing an execution context value that identifies an execution context object for an agent runtime;

an event topic field for storing an event topic value that represents a type of event that is to be processed by the agent runtime in the execution context object identified by the execution context value stored in the execution context field;

a time out field for storing a time out value that represents an amount of time after which the attempt to asynchronously invoke an agent runtime will time out;

a callback field for storing a callback value that represents an object to be called when the asynchronous invocation of the agent runtime is complete;

a state field for storing a state value that distinguishes the asynchronous invocation of the agent runtime from other invocations of the agent runtime; and

a contexts field for storing a list of context values that represent a corresponding list of objects from a data processing pipeline that can be accessed from the execution context object identified by the execution context value stored in the execution context field;

diverting pipeline data related to the represented messaging command out of the data pipeline to the identified agent to extend the functionality of the data pipeline subsequent to loading the agent; and

injecting results from the identified agent back into the data pipeline for further processing.

27. The method as recited in claim 26, wherein the time out field for storing a time out value comprises a time out field for storing a time out value in milliseconds.

28. The method as recited in claim 26, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP verbs.

29. The method as recited in claim 28, wherein the contexts field for storing a list of objects corresponding to SMTP verbs comprises a contexts field for storing a list of objects corresponding to SMTP verbs selected from among:

DATA, HELO, MAIL, NOOP, QUIT, RCPT, RSET, and VRFY SMTP verbs.

30. The method as recited in claim 26, wherein the contexts field for storing a list of context values comprises a contexts field for storing a list of objects corresponding to SMTP keywords.