Systems and methods for optimal component composition in a stream processing system

• US 7,562,355 B2
• Filed: 03/01/2005
• Issued: 07/14/2009
• Est. Priority Date: 03/01/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yu, Philip S., Gu, Xiaohui
• Primary Examiner(s): CHAVIS, JOHN Q
• Application Number:
  US11/068,785
• Publication Number:
  US 20060200251A1
• Time in Patent Office: 1,596 Days
• Field of Search: 717/151
• US Class Current: 717/151
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272

Abstract

A system and method are provided for optimizing component composition in a distributed stream-processing environment having a plurality of nodes capable of being associated with one or more of a plurality of stream processing components. The system includes an adaptive composition probing (ACP) module and a hierarchical state manager. The ACP module probes a subset of the plurality of stream processing components to determine the optimal component composition in response to a stream processing request. The hierarchical state manager manages local and global information for use by said ACP module in determining the optimal component composition.

Claims

1. A method for optimizing component composition in a distributed stream-processing environment having a plurality of nodes capable of being associated with one or more of a plurality of stream processing components, the method comprising the steps of:

performing an adaptive composition probing (ACP) which probes a subset of the plurality of stream processing components to determine the optimal component composition in response to a stream processing request; and

managing local and global information for use by said performing step in determining the optimal component composition,

wherein said performing step adaptively determines the number of probes used for the stream processing request based on at least one of system performance targets and current system conditions, and

wherein said performing step adaptively performs per-hop component selection based on at least one of coarse-grain global state information, accumulated probing results, and stream rate matching.