×

Asymmetric streaming record data processor method and apparatus

  • US 7,529,752 B2
  • Filed: 09/18/2003
  • Issued: 05/05/2009
  • Est. Priority Date: 09/18/2002
  • Status: Expired due to Fees
First Claim
Patent Images

1. An asymmetric data processor comprising:

  • one or more host computers, each including a memory, a network interface and at least one CPU, each host computer being responsive to requests from end users and applications to process data;

    a plurality of Job Processing Units (JPUs), each having a memory, a network interface, one or more storage devices, and at least one CPU, each JPU being responsive to requests from host computers and from other JPUs to process data;

    a network enabling the host computers and the JPUs to communicate between and amongst each other, each of the host computers and JPUs forming a respective node on the network; and

    a plurality of software operators configured to process data at the nodes according to a logical data flow, wherein (i) for each operator in a given sequence of operators in the logical data flow, output of the operator is input to a respective succeeding operator in the sequence in a manner free of necessarily materializing data, and (ii) data processing at each operator is based on readiness of a record, such that the operator transmits ready record data for processing at a successive operator in the logical data flow independent of transmission at other operators, the transmission of ready record data during data processing being substantially continuous so as to form a stream of record processing from operator to operator within nodes and across nodes of the network;

    wherein record data are processed at intermediate parts on the logical data flow as a collection of data field values in a manner free of being materialized as whole records between two successive operators; and

    wherein the plurality of operators includes one or more join operators, each join operator having multiple input streams and an output stream with references to original records in their packed form, and the output stream for the operator referring to data field values within the record data of the input streams at known offsets from a base pointer to a start of a packed record.

View all claims
  • 8 Assignments
Timeline View
Assignment View
    ×
    ×