×

HASH PARTITIONING STREAMED DATA

  • US 20110154359A1
  • Filed: 12/18/2009
  • Published: 06/23/2011
  • Est. Priority Date: 12/18/2009
  • Status: Active Grant
First Claim
Patent Images

1. At a computer system including one or more processors and system memory, the computer system also including an input data stream, a plurality of worker threads, and a corresponding plurality of output data streams, each worker thread in the plurality of worker threads directly corresponding to an output data stream in the plurality of output data streams, each worker thread in the plurality of worker threads configured to determine the equivalence of data elements from the input data stream in accordance with an equivalence function and to assign data elements determined to be equivalent to the same worker thread for output on the worker'"'"'s corresponding output data stream such that equivalent data elements are output on the same output data stream, a method for distributing data elements from the input data stream across the plurality of output data streams, the method comprising:

  • an act of a first worker thread, from among the plurality of worker threads, acquiring a lock on the input data stream, the lock shared among the plurality of worker threads;

    an act of the first worker thread accessing one or more data elements from the input data stream subsequent to locking the input data stream;

    an act of the first worker thread releasing the lock on the input data stream;

    for each data element in the one or more data elements;

    an act of the first worker thread using the equivalence function to identify the appropriate output data stream, from among the plurality of output data streams, for outputting the data element; and

    an act of the first worker thread assigning the data element to a worker thread corresponding to the identified appropriate output data stream such that data elements determined to be equivalent by the equivalence function are assigned to the same worker thread; and

    an act of the first worker thread determining if other worker threads assigned any data elements from the input data stream to the first worker thread for output on the first worker thread'"'"'s corresponding output data stream.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×