Dynamic Monitoring of Ability to Reassemble Streaming Data Across Multiple Channels Based on History
First Claim
1. A method, in a data processing system, for processing streaming data at high sustained data rates, comprising:
- receiving a plurality of data elements, in the data processing system, over a plurality of non-sequential communication channels;
writing the plurality of data elements directly to a file system of the data processing system in an unassembled manner;
determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner;
performing a data scrubbing operation, in response to a determination to perform data scrubbing, to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system; and
assembling the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements.
2 Assignments
0 Petitions
Accused Products
Abstract
Mechanisms are provided for processing streaming data at high sustained data rates. These mechanisms receive a plurality of data elements over a plurality of non-sequential communication channels and write the plurality of data elements directly to the file system of the data processing system in an unassembled manner. The mechanisms determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner. The mechanisms perform a data scrubbing operation, in response to a determination to perform data scrubbing, to identify any missing data elements in the plurality of data elements written to the file system and assemble the plurality of data elements into a plurality of data streams in response to results of the data scrubbing indicating that there are no missing data elements.
-
Citations
20 Claims
-
1. A method, in a data processing system, for processing streaming data at high sustained data rates, comprising:
-
receiving a plurality of data elements, in the data processing system, over a plurality of non-sequential communication channels; writing the plurality of data elements directly to a file system of the data processing system in an unassembled manner; determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner; performing a data scrubbing operation, in response to a determination to perform data scrubbing, to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system; and assembling the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A computer program product comprising a computer recordable medium having a computer readable program recorded thereon, wherein the computer readable program, when executed on a computing device, causes the computing device to:
-
receive a plurality of data elements, in the computing device, over a plurality of non-sequential communication channels; write the plurality of data elements directly to a file system of the computing device in an unassembled manner; determine whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner; perform a data scrubbing operation, in response to a determination to perform data scrubbing, to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system; and assemble the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. An apparatus, comprising:
-
a processor; and a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to; receive a plurality of data elements, in the apparatus, over a plurality of non-sequential communication channels; write the plurality of data elements directly to a file system of the apparatus in an unassembled manner; determine whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner; perform a data scrubbing operation, in response to a determination to perform data scrubbing, to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system; and assemble the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements.
-
Specification