Field oriented pipeline architecture for a programmable data streaming processor
First Claim
1. A data engine, located in a programmable pipeline processor, for processing non-field delineated, streaming, application level database records received from a mass storage device, the data engine comprising:
- a data parser configured to recognize the record and field structure of the non-field delineated database records, determine field boundaries in the non-field delineated database records, parse the non-field delineated database records into field delineated data, and process fields to select one or more fields to be assembled into output tuples;
filter logic configured to receive the field delineated data from the data parser, and to filter the field delineated data by performing a field operation on the field delineated data; and
an output tuple generator configured to assemble filtered field delineated data into an output tuple.
8 Assignments
0 Petitions
Accused Products
Abstract
A field oriented pipeline processor for a data engine that can be programmed to recognize record and field structures of data received from a source such as a mass storage device. The processor has an interface for receiving field-delineated data from a field parser. The field parser parses non-field delineated data from a streaming data source into field delineated data under instruction from an external processing unit. The pipeline processor receives a field delineated data stream and employs logical arithmetic methods to compare fields with one another, or with values otherwise supplied by general purpose processors to precisely determine which records are worth transferring to memory of the more general purpose distributed processors. The architecture of the data engine allows for the use of substitution tables, temporary registers, and a data string register to assist in the efficiency and accuracy of the data engine processing.
79 Citations
20 Claims
-
1. A data engine, located in a programmable pipeline processor, for processing non-field delineated, streaming, application level database records received from a mass storage device, the data engine comprising:
-
a data parser configured to recognize the record and field structure of the non-field delineated database records, determine field boundaries in the non-field delineated database records, parse the non-field delineated database records into field delineated data, and process fields to select one or more fields to be assembled into output tuples; filter logic configured to receive the field delineated data from the data parser, and to filter the field delineated data by performing a field operation on the field delineated data; and an output tuple generator configured to assemble filtered field delineated data into an output tuple. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for processing, in a programmable pipeline processor, non-field delineated, streaming, application level database records received from a mass storage device, the method comprising:
-
receiving the non-field delineated database records in a field buffer as an input data stream; recognizing the record and field structure of the non-field delineated database records in the field buffer; determining field boundaries in the non-field delineated database records by separating the input data stream into field delineated data in the field buffer under instruction from an external central processing unit; processing fields to select one or more fields to be assembled into output tuples; filtering the field delineated data by sending field delineated data from the field buffer to at least one logic unit that performs at least one field operation on the field delineated data; and assembling the filtered field delineated data into an output tuple. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification