Method and apparatus for accelerated record layout detection
First Claim
Patent Images
1. An apparatus for processing a record to determine its record layout, the apparatus comprising:
- a hardware record layout detector for processing a plurality of records, the records comprising data arranged in a format and exhibiting any of a plurality of different record layouts within the format, wherein the hardware record layout detector is deployed on at least one of a reconfigurable logic device, a graphics processing unit (GPU), a multi-core processor, and a cell processor, wherein the hardware record layout detector comprises (1) a plurality of data analysis components arranged in parallel, and (2) logic downstream from the parallel data analysis components; and
a memory;
wherein the parallel data analysis components are configured to analyze the records in parallel, each parallel data analysis component configured to (1) test data in the records against a corresponding condition, and (2) output data indicative of whether the tested data satisfies the corresponding condition, wherein the corresponding conditions for the parallel data analysis components in the aggregate serve as criteria for determining whether the record exhibits any of a plurality of different record layouts, and wherein the corresponding conditions include byte offsets that identify record portions to be processed by different ones of the parallel data analysis components;
wherein the memory is configured to store a configuration table, the configuration table configured to store data representative of the corresponding conditions;
wherein the hardware record layout detector is further configured to define the corresponding conditions for the parallel data analysis components based on the corresponding conditions data in the configuration table; and
wherein the logic is configured to process the output data from the parallel data analysis components to determine, for each of a plurality of the records, whether that record exhibits a record layout from among the plurality of different record layouts;
wherein the hardware record layout detector, in response to a determination that the record exhibits a record layout from among the plurality of different record layouts, is further configured to associate the record with data indicative of the determined record layout; and
wherein the hardware record layout detector is adaptable to detect a new record layout in response to an update to the configuration table that adds corresponding conditions for the new record layout to the configuration table.
1 Assignment
0 Petitions
Accused Products
Abstract
Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.
-
Citations
38 Claims
-
1. An apparatus for processing a record to determine its record layout, the apparatus comprising:
-
a hardware record layout detector for processing a plurality of records, the records comprising data arranged in a format and exhibiting any of a plurality of different record layouts within the format, wherein the hardware record layout detector is deployed on at least one of a reconfigurable logic device, a graphics processing unit (GPU), a multi-core processor, and a cell processor, wherein the hardware record layout detector comprises (1) a plurality of data analysis components arranged in parallel, and (2) logic downstream from the parallel data analysis components; and a memory; wherein the parallel data analysis components are configured to analyze the records in parallel, each parallel data analysis component configured to (1) test data in the records against a corresponding condition, and (2) output data indicative of whether the tested data satisfies the corresponding condition, wherein the corresponding conditions for the parallel data analysis components in the aggregate serve as criteria for determining whether the record exhibits any of a plurality of different record layouts, and wherein the corresponding conditions include byte offsets that identify record portions to be processed by different ones of the parallel data analysis components; wherein the memory is configured to store a configuration table, the configuration table configured to store data representative of the corresponding conditions; wherein the hardware record layout detector is further configured to define the corresponding conditions for the parallel data analysis components based on the corresponding conditions data in the configuration table; and wherein the logic is configured to process the output data from the parallel data analysis components to determine, for each of a plurality of the records, whether that record exhibits a record layout from among the plurality of different record layouts; wherein the hardware record layout detector, in response to a determination that the record exhibits a record layout from among the plurality of different record layouts, is further configured to associate the record with data indicative of the determined record layout; and wherein the hardware record layout detector is adaptable to detect a new record layout in response to an update to the configuration table that adds corresponding conditions for the new record layout to the configuration table. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. A method comprising:
-
adapting a hardware record layout detector to detect a new record layout in streaming records, wherein the hardware record layout detector is deployed on at least one of a reconfigurable logic device, a graphics processing unit (GPU), a multi-core processor, and a cell processor, wherein the hardware record layout detector comprises (1) a plurality of data analysis components arranged in parallel, and (2) logic downstream from the parallel data analysis components, and wherein the adapting step comprises; changing a configuration table in a memory, wherein the configuration table comprises data that defines a plurality of portions of records to be tested to determine their record layouts, wherein the portion definition data is indicative of a plurality of byte offsets for record portions to be tested against a predicate for a record layout test, and wherein the changing step comprises changing the portion definition data in the configuration table; and modifying the hardware record layout detector to detect the new record layout based on the changed configuration table; the modified hardware record layout detector processing the streaming records to determine whether any of the streaming records exhibit the new record layout, wherein the processing step comprises; the parallel data analysis components analyzing the streaming records in parallel, wherein the analyzing step comprises each parallel data analysis component (1) testing data in the streaming records against a corresponding condition that is defined based on data in the changed configuration table, and (2) outputting data indicative of whether the tested data satisfies the corresponding condition, wherein a plurality of the corresponding conditions for the parallel data analysis components in the aggregate serve as criteria for determining whether the record exhibits the new record layout; and the logic processing the output data from the parallel data analysis components to determine whether the any of the streaming records exhibit the new record layout. - View Dependent Claims (25, 26, 27, 28)
-
-
29. A method comprising:
-
streaming a plurality of records through a hardware record layout detector, the records comprising data arranged in a format and exhibiting any of a plurality of different record layouts within the format, wherein the hardware record layout detector is deployed on at least one of a reconfigurable logic device, a graphics processing unit (GPU), a multi-core processor, and a cell processor, and wherein the hardware record layout detector comprises (1) a plurality of data analysis components arranged in parallel, and (2) logic downstream from the parallel data analysis components; the parallel data analysis components analyzing the streaming records in parallel, wherein the analyzing step comprises each parallel data analysis component (1) testing data in the streaming records against a corresponding condition, and (2) outputting data indicative of whether the tested data satisfies the corresponding condition, wherein the corresponding conditions for the parallel data analysis components in the aggregate serve as criteria for determining whether the records exhibit any of a plurality of different record layouts and wherein the corresponding conditions include byte offsets that identify record portions to be processed by different ones of the parallel data analysis components; accessing a configuration table from a memory, the configuration table storing data representative of the corresponding conditions; the hardware record layout detector defining the corresponding conditions for the parallel data analysis components based on the corresponding conditions data in the accessed configuration table; the logic processing the output data from the parallel data analysis components for each of a plurality of the streaming records to determine which of the different record layouts are exhibited by those streaming records; the hardware record layout detector, in response to a determination that a streaming record exhibits a record layout from among the plurality of different record layouts, associating that streaming record with data indicative of the determined record layout; and adapting the hardware record layout detector to detect a new record layout in response to an update to the configuration table that adds corresponding conditions for the new record layout to the configuration table. - View Dependent Claims (30, 31, 32, 33, 34, 35, 36, 37, 38)
-
Specification