Method and apparatus for accelerated format translation of data in a delimited data format
First Claim
Patent Images
1. A method for low latency and high throughput data translation, the method comprising:
- receiving, by a pipeline, an incoming stream comprising a plurality of bytes arranged in a delimited data format, the incoming byte stream being representative of data arranged in a plurality of fields, the incoming byte stream comprising a plurality of data characters, a plurality of shield characters, and a plurality of field delimiter characters, the field delimiter characters defining a plurality of boundaries between the fields, wherein the pipeline is deployed on at least one of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), (3) an application-specific integrated circuit (ASIC), and/or (4) a chip multi-processor (CMP);
the pipeline processing the bytes of the received byte stream as the bytes stream through the pipeline, wherein the processing step includes the pipeline translating the received byte stream to an outgoing byte stream arranged in a fixed field format, the outgoing byte stream comprising a plurality of the data characters of the received byte stream arranged in a plurality of fixed-size fields and stripped of the field delimiter characters and the shield characters;
wherein the processing step comprises the pipeline repeatedly performing a shield character masking step and a field delimiter finding step in parallel and in a pipelined manner as the bytes stream through the pipeline to create the outgoing byte stream;
wherein the shield character masking step comprises the pipeline generating a shield character mask based on the shield characters in the received byte stream, wherein the shield character mask distinguishes between bytes in the received byte stream that may include field delimiter characters and bytes in the received byte stream that do not include field delimiter characters; and
wherein the field delimiter finding step comprises the pipeline identifying the field delimiter characters in the received byte stream based on the shield character mask.
3 Assignments
0 Petitions
Accused Products
Abstract
Various methods and apparatuses are described for performing high speed format translations of incoming data, where the incoming data is arranged in a delimited data format. As an example, the data in the delimited data format can be translated to a fixed field format using pipelined operations. A reconfigurable logic device can be used in exemplary embodiments as a platform for the format translation.
-
Citations
40 Claims
-
1. A method for low latency and high throughput data translation, the method comprising:
-
receiving, by a pipeline, an incoming stream comprising a plurality of bytes arranged in a delimited data format, the incoming byte stream being representative of data arranged in a plurality of fields, the incoming byte stream comprising a plurality of data characters, a plurality of shield characters, and a plurality of field delimiter characters, the field delimiter characters defining a plurality of boundaries between the fields, wherein the pipeline is deployed on at least one of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), (3) an application-specific integrated circuit (ASIC), and/or (4) a chip multi-processor (CMP); the pipeline processing the bytes of the received byte stream as the bytes stream through the pipeline, wherein the processing step includes the pipeline translating the received byte stream to an outgoing byte stream arranged in a fixed field format, the outgoing byte stream comprising a plurality of the data characters of the received byte stream arranged in a plurality of fixed-size fields and stripped of the field delimiter characters and the shield characters; wherein the processing step comprises the pipeline repeatedly performing a shield character masking step and a field delimiter finding step in parallel and in a pipelined manner as the bytes stream through the pipeline to create the outgoing byte stream; wherein the shield character masking step comprises the pipeline generating a shield character mask based on the shield characters in the received byte stream, wherein the shield character mask distinguishes between bytes in the received byte stream that may include field delimiter characters and bytes in the received byte stream that do not include field delimiter characters; and wherein the field delimiter finding step comprises the pipeline identifying the field delimiter characters in the received byte stream based on the shield character mask. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
-
-
21. An apparatus for low latency and high throughput data translation, the apparatus comprising:
-
at least one of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), (3) an application-specific integrated circuit (ASIC), and/or (4) a chip multi-processor (CMP) on which a pipeline is deployed; wherein the pipeline is configured to receive an incoming stream comprising a plurality of bytes arranged in a delimited data format, the incoming byte stream being representative of data arranged in a plurality of fields, the incoming byte stream comprising a plurality of data characters, a plurality of shield characters, and a plurality of field delimiter characters, wherein the field delimiter characters define a plurality of boundaries between the fields; wherein the pipeline is further configured to process the bytes of the received byte stream as the bytes stream through the pipeline to translate the received byte stream to an outgoing byte stream arranged in a fixed field format, the outgoing byte stream comprising a plurality of the data characters of the received byte stream arranged in a plurality of fixed-size fields and stripped of the field delimiter characters and the shield characters; wherein the pipeline comprises a plurality of processing modules that are configured to operate in parallel and in a pipelined manner to create the outgoing byte stream as the bytes stream through the pipeline, wherein the processing modules comprise a first processing module and a second processing module; wherein the first processing module is configured to generate a shield character mask based on the shield characters in the received byte stream, wherein the shield character mask distinguishes between bytes in the received byte stream that may include field delimiter characters and bytes in the received byte stream that do not include field delimiter characters; and wherein the second processing module is configured to identify the field delimiter characters in the received byte stream based on the shield character mask. - View Dependent Claims (22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40)
-
Specification