Method and Apparatus for Record Pivoting to Accelerate Processing of Data Fields

• US 20150310087A1
• Filed: 04/23/2015
• Published: 10/29/2015
• Est. Priority Date: 10/23/2012
• Status: Active Grant

• Current Assignee: Ip Reservoir, LLC
• Original Assignee: Ip Reservoir, LLC
• Inventors: Tidwell, Terry, St. John, Alex, Sewell, Daniel
• Granted Patent: US 9,633,097 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 21/71 to assure secure computing or processing of information

Abstract

Various methods and apparatuses are described for performing high speed translations of data. In an example embodiment, record layout detection can be performed for data. In another example embodiment, data pivoting prior to field-specific data processing can be performed.

Claims

1. A method comprising:

processing a plurality of records, the records comprising a plurality of fields, the fields corresponding to a plurality of different field types, and wherein a plurality of the records include fields of the same field type; and

based on the processing, pivoting the records to group a plurality of fields from different records that share the same field type; and

applying the grouped fields to a data processing operation; and

wherein the method steps are performed by a processor.

2. The method of claim 1 wherein the data processing operation is a field-specific data processing operation.

3. The method of claim 2 wherein the field-specific data processing operation is an encryption operation.

4. The method of claim 3 wherein the encryption operation is a format preserving encryption (FPE) operation.

5. The method of claim 1 further comprising:

performing the data processing operation on the applied grouped fields.

6. The method of claim 5 wherein the performing step is performed by a member of the group consisting of (1) a graphics processor unit (GPU), (2) a cell processor, (3) a multi-core processor, and (4) a reconfigurable logic device.

7. The method of claim 6 wherein the member comprises the GPU.

8. The method of claim 5 wherein the performing step comprises the member performing the data processing operation on the grouped fields as a batch.

9. The method of claim 5 further comprising:

depivoting the processed fields downstream from the data processing operation to merge the processed fields into their respective corresponding records.

10. The method of claim 1 wherein the records include a plurality of social security number fields, wherein the pivoting step comprises grouping the social security number fields from a plurality of different records, and wherein the applying step comprises encrypting the grouped social security number fields.

11. An apparatus comprising:

a processor configured to (1) process a plurality of records, the records comprising a plurality of fields, the fields corresponding to a plurality of different field types, and wherein a plurality of the records include fields of the same field type, (2) based on the processing, pivot the records to group a plurality of fields from different records that share the same field type, and (3) apply the grouped fields to a data processing operation.

12. The apparatus of claim 11 wherein the data processing operation is a field-specific data processing operation.

13. The apparatus of claim 12 wherein the field-specific data processing operation is an encryption operation.

14. The apparatus of claim 13 wherein the encryption operation is a format preserving encryption (FPE) operation.

15. The apparatus of claim 11 wherein the processor is further configured to perform the data processing operation on the applied grouped fields.

16. The apparatus of claim 15 wherein the processor comprises a member of the group consisting of (1) a graphics processor unit (GPU), (2) a cell processor, (3) a multi-core processor, and (4) a reconfigurable logic device, and wherein the member is configured to perform the data processing operation.

17. The apparatus of claim 15 wherein member is further configured to perform the data processing operation on the grouped fields as a batch.

18. The apparatus of claim 17 wherein the member comprises the GPU.

19. The apparatus of claim 15 wherein the processor is further configured to depivot the processed fields downstream from the data processing operation to merge the processed fields into their respective corresponding records.

20. The apparatus of claim 11 wherein the processor comprises a first processor and a second processor, the second processor comprising a member of the group consisting of (1) a graphics processor unit (GPU), (2) a multi-core processor, (3) a cell processor, and (4) a reconfigurable logic device, wherein the records include a plurality of social security number fields;

wherein the first processor is configured to, as part of the pivot, group the social security number fields from a plurality of different records; and

wherein the second processor is configured to encrypt the grouped social security number fields.

21. An apparatus comprising:

a processor;

a memory comprising a first buffer, a second buffer, and a third buffer; and

a member of the group consisting of (1) a graphics processor unit (GPU), (2) a multi-core processor, (3) a cell processor, and (4) a reconfigurable logic device;

wherein the processor is configured to (1) receive a plurality of incoming records, the incoming records comprising a plurality of fields, (2) write the incoming records to the first buffer, (3) identify which of the fields of the incoming records are to be processed by the member, and (4) write the identified fields to the second buffer;

wherein the member is configured to (1) read the fields in the second buffer, (2) perform a data processing operation on the read fields to generate a plurality of processed fields, and (3) write the processed fields to the third buffer; and

wherein the processor is further configured to (1) read the processed fields in the third buffer, (2) read the fields for the incoming records in the first buffer, and (3) merge the processed fields with the fields from the incoming records to generate a plurality of outgoing records such that the outgoing records include the processed fields in place of their corresponding fields of the incoming records.

22. The apparatus of claim 21 wherein the processor is configured to execute a first software module and a second software module;

the first software module comprising a plurality of instructions configured to cause the processor to (1) receive the incoming records, (2) write the incoming records to the first buffer, (3) identify which of the fields of the incoming records are to be processed by the member, and (4) write the identified fields to the second buffer;

the second software module comprising a plurality of instructions configured to cause the processor to (1) read the processed fields in the third buffer, (2) read the fields for the incoming records in the first buffer, and (3) merge the processed fields with the fields from the incoming records to generate the outgoing records such that the outgoing records include the processed fields in place of their corresponding fields of the incoming records.

23. The apparatus of claim 21 wherein the memory further comprises:

an input ring buffer in which the incoming records are stored; and

an output ring buffer in which the outgoing records are stored;

wherein the processor is configured to (1) receive the incoming records from the input ring buffer and (2) write the outgoing records into the output ring buffer.

24. The apparatus of claim 21 wherein the member is configured to perform an encryption operation on the fields read from the second buffer.

25. The apparatus of claim 24 wherein the encryption operation is format preserving encryption (FPE).

26. The apparatus of claim 21 wherein the second buffer comprises a pool of ingress buffers.

27. The apparatus of claim 26 wherein different fields of the incoming records are written to different ingress buffers to support multi-field processing by the member.

28. The apparatus of claim 21 wherein the third buffer comprises a pool of egress buffers.

29. The apparatus of claim 21 wherein the second buffer comprises a pool of buffers that can be selectively allocated as ingress buffers or egress buffers.

30. The apparatus of claim 21 wherein the member comprises the GPU.

31. The apparatus of claim 21 wherein the member comprises the multi-core processor.

32. The apparatus of claim 21 wherein the member comprises the cell processor.

33. The apparatus of claim 21 wherein the member comprises the reconfigurable logic device.