Intelligent data storage and processing using FPGA devices

• US 9,176,775 B2
• Filed: 06/26/2014
• Issued: 11/03/2015
• Est. Priority Date: 05/23/2003
• Status: Active Grant

• Current Assignee: Ip Reservoir, LLC
• Original Assignee: Ip Reservoir, LLC
• Inventors: Chamberlain, Roger D., Franklin, Mark Allen, Indeck, Ronald S., Cytron, Ron K., Cholleti, Sharath R.
• Primary Examiner(s): Fleurantin, Jean B
• Application Number:
  US14/315,560
• Publication Number:
  US 20140310717A1
• Time in Patent Office: 495 Days
• Field of Search: 707/636, 707/706, 707/707, 707/770, 707/796, 707/812, 709/214, 711/103, 711/111, 711/112, 711/145, 713/189, 714/752, 705/51, 380/37
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/72 in cryptographic circuits
• CPC Class Codes: G06F 21/76 in application-specific integrated circuits [ASIC] or field-programmable devices, e.g. field-programmable gate arrays [FPGA] or programmable logic devices [PLD]
• CPC Class Codes: G06F 21/85 interconnection devices, e.g. bus-connected or in-line devices
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06Q 40/06 Asset management; Financial planning or analysis

Abstract

A re-configurable logic device such as a field programmable gate array (FPGA) can be used to deploy a data processing pipeline, the pipeline comprising a plurality of pipelined data processing engines, the plurality of pipelined data processing engines being configured to perform processing operations, wherein the pipeline comprises a multi-functional pipeline, and wherein the re-configurable logic device is further configured to controllably activate or deactivate each of the pipelined data processing engines in the pipeline in response to control instructions and thereby define a function for the pipeline, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline.

Claims

1. A data processing apparatus comprising:

a processing device for communicating with a computer system, the computer system including a processor, wherein the processing device comprises a re-configurable logic device configured to perform a plurality of tasks, thereby freeing the processor to perform other tasks;

wherein the re-configurable logic device is configured to receive and process streaming data through a multi-functional pipeline deployed on the re-configurable logic device;

wherein the multi-functional pipeline comprises a plurality of pipelined data processing engines, each pipelined data processing engine being configured to (1) receive streaming data and selectively perform a processing operation on the received streaming data, and (2) be responsive to a control instruction that defines whether that pipelined data processing engine is an activated data processing engine or a deactivated data processing engine, wherein an activated data processing engine is configured to perform the processing operation of that activated data processing engine on streaming data received thereby, and wherein a deactivated data processing engine remains in the pipeline but does not perform the processing operation of that activated data processing engine on streaming data received thereby, the deactivated data processing engine thereby being configured to act as a pass through, the multi-functional pipeline thereby being configured to provide a plurality of different pipeline functions in response to control instructions that are configured to selectively activate and deactivate the pipelined data processing engines, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline at a given time.

2. The apparatus of claim 1 wherein the activated data processing engines in the pipeline are configured to perform their respective processing operations simultaneously such that, with respect to data streaming through the pipeline, a downstream activated data processing engine operates on streaming data that was previously processed by an upstream activated data processing engine.

3. The apparatus of claim 2 wherein the pipelined data processing engines comprise a decryption engine.

4. The apparatus of claim 3 wherein the pipelined data processing engines further comprise a search engine.

5. The apparatus of claim 4 wherein the pipelined data processing engines further comprise an encryption engine.

6. The apparatus of claim 5 wherein the pipelined data processing engines further comprise a compression engine.

7. The apparatus of claim 6 wherein the pipelined data processing engines further comprise a decompression engine.

8. The apparatus of claim 4 wherein, with respect to data streaming through the pipeline, the search engine is downstream from the decryption engine;

and

wherein the pipeline is further configured to receive control instructions that are operative to activate the decryption engine and the search engine such that the pipeline is configured to provide a crypto-searching function on encrypted data that streams through the pipeline.

9. The apparatus of claim 8 wherein the reconfigurable logic device is further configured to store decrypted data from the decryption engine only in a memory that is not accessible to the processor, wherein the memory comprises a member of the group consisting of (i) internal memory of the reconfigurable logic device that is inaccessible to the processor, (ii) a memory device that is deployed on the processing device, the memory device being inaccessible to the processor, and wherein the reconfigurable logic device is also deployed on the processing board, and (iii) a combination of the internal memory and the memory device.

10. The apparatus of claim 3 wherein the pipelined data processing engines further comprise an encryption engine;

wherein, with respect to data streaming through the pipeline, the encryption engine is downstream from the decryption engine; and

wherein the pipeline is further configured to receive control instructions that are operative to activate the decryption engine and the encryption engine such that the pipeline is configured to provide a trans-encryption function on encrypted data that streams through the pipeline.

11. The apparatus of claim 10 wherein the reconfigurable logic device is further configured to store decrypted data from the decryption engine only in a memory that is not accessible to the processor, wherein the memory comprises a member of the group consisting of (i) internal memory of the reconfigurable logic device that is inaccessible to the processor, (ii) a memory device that is deployed on the processing device, the memory device being inaccessible to the processor, and wherein the reconfigurable logic device is also deployed on the processing board, and (iii) a combination of the internal memory and the memory device.

12. The apparatus of claim 10 wherein the pipelined data processing engines further comprise a search engine;

wherein, with respect to data streaming through the pipeline, the search engine is downstream from the decryption engine and upstream from the encryption engine; and

wherein the pipeline is further configured to receive control instructions that are operative to activate the decryption engine, the search engine, and the encryption such that the pipeline is configured to provide a search-enabled trans-encryption function on encrypted data that streams through the pipeline.

13. The apparatus of claim 12 wherein the reconfigurable logic device is further configured to store decrypted data from the decryption engine only in a memory that is not accessible to the processor, wherein the memory comprises a member of the group consisting of (i) internal memory of the reconfigurable logic device that is inaccessible to the processor, (ii) a memory device that is deployed on the processing device, the memory device being inaccessible to the processor, and wherein the reconfigurable logic device is also deployed on the processing board, and (iii) a combination of the internal memory and the memory device.

14. The apparatus of claim 2 wherein the pipelined data processing engines comprise an encryption engine.

15. The apparatus of claim 14 wherein the pipelined data processing engines further comprise a search engine.

16. The apparatus of claim 1 further comprising a mass storage medium, and wherein the pipeline is configured to receive and process data streaming to and from the mass storage medium.

17. The apparatus of claim 16 wherein the pipelined data processing engines comprise a decompression engine and a compression engine.

18. The apparatus of claim 16 wherein the pipelined data processing engines comprise an encryption engine and a decryption engine.

19. The apparatus of claim 1 further comprising the computer system, and wherein the processing device is further configured to communicate with the computer system over a bus.

20. A data processing apparatus comprising:

a field programmable gate array (FPGA) for communicating with a computer system, the computer system including a processor, wherein the FPGA is configured to perform a plurality of tasks, thereby freeing the processor to perform other tasks;

wherein the FPGA is configured to receive and process streaming data through a multi-functional pipeline deployed on the FPGA;

wherein the multi-functional pipeline comprises a plurality of pipelined data processing engines, each pipelined data processing engine being configured to (1) receive streaming data and selectively perform a processing operation on the received streaming data, and (2) be responsive to a control instruction that defines whether that pipelined data processing engine is an activated data processing engine or a deactivated data processing engine, wherein an activated data processing engine is configured to perform the processing operation of that activated data processing engine on streaming data received thereby, and wherein a deactivated data processing engine remains in the pipeline but does not perform the processing operation of that activated data processing engine on streaming data received thereby, the deactivated data processing engine thereby being configured to act as a pass through, the multi-functional pipeline thereby being configured to provide a plurality of different pipeline functions in response to control instructions that are configured to selectively activate and deactivate the pipelined data processing engines, each pipeline function being the combined functionality of each activated pipelined data processing engine in the pipeline at a given time.

21. The apparatus of claim 20 further comprising:

a data storage medium, wherein a plurality of data files are stored in the data storage medium, each data file being stored as a sequence of segments, each segment having a size that is a power of 2;

the computer system; and

wherein the computer system is configured to (1) write the data files to the data storage medium as the sequence of segments, and (2) manage a flow of a plurality of the data files to the FPGA for processing via the pipeline.