HARDWARE ACCELERATION FOR A COMPRESSED COMPUTATION DATABASE

• US 20160098420A1
• Filed: 05/04/2015
• Published: 04/07/2016
• Est. Priority Date: 10/03/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dickie, Garth A., Sporer, Michael, Viehland, Jason A.
• Granted Patent: US 9,858,285 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity

Abstract

According to embodiments of the present invention, machines, systems, methods and computer program products for hardware acceleration are presented. A plurality of computational nodes for processing data is provided, each node performing a corresponding operation for data received at that node. A metric module is used to determine a compression benefit metric pertaining to performance of the corresponding operations of one or more computational nodes with recompressed data. An accelerator module recompresses data for processing by the one or more computational nodes based on the compression benefit metric indicating a benefit gained by using the recompressed data. A distribution function may be used to distribute data among a plurality of nodes.

Claims

1. A method of processing data using a plurality of computational nodes, each node performing a corresponding operation for data received at that node, and said method comprising:

determining a compression benefit metric pertaining to performance of the corresponding operations of one or more computational nodes with recompressed data; and

recompressing the data for processing by the one or more computational nodes based on the compression benefit metric indicating a benefit gained by using the recompressed data.

2. The method of claim 1, wherein the compression benefit metric is determined using one or more of the following:

(a) a measure of data compression preserved by one or more destination computational nodes;

(b) a measure of an estimated reduction in a size of the data resulting from recompression;

(c) a measure of an estimated computational benefit from a destination node performing an operation on recompressed data; and

(d) a measure of a computational benefit of a destination node performing an operation on data in a particular compressed form.

3. The method of claim 1, wherein determining a compression benefit metric further determines a priority for recompressing data based on the compression benefit metric, and recompressing data according to the priority.

4. The method of claim 1, wherein one or more computational nodes process data without recompression in response to data awaiting recompression and being unavailable for processing.

5. The method of claim 1, said method comprising:

decompressing compressed data; and

compressing the decompressed data and producing recompressed data.

6. The method of claim 1, comprising utilizing a plurality of processors interconnected by a network to process data in parallel, said method comprising:

applying a distribution function to distribute data among the plurality of processors;

compressing data prior to transmission to the plurality of processors; and

compressing data received from the plurality of processors.