Adaptive compression and transmission for big data migration
First Claim
1. A method, the method comprising:
- identifying an information entropy of a first data block having a block size;
receiving a real-time resource status of a system, wherein the system includes a first computer, a second computer, and a communication channel having a bandwidth between the first computer and the second computer;
determining a first preferred compression method to compress the first data block based at least in part on the information entropy and the real-time resource status;
generating a compressed first data block according to the first preferred compression method;
transferring the compressed first data block over the communication channel from the first computer to the second computer; and
decompressing the compressed first data block upon arriving at the second computer.
1 Assignment
0 Petitions
Accused Products
Abstract
A method for optimizing migration efficiency of a data file over network is provided. Specifically, a total time of compression time of the data file, transfer time of the data file over the network, and decompression time of the data file, is minimized by adaptively selecting compression methods to compress each data block of the data file. For selecting a compression method for a data block, information entropy of the data block is analyzed, and a real status of computing and system resources is considered. Further, trade-off among the resource usage, compassion speed and compression ratio is made to calculate an optimized transmission solution over the network for each data block of the data file.
-
Citations
20 Claims
-
1. A method, the method comprising:
-
identifying an information entropy of a first data block having a block size; receiving a real-time resource status of a system, wherein the system includes a first computer, a second computer, and a communication channel having a bandwidth between the first computer and the second computer; determining a first preferred compression method to compress the first data block based at least in part on the information entropy and the real-time resource status; generating a compressed first data block according to the first preferred compression method; transferring the compressed first data block over the communication channel from the first computer to the second computer; and decompressing the compressed first data block upon arriving at the second computer. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A computer program product comprising a non-transitory computer readable storage medium having a set of instructions stored therein which, when executed by a processor, causes the processor to compress adaptively and transmit big data by:
-
identifying an information entropy of a first data block having a block size; receiving a real-time resource status of a system, wherein the system includes a first computer, a second computer, and a communication channel having a bandwidth between the first computer and the second computer; determining a first preferred compression method to compress the first data block based at least in part on the information entropy and the real-time resource status; generating a compressed first data block according to the first preferred compression method; transferring the compressed first data block over the communication channel from the first computer to the second computer; and decompressing the compressed first data block upon arriving at the second computer. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A computer system comprising:
-
a processor set; and a computer readable storage medium; wherein; the processor set is structured, located, connected, and/or programmed to run program instructions stored on the computer readable storage medium; and the program instructions which, when executed by the processor set, cause the processor set to compress adaptively and transmit big data by; identifying an information entropy of a first data block having a block size; receiving a real-time resource status of a system, wherein the system includes a first computer, a second computer, and a communication channel having a bandwidth between the first computer and the second computer; determining a first preferred compression method to compress the first data block based at least in part on the information entropy and the real-time resource status; generating a compressed first data block according to the first preferred compression method; transferring the compressed first data block over the communication channel from the first computer to the second computer; and decompressing the compressed first data block upon arriving at the second computer. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification