Dynamic data compression selection
First Claim
1. A method for data compression, the method comprising:
- receiving uncompressed source data at a data compression system from a data storage device, the uncompressed source data comprising a plurality of uncompressed data chunks;
determining, at a first time, a first number of threads to employ to compress the uncompressed source data;
identifying, based at least in part on the first number of threads, a first compression technique from a plurality of compression techniques to use to compress one or more first uncompressed data chunks of the plurality of uncompressed data chunks;
compressing, via one or more threads, the one or more first uncompressed data chunks using the first compression technique to produce one or more first compressed data chunks, a first quantity of the one or more threads being equal to the first number of threads;
routing the one or more first compressed data chunks to a data target over a network; and
at each second time of one or more second times;
determining, at the second time, a next number of threads to employ to compress the uncompressed source data, the next number of threads being different than the first number of threads;
identifying, based at least in part on the next number of threads and utilizing one or more hardware processors of the data compression system, a next compression technique from the plurality of compression techniques to use to compress one or more next uncompressed data chunks of the plurality of uncompressed data chunks, wherein, for each of at least one second time of the one or more second times, the next compression technique is different than the first compression technique;
compressing, via at least one thread, the one or more next uncompressed data chunks of the plurality of uncompressed data chunks using the next compression technique to produce one or more next compressed data chunks, a quantity of the at least one thread being equal to the next number of threads; and
routing the one or more next compressed data chunks to the data target over the network.
1 Assignment
0 Petitions
Accused Products
Abstract
Aspects of dynamic data compression selection are presented. In an example method, as uncompressed data chunks of a data stream are compressed, at least one performance factor affecting selection of one of multiple compression algorithms for the uncompressed data chunks of the data stream may be determined. Each of the multiple compression algorithms may facilitate a different expected compression ratio. One of the multiple compression algorithms may be selected separately for each uncompressed data chunk of the data stream based on the at least one performance factor. Each uncompressed data chunk may be compressed using the selected one of the multiple compression algorithms for the uncompressed data chunk.
30 Citations
20 Claims
-
1. A method for data compression, the method comprising:
-
receiving uncompressed source data at a data compression system from a data storage device, the uncompressed source data comprising a plurality of uncompressed data chunks; determining, at a first time, a first number of threads to employ to compress the uncompressed source data; identifying, based at least in part on the first number of threads, a first compression technique from a plurality of compression techniques to use to compress one or more first uncompressed data chunks of the plurality of uncompressed data chunks; compressing, via one or more threads, the one or more first uncompressed data chunks using the first compression technique to produce one or more first compressed data chunks, a first quantity of the one or more threads being equal to the first number of threads; routing the one or more first compressed data chunks to a data target over a network; and at each second time of one or more second times; determining, at the second time, a next number of threads to employ to compress the uncompressed source data, the next number of threads being different than the first number of threads; identifying, based at least in part on the next number of threads and utilizing one or more hardware processors of the data compression system, a next compression technique from the plurality of compression techniques to use to compress one or more next uncompressed data chunks of the plurality of uncompressed data chunks, wherein, for each of at least one second time of the one or more second times, the next compression technique is different than the first compression technique; compressing, via at least one thread, the one or more next uncompressed data chunks of the plurality of uncompressed data chunks using the next compression technique to produce one or more next compressed data chunks, a quantity of the at least one thread being equal to the next number of threads; and routing the one or more next compressed data chunks to the data target over the network. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform actions including:
-
receiving uncompressed source data from a data storage device, the uncompressed source data comprising a plurality of uncompressed data chunks; determining, at a first time, a first number of threads to employ to compress the uncompressed source data; identifying, based at least in part on the first number of threads, a first compression technique from a plurality of compression techniques to use to compress one or more first uncompressed data chunks of the plurality of uncompressed data chunks; compressing, via one or more threads, the one or more first uncompressed data chunks using the first compression technique to produce one or more first compressed data chunks, a first quantity of the one or more threads being equal to the first number of threads; routing the one or more first compressed data chunks to a data target over a network; and at each second time of one or more second times; determining, at the second time, a next number of threads to employ to compress the uncompressed source data, the next number of threads being different than the first number of threads; identifying, based at least in part on the next number of threads, a next compression technique from the plurality of compression techniques to use to compress one or more next uncompressed data chunks of the plurality of uncompressed data chunks, wherein, for each of at least one second time of the one or more second times, the next compression technique is different than the first compression technique; compressing, via at least one thread, the one or more next uncompressed data chunks of the plurality of uncompressed data chunks using the next compression technique to produce one or more next compressed data chunks, a quantity of the at least one thread being equal to the next number of threads; and routing the one or more next compressed data chunks to the data target over the network. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for data compression including:
-
one or more data processors; and a non-transitory computer readable storage medium containing instructions which when executed on the one or more data processors, cause the one or more data processors to perform actions including; receiving uncompressed source data from a data storage device, the uncompressed source data comprising a plurality of uncompressed data chunks; determining, at a first time, a first number of threads to employ to compress the uncompressed source data; identifying, based at least in part on the first number of, a first compression technique from a plurality of compression techniques to use to compress one or more first uncompressed data chunks of the plurality of uncompressed data chunks; compressing, via one or more threads, the one or more first uncompressed data chunks using the first compression technique to produce one or more first compressed data chunks, a first quantity of the one or more threads being equal to the first number of threads; routing the one or more first compressed data chunks to a data target over a network; and at each second time of one or more second times; determining, at the second time, a next number of threads to employ to compress the uncompressed source data, the next number of threads being different than the first number of threads; identifying, based at least in part on the next number of threads, a next compression technique from the plurality of compression techniques to use to compress one or more next uncompressed data chunks of the plurality of uncompressed data chunks, wherein, for each of at least one second time of the one or more second times, the next compression technique is different than the first compression technique; compressing, via at least one thread, the one or more next uncompressed data chunks of the plurality of uncompressed data chunks using the next compression technique to produce one or more next compressed data chunks, a quantity of the at least one thread being equal to the next number of threads; and routing the one or more next compressed data chunks to the data target over the network. - View Dependent Claims (20)
-
Specification