Efficient data compression and analysis as a service

• US 9,934,235 B2
• Filed: 06/30/2016
• Issued: 04/03/2018
• Est. Priority Date: 05/22/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gupta, Anurag Windlass
• Primary Examiner(s): Hoang, Son T
• Application Number:
  US15/199,627
• Publication Number:
  US 20160314140A1
• Time in Patent Office: 642 Days
• Field of Search: None
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3073 Time
• CPC Class Codes: H03M 7/3079 Context modeling
• CPC Class Codes: H03M 7/607 Selection between different types of compressors
• CPC Class Codes: H03M 7/6082 Selection strategies
• CPC Class Codes: H03M 7/6088 according to the data type

Abstract

Data may be efficiently analyzed and compressed as part of a data compression service. A data compression request may be received from a client indicating data to be compressed. An analysis of the data or metadata associated with the data may be performed. In at least some embodiments, this analysis may be a rules-based analysis. Some embodiments may employ one or more machine learning techniques to historical compression data to update the rules-based analysis. One or more compression techniques may be selected out of a plurality of compression techniques to be applied to the data. Data compression candidates may then be generated according to the selected compression techniques. In some embodiments, a compression service restriction may be enforced. One of the data compression candidates may be selected and sent in a response.

Claims

1. A system, comprising:

at least one hardware processor; and

a memory to store program instructions that, when executed, cause the at least one processor to implement a method comprising;

accessing historical compression data obtained for previously compressed data;

applying one or more machine learning techniques to the historical compression data to determine one or more compression selection rules;

receiving data from a client to be compressed;

analyzing the received data or metadata associated with the received data in order to select, based at least in part on the determined one or more compression selection rules, two or more compression techniques out of a plurality of different compression techniques to generate two or more compression data candidates to evaluate;

generating the two or more compression data candidates of the received data according to the selected two or more compression techniques;

selecting one of the data compression candidates to send as the compressed data; and

sending the selected data compression candidate as the compressed data to the client.

2. The system of claim 1,

wherein the method further comprises determining an ordering to apply the selected two or more compression techniques according to the one or more compression selection rules;

wherein the data compression candidates generated according to the ordering of the selected two or more compression techniques.

3. The system of claim 1, wherein the method further comprises storing one or more performance characteristics or one or more data characteristics of the two or more compression data candidates as part of the historical compression data.

4. The system of claim 1, wherein the previously compressed data is associated with the client.

5. The system of claim 1, wherein the metadata indicates a type or other one or more characteristics of the data to be compressed, and wherein the one or more machine-learning techniques are applied based, at least in part, on the type or the other one or more characteristics for the data to be compressed.

6. The system of claim 1, wherein the data is received as part of a compression request formatted according to an Application Programmer Interface (API).

7. The system of claim 6, wherein the at least one processor and the memory are implemented as part of a network-based data compression service that includes a service interface for a plurality of different clients, including the client, to obtain compressed data from the data compression service, wherein the data compression service is implemented on a network distinct from one or more client networks of the plurality of different clients, wherein the service interface comprises the API for the compression request.

8. A method, comprising:

performing, by at least one computing device;

accessing historical compression data obtained for previously compressed data;

applying one or more machine learning techniques to the historical compression data to determine one or more compression selection rules;

receiving data from a client to be compressed;

analyzing the received data or metadata associated with the received data in order to select, based at least in part on the determined one or more compression selection rules, two or more compression techniques out of a plurality of different compression techniques to generate two or more compression data candidates to evaluate;

generating the two or more compression data candidates of the received data according to the selected compression techniques;

selecting one of the data compression candidates to send as compressed data; and

sending the selected data compression candidate as the compressed data to the client.

9. The method of claim 8,

wherein the method further comprises determining an ordering to apply the selected two or more compression techniques according to the one or more compression selection rules;

wherein the two or more data compression candidates are generated according to the ordering of the selected compression techniques.

10. The method of claim 8, wherein the method further comprises storing one or more performance characteristics or one or more data characteristics of the two or more compression data candidates as part of the historical compression data.

11. The method of claim 8, wherein the previously compressed data is associated with the client.

12. The method of claim 8, wherein the metadata indicates a type or other one or more characteristics of the received data, and wherein the one or more machine-learning techniques are performed based, at least in part, on the type or the other one or more characteristics for the received data.

13. The method of claim 8, wherein the data is received as part of a compression request formatted according to an Application Programmer Interface (API).

14. The method of claim 13, wherein the accessing, the applying, the receiving, the analyzing, the generating, the selecting, and the sending are performed by a network-based data compression service that includes a service interface for a plurality of different clients, including the client, to obtain compressed data from the data compression service, wherein the data compression service is implemented on a network distinct from one or more client networks of the plurality of different clients, wherein the service interface comprises the API for the compression request.

15. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:

accessing historical compression data obtained for previously compressed data;

applying one or more machine learning techniques to the historical compression data to determine one or more compression selection rules;

receiving data from a client to be compressed;

analyzing the received data or metadata associated with the received data in order to select, based at least in part on the determined one or more compression selection rules, two or more compression techniques out of a plurality of different compression techniques to generate two or more compression data candidates to evaluate;

generating the two or more compression data candidates of the data according to the selected two or more compression techniques;

selecting one of the data compression candidates to send as the compressed data; and

sending the selected data compression candidate as the compressed data to the client.

16. The non-transitory, computer-readable storage medium of claim 15,

wherein the program instructions cause the one or more computing devices to further implement determining an ordering to apply the selected two or more compression techniques according to the one or more compression selection rules;

wherein the data compression candidates are generated according to the ordering of the selected two or more compression techniques.

17. The non-transitory, computer-readable storage medium of claim 15, wherein the program instructions cause the one or more computing devices to further implement storing one or more performance characteristics or one or more data characteristics of the two or more compression data candidates as part of the historical compression data.

18. The non-transitory, computer-readable storage medium of claim 15, wherein the previously compressed data is associated with the client.

19. The non-transitory, computer-readable storage medium of claim 15, wherein the metadata indicates a type or other one or more characteristics of the data to be compressed, and wherein the one or more machine-learning techniques are applied based, at least in part, on the type or the other one or more characteristics for the data to be compressed.

20. The non-transitory, computer-readable storage medium of claim 15, wherein the data is received as part of a compression request formatted according to an Application Programmer Interface (API), wherein the one or more computing devices are implemented as part of a network-based data compression service that includes a service interface for a plurality of different clients, including the client, to obtain compressed data from the data compression service, wherein the data compression service is implemented on a network distinct from one or more client networks of the plurality of different clients, wherein the service interface comprises the API for the compression request.