Selecting a compression technique

• US 9,141,632 B1
• Filed: 12/19/2012
• Issued: 09/22/2015
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Teradata US, Inc. (Teradata Corporation)
• Original Assignee: Teradata US, Inc. (Teradata Corporation)
• Inventors: Ramesh, Bhashyam
• Primary Examiner(s): Vital, Pierre
• Assistant Examiner(s): Sultana, Nargis
• Application Number:
  US13/719,390
• Time in Patent Office: 1,007 Days
• Field of Search: 707/693
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files

Abstract

Demographics for data are determined. A compression ratio (“CR”) is determined for each of a plurality of compression techniques. CR is a size of the data before compression divided into a predicted size of the data after compression. The predicted size of the data after compression is determined as a function of the determined demographics. An access efficiency of each of the compression techniques is determined as a function of the determined demographics. The compression techniques are ranked by CR and access efficiency. A compression technique is selected based on the ranking. The data is compressed using the selected compression technique. The compressed data is stored.

Claims

1. A method comprising:

determining demographics for data;

determining a compression ratio (“

CR”

) of each of a plurality of compression techniques, wherein CR is a size of the data before compression divided into a predicted size of the data after compression, wherein the predicted size of the data after compression is determined as a function of the determined demographics;

determining an access efficiency of each of the compression techniques as a function of the determined demographics;

ranking the compression techniques by CR and access efficiency;

selecting a compression technique based on the ranking;

compressing the data using the selected compression technique; and

storing the compressed data;

wherein each of the plurality of compression techniques stores data using;

a information data structure that specifies information about the data, and

a value data structure that stores the values of the data;

wherein access efficiency is defined to have four categories;

a first category in which;

the information data structure is accessed directly, and

the value data structure is accessed directly,

a second category in which;

the information data structure is accessed directly, and

the value data structure is accessed sequentially,

a third category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed directly, and

a fourth category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed sequentially.

2. The method of claim 1, wherein:

the first category is ranked higher than the other categories,

the second and third categories are ranked equally, and

the fourth category is ranked lower than the other categories.

3. The method of claim 2 wherein ranking the compression techniques by CR and access efficiency comprises:

assigning each of the plurality of compression techniques to one of the access efficiency categories by information data structure access technique and value data structure access technique; and

ranking the compression techniques within each category by CR.

4. The method of claim 3 wherein selecting a compression technique based on the ranking comprises:

selecting the highest-ranked compression technique within each category;

ranking the selected compression techniques by category and CR; and

selecting the highest-ranked compression technique.

5. The method of claim 4 wherein ranking the selected compression techniques by category and CR comprises:

adjusting the CR associated with a first category by a preference amount when comparing it with the CR of a second category.

6. The method of claim 2 wherein:

access efficiency is defined to have additional categories depending on;

the data type of the data in the value store; and

the access technique for the value data store for the compression technique being categorized.

7. A database system comprising:

one or more nodes;

a plurality of CPUs, each of the one or more nodes providing access to one or more CPUs;

a plurality of virtual processes, each of the one or more CPUs providing access to one or more virtual processes;

each virtual process configured to manage data, including rows from the set of database table rows, stored in one of a plurality of data-storage facilities;

a process to;

determine demographics for data;

determine a compression ratio (“

CR”

) of each of a plurality of compression techniques, wherein CR is a size of the data before compression divided into a predicted size of the data after compression, wherein the predicted size of the data after compression is determined as a function of the determined demographics;

determine an access efficiency of each of the compression techniques as a function of the determined demographics;

rank the compression techniques by CR and access efficiency;

select a compression technique based on the ranking;

compress the data using the selected compression technique; and

store the compressed data;

wherein each of the plurality of compression techniques stores data using;

a information data structure that specifies information about the data, and

a value data structure that stores the values of the data;

wherein access efficiency is defined to have four categories;

a first category in which;

the information data structure is accessed directly, and

the value data structure is accessed directly,

a second category in which;

the information data structure is accessed directly, and

the value data structure is accessed sequentially,

a third category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed directly, and

a fourth category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed sequentially.

8. The database system of claim 7, wherein:

the first category is ranked higher than the other categories,

the second and third categories are ranked equally, and

the fourth category is ranked lower than the other categories.

9. The database system of claim 8 wherein ranking the compression techniques by CR and access efficiency comprises:

assigning each of the plurality of compression techniques to one of the access efficiency categories by information data structure access technique and value data structure access technique; and

ranking the compression techniques within each category by CR.

10. The database system of claim 9 wherein selecting a compression technique based on the ranking comprises:

selecting the highest-ranked compression technique within each category;

ranking the selected compression techniques by category and CR; and

selecting the highest-ranked compression technique.

11. The database system of claim 10 wherein ranking the selected compression techniques by category and CR comprises:

adjusting the CR associated with a first category by a preference amount when comparing it with the CR of a second category.

12. The database system of claim 8 wherein:

access efficiency is defined to have additional categories depending on;

the data type of the data in the value store; and

the access technique for the value data store for the compression technique being categorized.

13. A computer program stored in a non-transitory computer readable storage medium, the program comprising executable instructions that cause a computer to:

determine demographics for data;

determine a compression ratio (“

CR”

) of each of a plurality of compression techniques,

wherein CR is a size of the data before compression divided into a predicted size of the data after compression, wherein the predicted size of the data after compression is determined as a function of the determined demographics;

determine an access efficiency of each of the compression techniques as a function of the determined demographics;

rank the compression techniques by CR and access efficiency;

select a compression technique based on the ranking;

compress the data using the selected compression technique; and

store the compressed data;

wherein each of the plurality of compression techniques stores data using;

a information data structure that specifies information about the data, and

a value data structure that stores the values of the data;

wherein access efficiency is defined to have four categories;

a first category in which;

the information data structure is accessed directly, and

the value data structure is accessed directly,

a second category in which;

the information data structure is accessed directly, and

the value data structure is accessed sequentially,

a third category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed directly, and

a fourth category in which;

the information data structure is accessed sequentially, and

the value data structure is accessed sequentially.

14. The computer program of claim 13, wherein:

the first category is ranked higher than the other categories,

the second and third categories are ranked equally, and

the fourth category is ranked lower than the other categories.

15. The computer program of claim 14 wherein, when ranking the compression techniques by CR and access efficiency, the computer:

assigns each of the plurality of compression techniques to one of the access efficiency categories by information data structure access technique and value data structure access technique; and

ranks the compression techniques within each category by CR.

16. The computer program of claim 15 wherein, when selecting a compression technique based on the ranking, the computer:

selects the highest-ranked compression technique within each category;

ranks the selected compression techniques by category and CR; and

selects the highest-ranked compression technique.

17. The computer program of claim 16 wherein, when ranking the selected compression techniques by category and CR, the computer:

adjusts the CR associated with a first category by a preference amount when comparing it with the CR of a second category.

18. The computer program of claim 14 wherein:

access efficiency is defined to have additional categories depending on;

the data type of the data in the value store; and

the access technique for the value data store for the compression technique being categorized.