Data compression for reducing storage requirements in a database system

• US 8,706,727 B2
• Filed: 06/19/2009
• Issued: 04/22/2014
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: Sybase Incorporated (SAP SE)
• Original Assignee: Sybase Incorporated (SAP SE)
• Inventors: Qian, Xudong, Yang, Xinjun
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US12/488,022
• Publication Number:
  US 20100325094A1
• Time in Patent Office: 1,768 Days
• Field of Search: 707/693, 707/796, 707/999.101, 707/736
• US Class Current: 707/736
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures

Abstract

A system, method, and computer program product for reducing data storage requirements in a database system are described herein. An embodiment includes identifying at least one data candidate of fixed length data type in at least one row of database data for compression based upon a predetermined threshold level and a boundary of compression, providing at least one bit within the at least one row for an identified data candidate according to the boundary of compression, and storing the at least one row as compressed data in the database system. For compression based on a row boundary, the identified data candidates for compression include fixed length columns having lengths that do not fall below the predetermined threshold level in a row of data and the at least one bit comprises a bitmap for a length of the identified data candidates following compression. For compression based on a page boundary, the identified data candidates for compression include redundant byte string data in a page of data, the redundant byte string data including matching data across columns having lengths that do not exceed the predetermined threshold level.

Claims

1. A method for reducing data storage requirements in a database system, comprising:

identifying, by a computing device, a data candidate of column data specified to be of a fixed length data type property for compression in a row of data having an uncompressed row format of column positioning based on;

a predetermined threshold configured to identify compressible column data of fixed length data types according to data type properties; and

a boundary of compression;

providing, by the computing device, an offset bitmap for the identified data candidate within the row, according to the boundary of compression, wherein the offset bitmap indicates lengths of compressed columns of the row following compression and variable length columns of the row; and

storing, by a computing device, the row containing the offset bitmap for the identified data candidate and the identified data candidate as compressed data with a compressed row format of column positioning within a database system, wherein the compressed row format of the row is repositioned from the uncompressed row format based on the identified data candidate.

2. The method of claim 1, wherein the data candidate identified for compression comprise fixed length columns having lengths that do not fall below the predetermined threshold in a row of data for compression based on a row boundary.

3. The method of claim 2, wherein the offset bitmap includes at least one bit indicating a length of the identified data candidate following compression.

4. The method of claim 1, wherein the data candidate identified for compression comprise redundant byte string data in a page of data for compression based on a page boundary.

5. The method of claim 4, wherein the redundant byte string data comprises matching data across columns having lengths that do not exceed the predetermined threshold.

6. The method of claim 5, wherein the at least one bit includes at least one bit in a row status to indicate compression of the identified data candidate.

7. The method of claim 4, further comprising extracting the redundant byte string data into a page index.

8. The method of claim 7, wherein the page index supplements a page dictionary for the page of data.

9. The method of claim 1, wherein the boundary of compression comprises a row boundary and a page boundary.

10. The method of claim 1, wherein the predetermined threshold comprises a column length.

11. The method of claim 1, wherein the predetermined threshold sets opposing limits depending upon the boundary of compression.

12. The method of claim 1, wherein the at least one bit provided for the identified data candidate indicates length of the identified data candidate after compression.

13. The method of claim 1 further comprising:

receiving, from a requesting module, a request to access the at least one row;

decompressing the at least one row based on the request; and

transferring, to the requesting module, the decompressed at least one row based on the request, wherein the at least one row remains in a decompressed state.

14. The method of claim 1, wherein the offset bitmap includes additional bits indicating lengths of variable length columns of the at least one row following compression.

15. A system configured to reduce data storage requirements in a database system, comprising:

one or more processors; and

a data management module configured to;

identify, using the one or more processors, a data candidate of column data specified to be of a fixed length data type property for compression in a row of data having an uncompressed row format of column positioning based on;

a predetermined threshold configured to identify compressible column data of fixed length data types according to data type properties; and

a boundary of compression;

provide, using the one or more processors, an offset bitmap for the identified data candidate within the row, according to the boundary of compression, wherein the offset bitmap indicates lengths of compressed columns of the row following compression and variable length columns of the row; and

store, using the one or more processors, the row containing the offset bitmap for the identified data candidate and the identified data candidate as compressed data with a compressed row format of column positioning within a database system, wherein the compressed row format of the row is repositioned from the uncompressed row format based on the identified data candidate.

16. The system of claim 15, wherein the data candidate identified for compression comprise fixed length columns having lengths that do not fall below the predetermined threshold in a row of data for compression based on a row boundary.

17. The system of claim 16, wherein the offset bitmap includes at least one bit indicating a length of the identified data candidate following compression.

18. The system of claim 15, wherein the data candidate identified for compression comprise redundant byte string data in a page of data for compression based on a page boundary.

19. The system of claim 18, wherein the redundant byte string data comprises matching data across columns having lengths that do not exceed the predetermined threshold.

20. The system of claim 19, wherein the at least one bit includes at least one bit in a row status to indicate compression of the identified data candidate.

21. The system of claim 15, wherein the predetermined threshold comprises a column length.

22. The system of claim 15, wherein the predetermined threshold sets opposing limits depending upon the boundary of compression.

23. A computer-readable storage device having instructions stored thereon that when executed by a processor, causes the processor to perform operations comprising:

identifying a data candidate of column data specified to be of a fixed length data type property for compression in a row of data having an uncompressed row format of column positioning based on;

a predetermined threshold configured to identify compressible column data of fixed length data types according to data type properties, and a boundary of compression;

providing an offset bitmap for the identified data candidate within the row, according to the boundary of compression, wherein the offset bitmap indicates lengths of compressed columns of the row following compression and variable length columns of the row; and

storing the row containing the offset bitmap for the identified data candidate and the identified data candidate as compressed data with a compressed row format of column positioning within a database system, wherein the compressed row format of the row is repositioned from the uncompressed row format based on the identified data candidate.

24. The storage device of claim 23, wherein the data candidate identified for compression comprise fixed length columns having lengths that do not fall below the predetermined threshold in a row of data for compression based on a row boundary.

25. The storage device of claim 23, wherein the offset bitmap includes at least one bit indicating a length of the identified data candidate following compression.

26. The storage device of claim 23, wherein the redundant byte string data comprises matching data across columns having lengths that do not exceed the predetermined threshold.

27. The storage device of claim 26, further comprising extracting the redundant byte string data into a page index.

28. The storage device of claim 27, wherein the page index supplements a page dictionary for the page of data.

29. The storage device of claim 23, wherein the at least one bit includes at least one bit in a row status to indicate compression of the identified data candidate.

30. The storage device of claim 23, wherein the boundary of compression comprises a row boundary and a page boundary.

31. The storage device of claim 23, wherein the predetermined threshold comprises a column length.

32. The storage device of claim 23, wherein the predetermined threshold sets opposing limits depending upon the boundary of compression.