SYSTEM, METHOD, AND RECORDING MEDIUM FOR REDUCING MEMORY CONSUMPTION FOR IN-MEMORY DATA STORES

• US 20170139596A1
• Filed: 11/18/2015
• Published: 05/18/2017
• Est. Priority Date: 11/18/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hack, Michel H. T., Wu, Xingbo, Wang, Yandong, Zhang, Li, Ren, Yufei
• Granted Patent: US 10,318,199 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

A method for compressing a group of key-value pairs, the method including dividing the group of key-value pairs into a plurality of segments, creating a plurality of blocks, each block of the plurality of blocks corresponding to a segment of the plurality of segments, and compressing each block of the plurality of blocks.

Claims

1. A method for compressing a group of key-value pairs, the method comprising:

dividing the group of key-value pairs into a plurality of segments;

creating a plurality of blocks, each block of the plurality of blocks corresponding to a segment of the plurality of segments; and

compressing each block of the plurality of blocks.

2. The method of claim 1, further comprising storing the compressed blocks in a memory data storage area of a back-end server.

3. The method of claim 2, further comprising sending, from the back-end server, a compressed block of the compressed blocks to a front-end server.

4. The method of claim 2, further comprising:

sending, from the back-end server, a compressed block of the compressed blocks to a front-end server; and

decompressing the compressed block in the front-end server.

5. The method of claim 1, wherein a size of the segments dynamically changes according to a user input.

6. The method of claim 1, wherein a size of the segments dynamically changes according to a threshold value of a memory data storage area of a back-end server.

7. The method of claim 1, further comprising:

sending, from a front-end server, a first request for a key-value pair;

locating a block of the plurality of blocks that contains the key-value pair in a memory data storage area of a back-end server; and

sending, from the back-end server, the block of the plurality of blocks that contains the key-value pair to the front-end server.

8. The method of claim 7, further comprising:

verifying that the block sent from the back-end server contains the key-value pair requested in said sending the first request;

decompressing the block; and

locating the key-value pair requested in said sending the first request within the decompressed key-value pairs of the block.

9. The method of claim 1, further comprising:

decompressing a block of the plurality of blocks that contains the key-value pair;

absorbing a second request into the key-value pair; and

compressing the block of the plurality of blocks that absorbed the second request.

10. The method of claim 1, further comprising:

sending, from a front-end server, an second request for a key-value pair;

locating a block of the plurality of blocks that contains the key-value pair in a memory data storage area of a back-end server;

decompressing the block that contains the key-value pair;

absorbing the second request into the key-value pair; and

compressing the block of the plurality of blocks that absorbed the second request.

11. The method of claim 1, further comprising:

decompressing a block of the plurality of blocks to insert a new key-value pair;

inserting the new key-value pair into the block; and

splitting the block into a first block and a second block based on a size of the block after the new key-value pair is inserted.

12. The method of claim 1, further comprising:

sending a third request from a front-end server to insert a new key-value pair;

locating a block of the plurality of blocks to insert the new key-value pair in a memory data storage area of a back-end server;

decompressing the block;

inserting the new key-value pair into the block; and

compressing the block that contains the new key-value pair.

13. The method of claim 1, further comprising:

sending a third request from a front-end server to insert a new key-value pair;

locating a block of the plurality of blocks to insert the new key-value pair in a memory data storage area of a back-end server;

counting a number of times that the block is located by the locating;

decompressing the block;

inserting the new key-value pair into the block; and

if the number of times that the block is located is greater than a threshold value, creating a cache file to store the block.

14. The method of claim 1, further comprising:

sending a third request from a front-end server to insert a new key-value pair;

locating a block of the plurality of blocks to insert the new key-value pair in a memory data storage area of a back-end server; and

if the number of times that the block is located is greater than a threshold value, creating a cache file to store the block.

15. The method of claim 1, further comprising:

sending a third request from a front-end server to insert a new key-value pair;

locating a block of the plurality of blocks to insert the new key-value pair in a memory data storage area of a back-end server;

decompressing a block of the plurality of blocks;

inserting the new key-value pair into the block;

splitting the block into a first block and a second block based on a size of the block after the new key-value pair is inserted; and

if the number of times that the block is located is greater than a threshold value, creating a cache file to store the first block and the second block.

16. The method of claim 1, further comprising:

decompressing a block of the plurality of blocks to delete a key-value pair from the block;

deleting the key-value pair from the block; and

checking a size of the block; and

combining the block with a second block if the size of the block is less than a threshold value after the deleting.

17. The method of claim 1, further comprising:

sending a fourth request from a front-end server to delete a key-value pair;

locating a block of the plurality of blocks to delete the key-value pair in a memory data storage area of a back-end server;

decompressing a block of the plurality of blocks;

deleting the key-value pair from the block; and

compressing the block.

18. The method of claim 1, further comprising:

sending, from a front-end server, a fourth request to delete a key-value pair;

locating a block of the plurality of blocks to delete the key-value pair in a memory data storage area of a back-end server;

decompressing a block of the plurality of blocks;

deleting the key-value pair from the block;

checking a size of the block;

combining the block with a second block to make a new block if the size of the block is less than a threshold value after the deleting; and

compressing the new block or the block.

19. A non-transitory computer-readable recording medium recording a compression program for compressing a group of key-value pairs, the program causing a computer to perform:

dividing the group of key-value pairs into a plurality of segments;

creating a plurality of blocks, each block of the plurality of blocks corresponding to a segment of the plurality of segments; and

compressing each block of the plurality of blocks.

20. A key-value pair compression system for compressing a group of key-value pairs, the system comprising

a key-value pair dividing device configured to divide the group of key-value pairs into a plurality of segments;

a block creation device configured to create a plurality of blocks, each block of the plurality of blocks corresponding to a segment of the plurality of segments; and

a block compression device configured to compress each block of the plurality of blocks.