Memory Allocation in a Data Analytics System

• US 20200133504A1
• Filed: 12/31/2019
• Published: 04/30/2020
• Est. Priority Date: 10/16/2017
• Status: Active Grant

• Current Assignee: Alteryx, Inc.
• Original Assignee: Alteryx, Inc.
• Inventors: Harding, JR., Edward P., Riley, Adam David, Kingsley, Christopher H.
• Granted Patent: US 10,996,855 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A module manages memory in a computer. The module monitors usage of a primary memory associated with the computer. The primary memory stores memory blocks in a ready state. In response to primary memory usage by the memory blocks in the ready state exceeding a ready state threshold, the module compresses at least some of the memory blocks in the ready state to form memory blocks in a ready and compressed state. In response to primary memory usage by the memory blocks in the ready and compressed state exceeding a release threshold, the module releases at least some of the memory blocks in the ready and compressed state. In response to primary memory usage by the memory blocks in the compressed state exceeding a compressed threshold, the module transfers at least some memory blocks in the compressed state to a secondary memory associated with the computer.

Claims

1. A computer-implemented method of managing memory in a computer comprising:

storing uncompressed memory blocks in a memory;

in response to an amount of the memory occupied by the uncompressed memory blocks exceeding a first threshold,

compressing at least some of the uncompressed memory blocks to form compressed memory blocks,

storing the at least some of the uncompressed memory blocks as memory blocks in a ready and compressed state in the memory, and

storing the compressed blocks as memory blocks in a compressed state in the memory; and

in response to an amount of the memory occupied by the memory blocks in the ready and compressed state exceeding a second threshold, releasing at least some of the memory blocks in the ready and compressed state from the memory.

2. The method of claim 1, further comprising:

in response to an amount of the memory occupied by the memory blocks in the compressed state exceeding a third threshold, transferring at least some of the memory blocks in the compressed state to a secondary memory.

3. The method of claim 2, further comprising:

in response to an occupied amount of the memory falling below a fourth threshold, transferring at least some memory blocks in the secondary memory from the secondary memory to the memory.

4. The method of claim 3, further comprising:

retaining the transferred memory blocks in the secondary memory; and

in response to an occupied amount of the memory exceeding the third threshold, releasing the transferred memory blocks from the memory.

5. The method of claim 1, wherein storing uncompressed memory blocks in a memory comprises storing the uncompressed memory blocks in the memory in a ready state, further comprising:

receiving a request from a module executing on the computer, the request seeking to access a memory block;

determining whether the requested memory block is in the ready state; and

responsive to a determination that the requested memory block is not in the ready state, changing the requested memory block into the ready state; and

serving the requested memory block in the ready state in response to the access request.

6. The method of claim 1, further comprising:

receiving a write request from a module executing on the computer, the write request seeking to access a memory block in the ready and compressed state;

changing the memory block into a ready state; and

releasing a compressed memory block in the compressed state that was formed by compressing the memory block.

7. The method of claim 1, further comprising:

receiving a read request from a module executing on the computer to access a memory block in the ready and compressed state;

serving the read request using the memory block in the ready and compressed state; and

maintaining the memory block in the ready and compressed state.

8. A computer system for managing memory in a computer, the system comprising:

a non-transitory computer-readable storage medium storing executable computer program instructions, the computer program instructions comprising instructions that when executed cause a computer processor to perform steps, comprising;

storing uncompressed memory blocks in a memory;

in response to an amount of the memory occupied by the uncompressed memory blocks exceeding a first threshold,

compressing at least some of the uncompressed memory blocks to form compressed memory blocks,

storing the at least some of the uncompressed memory blocks as memory blocks in a ready and compressed state in the memory, and

storing the compressed blocks as memory blocks in a compressed state in the memory; and

in response to an amount of the memory occupied by the memory blocks in the ready and compressed state exceeding a second threshold, releasing at least some of the memory blocks in the ready and compressed state from the memory.

9. The system of claim 8, wherein the steps performed by the computer processor further comprise:

in response to an amount of the memory occupied by the memory blocks in the compressed state exceeding a third threshold, transferring at least some of the memory blocks in the compressed state to a secondary memory.

10. The system of claim 9, wherein the steps performed by the computer processor further comprise:

in response to an occupied amount of the memory falling below a fourth threshold, transferring at least some memory blocks in the secondary memory from the secondary memory to the memory.

11. The system of claim 10, wherein the steps performed by the computer processor further comprise:

retaining the transferred memory blocks in the secondary memory; and

in response to an occupied amount of the memory exceeding the third threshold, releasing the transferred memory blocks from the memory.

12. The system of claim 8, wherein storing uncompressed memory blocks in a memory comprises storing the uncompressed memory blocks in the memory in a ready state, wherein the steps performed by the computer processor further comprise:

receiving a request from a module executing on the computer, the request seeking to access a memory block;

determining whether the requested memory block is in the ready state; and

responsive to a determination that the requested memory block is not in the ready state, changing the requested memory block into the ready state; and

serving the requested memory block in the ready state in response to the access request.

13. The system of claim 8, wherein the steps performed by the computer processor further comprise:

receiving a write request from a module executing on the computer, the write request seeking to access a memory block in the ready and compressed state;

changing the memory block into the ready state; and

releasing a compressed memory block in the compressed state that was formed by compressing the memory block.

14. The system of claim 8, wherein the steps performed by the computer processor further comprise:

receiving a read request from a module executing on the computer to access a memory block in the ready and compressed state;

serving the read request using the memory block in the ready and compressed state; and

maintaining the memory block in the ready and compressed state.

15. A non-transitory computer readable storage medium storing executable computer program instructions for managing memory in a computer, the computer program instructions comprising instructions that when executed cause a computer processor to:

store uncompressed memory blocks in a memory;

in response to an amount of the memory occupied by the uncompressed memory blocks exceeding a first threshold,

compress at least some of the uncompressed memory blocks to form compressed memory blocks,

store the at least some of the uncompressed memory blocks as memory blocks in a ready and compressed state in the memory, and

store the compressed blocks as memory blocks in a compressed state in the memory; and

in response to an amount of the memory occupied by the memory blocks in the ready and compressed state exceeding a second threshold, release at least some of the memory blocks in the ready and compressed state from the memory.

16. The computer readable medium of claim 15, wherein the computer program instructions for managing memory in the computer further comprise instructions that when executed cause the computer processor to:

in response to an amount of the memory occupied by the memory blocks in the compressed state exceeding a third threshold, transfer at least some of the memory blocks in the compressed state to a secondary memory.

17. The computer readable medium of claim 16, wherein the computer program instructions for managing memory in the computer further comprise instructions that when executed cause the computer processor to:

in response to an occupied amount of the memory falling below a fourth threshold, transfer at least some memory blocks in the secondary memory from the secondary memory to the memory.

18. The computer readable medium of claim 17, wherein the computer program instructions for managing memory in the computer further comprise instructions that when executed cause the computer processor to:

retain the transferred memory blocks in the secondary memory; and

in response to an occupied amount of the memory exceeding the third threshold, release the transferred memory blocks from the memory.

19. The computer readable medium of claim 15, wherein the computer program instructions for storing uncompressed memory blocks in a memory comprise instructions that when executed cause the computer processor to store the uncompressed memory blocks in the memory in a ready state, wherein the computer program instructions for managing memory in the computer further comprise instructions that when executed cause the computer processor to:

receive a request from a module executing on the computer, the request seeking to access a memory block;

determine whether the requested memory block is in the ready state; and

responsive to a determination that the requested memory block is not in the ready state, change the requested memory block into the ready state; and

serve the requested memory block in the ready state in response to the access request.

20. The computer readable medium of claim 15, wherein the computer program instructions for managing memory in the computer further comprise instructions that when executed cause the computer processor to:

receive a write request from a module executing on the computer, the write request seeking to access a memory block in the ready and compressed state;

change the memory block into the ready state; and

release a compressed memory block in the compressed state that was formed by compressing the memory block.