Memory allocation buffer for reduction of heap fragmentation
First Claim
1. A method for reducing overall delays caused by garbage collection processes due to memory fragmentation in a region server hosting a plurality of regions for a distributed database system, the region server maintaining a heap in the region server'"'"'s memory, the method comprising:
- implementing, at the region server, an in-memory store for each region, wherein the in-memory store for each region is to hod data to be written until the data reaches a predetermined size and flush the data to a storage when the predetermined size is reached;
receiving a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region;
creating, in the heap, a memory arena dedicated to the target region by allocating one space of a select size in the heap; and
storing the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous.
5 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods of a memory allocation buffer to reduce heap fragmentation. In one embodiment, the memory allocation buffer structures a memory arena dedicated to a target region that is one of a plurality of regions in a server in a database cluster such as an HBase cluster. The memory area has a chunk size (e.g., 2 MB) and an offset pointer. Data objects in write requests targeted to the region are received and inserted to the memory arena at a location specified by the offset pointer. When the memory arena is filled, a new one is allocated. When a MemStore of the target region is flushed, the entire memory arenas for the target region are freed up. This reduces heap fragmentation that is responsible for long and/or frequent garbage collection pauses.
131 Citations
17 Claims
-
1. A method for reducing overall delays caused by garbage collection processes due to memory fragmentation in a region server hosting a plurality of regions for a distributed database system, the region server maintaining a heap in the region server'"'"'s memory, the method comprising:
-
implementing, at the region server, an in-memory store for each region, wherein the in-memory store for each region is to hod data to be written until the data reaches a predetermined size and flush the data to a storage when the predetermined size is reached; receiving a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; creating, in the heap, a memory arena dedicated to the target region by allocating one space of a select size in the heap; and storing the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A non-transitory machine-readable storage medium including executable instructions, which when executed by a processor, cause the processor to:
-
implement, at a server hosting a plurality of regions for a distributed database system and maintaining a heap in the server'"'"'s memory, an in-memory store for each region, wherein the in-memory store for each region is to hold data to be written until the data reaches a predetermined size and to flush the data to a storage when the predetermined size is reached; receive a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; create, in the heap, a memory arena dedicated to the target region by allocating one space of a select size in the heap; and store the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous. - View Dependent Claims (13)
-
-
14. A server hosting a plurality of regions for a distributed database system, the server maintaining a heap in a memory of the server, the server'"'"'s memory including instructions which, upon executed by a processor of the server, instruct the server to perform:
-
implementing, at the server, an in-memory store for each region, wherein the in-memory store for each region is to hold data to be written until the data reaches a predetermined size and to flush the data to a storage when the predetermined size is reached; receiving a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; creating, in the heap, a memory arena dedicated to the target region by allocating one space of a select size in the heap; and storing the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous.
-
-
15. A method for reducing overall delays caused by garbage collection processes due to memory fragmentation in a region server hosting a plurality of regions for a distributed database system, the region server maintaining a heap in the region server'"'"'s memory, the method comprising:
-
implementing, at the region server, an in-memory store for each region, wherein the in-memory store for each region is to hold data to be written until the data reaches a predetermined size and to flush the data to a storage when the predetermined size is reached; receiving a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; locating, in the heap, a memory arena dedicated to the target region; determining that the memory arena has enough space for the data object; and storing the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous.
-
-
16. A non-transitory machine-readable storage medium including executable instructions, which when executed by a processor, cause the processor to:
-
implement, at a server hosting a plurality of regions for a distributed database system, an in-memory store for each region, wherein the in-memory store for each region is to hold data to be written until the data reaches a predetermined size and to flush the data to a storage when the predetermined size is reached; receive a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; locate, in a heap maintained in the server'"'"'s memory, a memory arena dedicated to the target region; determine that the memory arena has enough space for the data object; and store the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous.
-
-
17. A server hosting a plurality of regions for a distributed database system, the server maintaining a heap in a memory of the server, the server'"'"'s memory including instructions which, upon executed by a processor of the server, instruct the server to perform:
-
implementing, at the server, an in-memory store for each region, wherein the in-memory store for each region is to hold data to be written until the data reaches a predetermined size and to flush the data to a storage when the predetermined size is reached; receiving a write request from a client, wherein the write request includes a target region and an data object for insertion into the target region; locating, in the heap, a memory arena dedicated to the target region; determining that the memory arena has enough space for the data object; and storing the data object in a space in the memory arena at a location that is continued from a previous location such that, when an in-memory store for the target region flushes the data it holds, the space freed from the memory arena dedicated to the target region is contiguous.
-
Specification