Optimization of in-memory data grid placement

• US 10,209,908 B2
• Filed: 02/20/2017
• Issued: 02/19/2019
• Est. Priority Date: 06/06/2011
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Gaur, Nitin, Bhogal, Kulvir S., Johnson, Christopher D., Kaplinger, Todd E., Berg, Douglas C.
• Primary Examiner(s): Dudek, Jr., Edward J
• Assistant Examiner(s): Verderamo, III, Ralph A
• Application Number:
  US15/436,937
• Publication Number:
  US 20170160965A1
• Time in Patent Office: 729 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0674 Disk device
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

A method, system, and program product for optimizing distribution and availability of data partitions is disclosed. Placement logic is run for data partition distribution which optimizes a configuration of a memory data grid. A determination of a change in performance of the memory data grid after adding new members to the memory data grid is provided. Replication of data partitions in the memory data grid is minimized. Optimum data partition distribution in the memory data grid is determined.

Claims

1. A method for optimizing data partition distribution in a memory data grid which includes a plurality of data partitions, comprising:

running placement logic for optimizing configuration of the memory data grid, wherein the memory data grid includes a plurality of data partitions and a plurality of containers;

providing a performance impact of adding new members to the memory data grid for adding available space to the memory data grid;

creating one or more replica data partitions from the plurality of data partitions;

converting one of the one or more replica data partitions to a corresponding primary data partition;

determining an optimal data partition distribution in the memory data grid based on network topology, and also based on removal of a container from the memory data grid;

determining whether the memory data grid is sufficient to replicate all of the plurality of data partitions;

determining whether adding a new container to the memory data grid will cause stress to in-memory housekeeping operations;

turning a replication of the data partitions on and off based on availability in the memory data grid to replicate all of the plurality of data partitions to minimize replication of one of a plurality of data partitions in the memory data grid for stabilizing the memory data grid;

creating a plurality of policies for deciding whether synchronization of the optimal data partition distribution should be synchronous or asynchronous;

reducing invalidation traffic by placing the plurality of data partitions in the plurality of containers in the memory data grid; and

dynamically changing a data partition synchronization to an optimal data partition synchronization based on one of the policies.

2. The method of claim 1, wherein determining the optimum partition distribution in the memory data grid includes a policy driven by a service level agreement with a grid transaction response criteria.

3. The method of claim 1, wherein determining the optimal data partition distribution in the memory data grid includes a data partition distribution algorithm which ensures that primary and replica data partitions are not in a same container.

4. The method of claim 1, wherein determining the optimal data partition distribution in the memory data grid includes monitoring data partition placements and grid membership changes.

5. The method of claim 1, wherein determining the optimal data partition distribution in the memory data grid includes monitoring transaction times with a performance management interface.

6. The method of claim 1, wherein determining the optimal data partition distribution in the memory data grid includes analyzing usage patterns of the synchronized plurality of data partitions.

7. The method of claim 1, determining the optimum data partition distribution in the memory data grid includes determining whether optimal replication of the synchronized plurality of data partitions is synchronous or asynchronous.

8. A computer program product for optimizing data partition distribution in a memory data grid, the computer program product including a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code that runs placement logic for optimizing configuration of the memory data grid, wherein the memory data grid includes a plurality of data partitions and a plurality of containers;

computer readable program code that provides a performance impact of adding new members to the memory data grid for adding available space to the memory data grid;

computer readable program code that creates one or more replica data partitions from the plurality of data partitions;

computer readable program code that converts one of the one or more replica data partitions to a corresponding primary data partition;

computer readable program code that determines an optimal data partition distribution in the memory data grid based on network topology and also based on an amount of congestion in the memory data grid;

computer readable program code that determines whether the memory data grid is sufficient to replicate all of the plurality of data partitions;

computer readable program code that determines whether adding a new container to the memory data grid will cause stress to in-memory housekeeping operations;

computer readable program code that turns a replication of the data partitions on and off based on availability in the memory data grid to replicate all of the plurality of data partitions to minimize replication of one of a plurality of data partitions in the memory data grid for stabilizing the memory data grid;

computer readable program code that creates a plurality of policies for deciding whether synchronization of the optimal data partition distribution should be synchronous or asynchronous;

computer readable program code that reduces invalidation traffic by placing the plurality of data partitions in the plurality of containers in the memory data grid; and

computer readable program code that dynamically changes a data partition synchronization to an optimal data partition synchronization based on one of the policies.

9. The computer program product of claim 8, wherein determining the optimal partition distribution in the memory data grid includes a policy driven by a service level agreement with grid transaction response criteria.

10. The computer program product of claim 8, wherein determining the optimal data partition distribution in the memory data grid includes a data partition distribution algorithm which ensures that primary and replica data partitions are not in a same container.

11. The computer program product of claim 8, wherein determining the optimal data partition distribution in the memory data grid includes monitoring data partition placements and grid membership changes.

12. The computer program product of claim 8, wherein determining the optimal data partition distribution in the memory data grid includes monitoring transaction times with a performance management interface.

13. The computer program product of claim 8, wherein determining the optimal data partition distribution in the memory data grid includes analyzing usage patterns of the synchronized plurality of data partitions.

14. The computer program product of claim 8, wherein determining the optimal data partition distribution in the memory data grid includes determining whether optimal replication of the synchronized plurality of data partitions is synchronous or asynchronous.

15. A system comprising:

a computer including a computer processor for optimizing data partition distribution in a memory data grid which includes a plurality of data partitions and a plurality of containers; and

a data partition optimization engine for synchronizing the plurality of data partitions, the data partition optimization engine being operable to;

run placement logic for optimizing configuration of the memory data grid, wherein the memory data grid includes a plurality of data partitions;

provide a performance impact of adding new members to the memory data grid for adding available space to the memory data grid;

create one or more replica data partitions from the plurality of data partitions;

convert one of the one or more replica data partitions to a corresponding primary data partition;

determine an optimal data partition distribution in the memory data grid based on network topology, and also based on a number of available and stored ones of the data partitions;

determine whether the memory data grid is sufficient to replicate all of the plurality of data partitions;

determine whether adding a new container to the memory data grid will cause stress to in-memory housekeeping operations;

turn a replication of the data partitions on and off based on availability in the memory data grid to replicate all of the plurality of data partitions to minimize replication of one of a plurality of data partitions in the memory data grid for stabilizing the memory data grid;

create a plurality of policies for deciding whether synchronization of the optimal data partition distribution should be synchronous or asynchronous;

reduce invalidation traffic by placing the plurality of data partitions in the plurality of containers in the memory data grid; and

dynamically change a data partition synchronization to an optimal data partition synchronization based on one of the policies.

16. The system of claim 15, wherein determining the optimal partition distribution in the memory data grid includes a policy driven by a service level agreement with grid transaction response criteria.

17. The system of claim 15, wherein determining the optimal data partition distribution in the memory data grid includes a data partition distribution algorithm which ensures that primary and replica data partitions are not in a same container.

18. The system of claim 15, wherein determining the optimal data partition distribution in the memory data grid includes monitoring data partition placements and grid membership changes.

19. The system of claim 15, wherein determining the optimal data partition distribution in the memory data grid includes monitoring transaction times with a performance management interface.

20. The system of claim 15, wherein determining the optimal data partition distribution in the memory data grid includes analyzing usage patterns of the synchronized plurality of data partitions.

21. The system of claim 15, determining the optimal data partition distribution in the memory data grid includes determining whether optimal replication of the synchronized plurality of data partitions is synchronous or asynchronous.