Data set management using transient data structures

• US 9,305,036 B2
• Filed: 03/27/2014
• Issued: 04/05/2016
• Est. Priority Date: 03/27/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ramos, Reinaldo G.
• Primary Examiner(s): Nguyen, Cam-Linh
• Application Number:
  US14/226,887
• Publication Number:
  US 20150278260A1
• Time in Patent Office: 740 Days
• Field of Search: 707/609, 707/802
• US Class Current: 1/1
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27

Abstract

A data set is stored in a linear data structure. A memory manager receives multiple change requests for the data set. In response to receiving, for example, a read request for the data set, the memory manager batch processes the change requests by first moving the data set into a non-linear data structure. The non-linear data structure consumes more memory resources than the linear data structure but may provide for faster processing of the change requests while eliminating duplicate data elements. The updated data set is then returned to the more memory-efficient linear data structure.

Claims

1. A method for managing a data set stored in a linear data structure, the data set in the linear data structure consuming first computer memory resources, the method comprising:

receiving a plurality of change requests for the data set;

determining that batch processing criteria for processing the plurality of change requests is satisfied;

storing, in response to the determining that the batch processing criteria is satisfied, the data set in a non-linear data structure, the data set in the non-linear data structure consuming second computer memory resources, the second computer memory resources greater than the first computer memory resources;

processing the plurality of change requests using the non-linear data structure, the processing resulting in an updated data set, the updated data set in the non-linear data structure consuming third computer memory resources; and

storing the updated data set in the linear data structure, the updated data set in the linear data structure consuming fourth computer memory resources, the fourth computer memory resources less than the third computer memory resources.

2. The method of claim 1, further comprising:

releasing the third computer memory resources after storing the updated data set in the linear data structure.

3. The method of claim 1, further comprising:

releasing the first computer memory resources after storing the data set in the non-linear data structure.

4. The method of claim 1, wherein the linear data structure is selected from the group consisting of an array and a list, and wherein the non-linear data structure is selected from the group consisting of a set, a tree, and a map.

5. The method of claim 1, wherein the batch processing criteria is at least one of a read request received for the data set, a maximum number of change requests received for the data set, and a timer expiration.

6. The method of claim 1, further comprising:

storing the received plurality of change requests in a secondary data structure, wherein the processing the plurality of change requests includes retrieving the plurality of received change requests from the secondary data structure.

7. The method of claim 6, wherein an entry in the secondary data structure comprises an operation and at least one data element, and wherein the operation is selected from the group consisting of an add operation, a remove operation, and an intersect operation.

8. A computer program product for managing a data set stored in a linear data structure, the data set in the linear data structure consuming first computer memory resources, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to perform a method comprising:

receiving a plurality of change requests for the data set;

determining that batch processing criteria for processing the plurality of change requests is satisfied;

storing, in response to the determining that the batch processing criteria is satisfied, the data set in a non-linear data structure, the data set in the non-linear data structure consuming second computer memory resources, the second computer memory resources greater than the first computer memory resources;

processing the plurality of change requests using the non-linear data structure, the processing resulting in an updated data set, the updated data set in the non-linear data structure consuming third computer memory resources; and

storing the updated data set in the linear data structure, the updated data set in the linear data structure consuming fourth computer memory resources, the fourth computer memory resources less than the third computer memory resources.

9. The computer program product of claim 8, wherein the method further comprises:

releasing the third computer memory resources after storing the updated data set in the linear data structure.

10. The computer program product of claim 8, wherein the method further comprises:

releasing the first computer memory resources after storing the data set in the non-linear data structure.

11. The computer program product of claim 8, wherein the linear data structure is selected from the group consisting of an array and a list, and wherein the non-linear data structure is selected from the group consisting of a set, a tree, and a map.

12. The computer program product of claim 8, wherein the batch processing criteria is at least one of a read request received for the data set, a maximum number of change requests received for the data set, and a timer expiration.

13. The computer program product of claim 8, wherein the method further comprises:

storing the received plurality of change requests in a secondary data structure, wherein the processing the plurality of change requests includes retrieving the plurality of received change requests from the secondary data structure.

14. The computer program product of claim 13, wherein an entry in the secondary data structure comprises an operation and at least one data element, and wherein the operation is selected from the group consisting of an add operation, a remove operation, and an intersect operation.

15. A system for managing a data set, the system comprising:

a linear data structure having a first data density;

a non-linear data structure having a second data density lower than the first data density; and

a memory manager configured to;

receive a plurality of change requests for the data set, the data set stored in the linear data structure, the data set in the linear data structure consuming first computer memory resources;

determine that batch processing criteria for processing the plurality of change requests is satisfied;

store, in response to the determining that the batch processing criteria is satisfied, the data set in the non-linear data structure, the data set in the non-linear data structure consuming second computer memory resources, the second computer memory resources greater than the first computer memory resources;

process the plurality of change requests using the non-linear data structure, the processing resulting in an updated data set, the updated data set in the non-linear data structure consuming third computer memory resources; and

store the updated data set in the linear data structure, the updated data set in the linear data structure consuming fourth computer memory resources, the fourth computer memory resources less than the third computer memory resources.

16. The system of claim 15, wherein the memory manager is further configured to release the third computer memory resources after storing the updated data set in the linear data structure.

17. The system of claim 15, wherein the linear data structure is selected from the group consisting of an array and a list, and wherein the non-linear data structure is selected from the group consisting of a set, a tree, and a map.

18. The system of claim 15, wherein the batch processing criteria is at least one of a read request received for the data set, a maximum number of change requests received for the data set, and a timer expiration.

19. The system of claim 15, further comprising:

a secondary data structure, wherein the memory manager is further configured to store the received plurality of change requests in the secondary data structure, and wherein the processing the plurality of change requests includes retrieving the plurality of received change requests from the secondary data structure.

20. The system of claim 19, wherein an entry in the secondary data structure comprises an operation and at least one data element, and wherein the operation is selected from the group consisting of an add operation, a remove operation, and an intersect operation.