Memory management system and method for storing and retrieving messages

• US 20070113031A1
• Filed: 11/16/2005
• Published: 05/17/2007
• Est. Priority Date: 11/16/2005
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Kevin, Spicer, Michael
• Application Number:
  US11/274,154
• Publication Number:
  US 20070113031A1
• Time in Patent Office: Days
• US Class Current: 711/160
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/123 with age lists, e.g. queue, most recently used [MRU] list or least recently used [LRU] list

Abstract

Embodiments of the present invention provide an efficient manner to systematically remove data from a memory that has been transferred or copied to disk storage, thereby facilitating faster querying of data residing in the memory. In particular, memory containing data received from data sources is partitioned into a fixed quantity of buckets each associated with a respective time interval. The buckets represent contiguous intervals of time, where each interval is preferably of the same duration. When data arrives, the data is associated with a timestamp and placed in the appropriate bucket associated with a time interval corresponding to that timestamp. If a timestamp falls outside the range of time intervals associated with the buckets, the data corresponding to that timestamp is placed in an additional bucket. Data within the oldest bucket in memory is periodically removed to provide storage capacity for new incoming information.

Claims

1. A system for managing data stored in a memory comprising:

a memory unit including a storage area partitioned into a plurality of storage sections each associated with a corresponding time interval;

a processing system to receive data items from at least one data source and to store and manage said data items within said memory unit, wherein said data items are each associated with a corresponding time indication and said processing system includes;

a memory store module to store each data item in a storage section associated with a time interval corresponding to said associated time indication of that data item; and

a memory purge module to remove data items from said storage section associated with the oldest time interval in response to expiration of a predetermined time period.

2. The system of claim 1 further including:

at least one disk storage unit, wherein said processing system further includes;

a disk manager module to store data items from said memory unit in at least one disk storage unit in response to expiration of a predetermined time period.

3. The system of claim 2, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and said disk manager module includes:

a location module to determine a database server to receive a data item from said memory unit for storage in said associated database; and

a format module to format said data item in accordance with said determined database and to send said formatted data item to said determined database server for storage in said associated database.

4. The system of claim 2, wherein said processing system further includes:

a query module to receive and process a query from an end-user system in communication with said processing system and to provide data items from at least one of said memory unit and said at least one disk storage unit satisfying said query.

5. The system of claim 4, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and each database server includes:

a forward module to receive a query from an end-user system in communication with that database server and forward said query to said query module for processing.

6. The system of claim 4, wherein said query includes a time range qualification and said query module includes:

a query memory module to examine particular storage sections within said memory unit associated with time intervals satisfying said time range and to collect data items from said examined storage sections satisfying said query.

7. The system of claim 1, wherein:

said memory unit includes a plurality of said storage areas each partitioned into said plurality of storage sections with each section associated with a time interval;

said memory store module stores each data item in a storage section of one of said storage areas associated with a time interval corresponding to said associated time indication of that data item, wherein said one storage area is associated with a characteristic of that data item; and

said memory purge module removes data items from a storage section associated with the oldest time interval within at least one storage area in response to expiration of said predetermined time period to provide storage capacity within said memory unit for new data items.

8. The system of claim 1, wherein said memory unit further includes at least one memory area, and wherein:

said memory store module stores data items associated with time indications outside a range of said time intervals associated with said storage sections within said at least one memory area; and

said memory purge module removes data items from said at least one memory area in response to expiration of said predetermined time period.

9. The system of claim 1, wherein said memory unit further includes a plurality of said storage areas with at least one storage area utilized as a reserve storage area, and wherein:

said memory store module stores each newly received data item in a storage section of a reserve storage area associated with a time interval corresponding to said associated time indication of that data item in response to said memory purge module removing data items from a storage section of another storage area.

10. A method of managing data stored in a memory unit coupled to a processing system and including a storage area partitioned into a plurality of storage sections each associated with a corresponding time interval, said method comprising:

receiving data items from at least one data source, wherein said data items are each associated with a corresponding time indication;

storing, via said processing system, each data item in a storage section associated with a time interval corresponding to said associated time indication of that data item; and

removing, via said processing system, data items from said storage section associated with the oldest time interval in response to expiration of a predetermined time period.

11. The method of claim 10 further including:

storing, via said processing system, data items from said memory unit in at least one disk storage unit in response to expiration of a predetermined time period.

12. The method of claim 11, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and said storing data items in at least one disk storage unit includes:

determining a database server to receive a data item from said memory unit for storage in said associated database; and

formatting said data item in accordance with said determined database and sending said formatted data item to said determined database server for storage in said associated database.

13. The method of claim 11 further including:

receiving and processing a query from an end-user system in communication with said processing system and providing data items from at least one of said memory unit and said at least one disk storage unit satisfying said query.

14. The method of claim 13, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and said method further includes:

receiving a query at a database server from an end-user system in communication with that database server and forwarding said query to said processing system for processing.

15. The method of claim 13, wherein said query includes a time range qualification and said query processing includes:

examining particular storage sections within said memory unit associated with time intervals satisfying said time range and collecting data items from said examined storage sections satisfying said query.

16. The method of claim 10, wherein said memory unit includes a plurality of said storage areas each partitioned into said plurality of storage sections with each section associated with a time interval, and said storing each data item in a storage section includes:

storing each data item in a storage section of one of said storage areas associated with a time interval corresponding to said associated time indication of that data item, wherein said one storage area is associated with a characteristic of that data item; and

said removing data items from a storage section includes;

removing data items from a storage section associated with the oldest time interval within at least one storage area in response to expiration of said predetermined time period to provide storage capacity within said memory unit for new data items.

17. The method of claim 10, wherein said memory unit further includes at least one memory area, and said storing each data item in a storage section includes:

storing data items associated with time indications outside a range of said time intervals associated with said storage sections within said at least one memory area; and

said removing data items from a storage section includes;

removing data items from said at least one memory area in response to expiration of said predetermined time period.

18. The method of claim 10, wherein said memory unit further includes a plurality of said storage areas with at least one storage area utilized as a reserve storage area, and said storing each data item in a storage section includes:

storing each newly received data item in a storage section of a reserve storage area associated with a time interval corresponding to said associated time indication of that data item in response to removing data items from a storage section of another storage area.

19. A program product apparatus including a computer useable medium with computer program logic recorded thereon for managing data items stored in a memory unit coupled to a processing system and including a storage area partitioned into a plurality of storage sections each associated with a corresponding time interval, wherein said data items are each associated with a corresponding time indication, said program product apparatus comprising:

a memory store module to store each data item in a storage section associated with a time interval corresponding to said associated time indication of that data item; and

a memory purge module to remove data items from said storage section associated with the oldest time interval in response to expiration of a predetermined time period.

20. The apparatus of claim 19 further including:

a disk manager module to store data items from said memory unit in at least one disk storage unit in response to expiration of a predetermined time period.

21. The apparatus of claim 20, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and said disk manager module includes:

a location module to determine a database server to receive a data item from said memory unit for storage in said associated database; and

a format module to format said data item in accordance with said determined database and to send said formatted data item to said determined database server for storage in said associated database.

22. The apparatus of claim 20 further including:

a query module to receive and process a query from an end-user system in communication with said processing system and to provide data items from at least one of said memory unit and said at least one disk storage unit satisfying said query.

23. The apparatus of claim 22, wherein each disk storage unit includes a database server and an associated database with said database server in communication with said processing system, and said apparatus further includes:

a forward module for a database server to receive a query from an end-user system in communication with that database server and forward said query to said query module for processing.

24. The apparatus of claim 22, wherein said query includes a time range qualification and said query module includes:

a query memory module to examine particular storage sections within said memory unit associated with time intervals satisfying said time range and to collect data items from said examined storage sections satisfying said query.

25. The apparatus of claim 19, wherein:

said memory unit includes a plurality of said storage areas each partitioned into said plurality of storage sections with each section associated with a time interval;

said memory store module stores each data item in a storage section of one of said storage areas associated with a time interval corresponding to said associated time indication of that data item, wherein said one storage area is associated with a characteristic of that data item; and

said memory purge module removes data items from a storage section associated with the oldest time interval within at least one storage area in response to expiration of said predetermined time period to provide storage capacity within said memory unit for new data items.

26. The apparatus of claim 19, wherein said memory unit further includes at least one memory area, and wherein:

said memory store module stores data items associated with time indications outside a range of said time intervals associated with said storage sections within said at least one memory area; and

said memory purge module removes data items from said at least one memory area in response to expiration of said predetermined time period.

27. The apparatus of claim 19, wherein said memory unit further includes a plurality of said storage areas with at least one storage area utilized as a reserve storage area, and wherein:

said memory store module stores each newly received data item in a storage section of a reserve storage area associated with a time interval corresponding to said associated time indication of that data item in response to said memory purge module removing data items from a storage section of another storage area.