Systems and methods for storing time-series data

• US 9,195,700 B1
• Filed: 01/30/2013
• Issued: 11/24/2015
• Est. Priority Date: 10/10/2007
• Status: Active Grant

• Current Assignee: United States Automobile Association (United Services Automobile Association)
• Original Assignee: United Services Automobile Association
• Inventors: Becker, Glen A.
• Primary Examiner(s): TRUONG, CAM Y T
• Application Number:
  US13/754,190
• Time in Patent Office: 1,028 Days
• Field of Search: 707/752, 707/748
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/24565 Triggers; Constraints
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

In various embodiments, a method for maintaining and analyzing a rolling history of performance data for a computer network is described. In one embodiment, one year'"'"'s data is stored, sorted by object and date. New data is appended to the end of the file each day and an index by object is stored. Using this method, data for twelve full months can be provided in object order, and data for one partial month that was added daily in date order can also be provided.

Claims

1. A system for merging data, the system comprising:

a non-transitory machine-readable storage memory comprising a first set of memory locations and a second set of memory locations,

the first set of memory locations of the memory storing sorted-data sequentially, the sorted-data being sorted according to one or more object types associated with the sorted-data and then sorted according to a time period of the sorted-data was received from one or more servers,

the second set of memory locations of the memory storing new-data received from the one or more servers and appended sequentially to the first set of memory locations of the memory, the new-data sorted according to the time period of the new-data was received from the one or more servers and then according to the one or more object types associated with the new-data;

an administrative computer comprising non-transitory machine-readable storage memory storing an index for the new-data, the index referencing the new-data according to the one or more object types associated with the new-data; and

a processor configured to re-sort the new-data stored in the second set of memory locations into second sorted-data, the second sorted-data sorted according to the one or more object types associated with the new-data and then according to the time period when the new-data was received,

wherein the processor is further configured to merge the second sorted-data of the second set of memory locations and the sorted-data in the first set of memory locations, thereby generating merged-data being sorted according to the time period of the merged-data was received and then according to the one or more object types associated with the merged-data, wherein the new-data and the sorted-data are merged when processing resources are below a threshold value, wherein the sorted-data corresponds to at least one server performance parameter regarding performance of at least one server computer.

2. The system of claim 1 wherein the time period is a day.

3. The system of claim 1 wherein the second set of memory locations containing the new-data is appended to the first set of memory locations every one month.

4. The system of claim 1, wherein the merged-data is stored in a combination of the first and second sets of memory locations for one year.

5. The system of claim 1, wherein the first set of memory locations is stored by Network Attached Storage.

6. The system of claim 1, wherein the first set of memory locations is stored by Direct Attached Storage.

7. The system of claim 1, wherein the first memory is stored by a Storage Area Network.

8. A method for merging data, the method comprising:

storing sequentially, by a first set of memory locations of a non-transitory machine-readable storage memory, sorted-data in locations of the first set of memory locations, the sorted-data being sorted according to one or more object types associated with the sorted-data and then according to a time period of the sorted-data was received from one or more servers;

receiving, by a second set of memory locations of the non-transitory machine-readable storage memory, new-data from the one or more server computers, the new-data being appended sequentially to the memory locations of the first set of memory locations, the new-data sorted according to the time period of the new-data was received from the one or more servers and then according to the one or more object types associated with the new-data;

storing, by a second memory, an index for an object type associated with the new-data, the index referencing the new-data according to the one or more object types associated with the new-data;

re-sorting, by the second memory, the new-data into second sorted-data, the second sorted-data being sorted according to the object type associated with the new-data and the time period of the new-data was received from the one or more servers; and

merging, by the first memory, the second sorted-data in the second set of memory locations and the sorted-data in the first set of memory locations, thereby generating merged-data sorted according to the time period of the merged-data was received and then according to the object type associated with the merged-data, wherein the new-data and the sorted-data are merged when processing resources are below a threshold value, wherein the sorted-data corresponds to at least one server performance parameter regarding performance of at least one server computer.

9. The method of claim 8 wherein the sorted-data is customer activity data.

10. The method of claim 8 wherein the sorted-data is sales data.

11. The method of claim 8 wherein the sorted-data corresponds to a plurality of measurements.

12. The method of claim 8 wherein the sorted-data corresponds to census data.

13. The method of claim 8 wherein the time period is greater than a day.

14. A non-volatile computer readable storage medium having instructions stored thereon that are executed by a processor to:

store sorted-data sequentially in a first set of memory locations of a non-transitory machine-readable storage memory, the sorted-data being sorted according to one or more object types associated with the sorted-data and then according to a time period of the sorted-data was received from one or more servers;

receive new-data from the one or more server computers, the new-data being appended sequentially to memory locations in a second set of memory locations of the non-transitory machine-readable storage memory, the new-data sorted according to the time period of the new-data was received from the one or more servers and then according to the one or more object types associated with the new-data;

store an index for the object type associated with the new-data, the index referencing the new-data according to the one or more object types associated with the new-data;

re-sort the new-data into second sorted-data, the second sorted data being sorted according to the time period of the new-data was received from the one or more servers and then according to the object type associated with the new-data; and

merge the second sorted data in the second set of memory locations and the sorted-data in the first set of memory locations, thereby generating merged-data sorted according to the time period of the merged-data was received from the one or more servers, and then according to the object type associated with the merged-data, wherein the new-data and the sorted-data are merged when processing resources are below a threshold value, wherein the sorted-data corresponds to at least one server performance parameter regarding performance of at least one server computer.

15. The non-volatile computer readable storage medium of claim 14 wherein the time period is less than a day.

16. The non-volatile computer readable storage medium of claim 14 wherein the sorted-data being sorted by the time period are sorted by date.

17. The non-volatile computer readable storage medium of claim 14 wherein the new-data being appended sequentially to the file is indexed.

18. The non-volatile computer readable storage medium of claim 17 wherein the index is accessed for retrieval of the new-data.