SYSTEMS, METHODS AND DEVICES FOR IMPLEMENTING DATA MANAGEMENT IN A DISTRIBUTED DATA STORAGE SYSTEM

• US 20150106578A1
• Filed: 10/15/2014
• Published: 04/16/2015
• Est. Priority Date: 10/15/2013
• Status: Active Application

• Current Assignee: Open Invention Network LLC
• Original Assignee: Coho Data, Inc.
• Inventors: Warfield, Andrew, Wires, Jacob Taylor, Ingram, Stephen Frowe
• Application Number:
  US14/514,729
• Publication Number:
  US 20150106578A1
• Time in Patent Office: Days
• US Class Current: 711/158
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/321 Display for diagnostics, e.g. diagnostic result display, self-test user interface
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Systems, methods and devices for monitoring data transactions in a data storage system, the data storage system being in network communication with a plurality of storage resources and comprising at least a data analysis module and a logging module, and receiving at the data analysis module at least one data transaction for data in the data storage system, each data transaction having at least one data-related characteristic; storing in the logging module the at least one data-related characteristic and a data transaction identifier that relates the data transaction to the associated at least one data-related characteristic in the logging module; analyzing at the data analysis module at least one data-related characteristic related to a first data transaction to determine if the first data transaction shares at least one data-related characteristic with other data transactions; and, in cases where the first data transaction shares at least one data-related characteristic with at least one other data transaction, logically linking the first data transaction with the other data transactions.

Claims

1. A computer-automated method for prioritizing storage resource allocation in a data storage system having a plurality of networked storage resources, the method comprising:

processing a plurality of data transactions for corresponding data in the data storage system, each one of said data transactions having at least one data-related characteristic related thereto;

logging each said at least one data-related characteristic in association with a respective data transaction identifier respectively identifying each of said processed data transactions;

analyzing said logged data-related characteristics to identify at least one shared data-related characteristic shared within respective subsets of said respectively identified data transactions;

logically linking said respectively identified data transactions within each of said respective subsets as a function of each said shared data-related characteristic; and

prioritizing allocation of the storage resources for data corresponding to at least one of said respective subsets as a function of said shared data-related characteristic.

2. The method of claim 1, further comprising allocating the storage resources as a result of said prioritizing.

3. The method of claim 1, wherein said data-related characteristic comprises a time-dependent characteristic, and wherein said prioritizing comprises dynamically prioritizing allocation as a function of said time-dependent characteristic.

4. The method of claim 1, wherein said analyzing comprises identifying a high data access frequency for a given time period in at least partially defining a given data transaction subset, and wherein said prioritizing comprises dynamically prioritizing allocation of a high performance storage resource during said given time period to data corresponding to said given subset.

5. The method of claim 4, wherein said analyzing further comprises identifying another data-related characteristic shared within another subset overlapping said given subset, and wherein said prioritizing further comprises dynamically prioritizing allocation of said high performance storage resource during said given time period to data corresponding to said other subset.

6. The method of claim 1, wherein said analyzing comprises identifying a low data access frequency for a given time period in at least partially defining a given data transaction subset, and wherein said prioritizing comprises dynamically prioritizing allocation of a low performance storage resource during said given time period to data corresponding to said given subset.

7. The method of claim 6, wherein said analyzing further comprises identifying another data-related characteristic shared within another subset overlapping said given subset, and wherein said prioritizing further comprises dynamically prioritizing allocation of said low performance storage resource during said given time period to data corresponding to said other subset.

8. The method of claim 1, wherein said allocation comprises dark storage allocation to storage locations characterized by infrequent use.

9. The method of claim 1, wherein said logging comprises logging to persistent memory.

10. The method of claim 1, wherein said logging comprises compacting logged data.

11. The method of claim 1, wherein said data-related characteristic comprises a priority of said corresponding data, and wherein said analyzing comprises analyzing said priority at prior intervals during at least one prior time period.

12. The method of claim 11, further comprising, at subsequent intervals during a subsequent time period that correspond with said prior intervals, allocating to a lower latency data storage resource select data whose priority at said prior intervals was greater than a preset priority threshold value.

13. The method of claim 12, wherein said allocating further comprises allocating to said lower latency data storage resource further data whose priority at said prior intervals was lesser than said preset priority threshold value but whose logged corresponding data transactions are logically linked with logged data transactions corresponding to said select data based on said at least one said distinctly shared data-related characteristic.

14. The method of claim 11, further comprising, at subsequent intervals during a subsequent time period that correspond with said prior intervals, allocating to a higher latency data storage resource select data whose priority at said prior intervals was lesser than a preset priority threshold value.

15. The method of claim 14, wherein said allocating further comprises allocating to said higher latency data storage resource further data whose priority at said prior intervals was higher than said preset priority threshold value but whose logged corresponding data transactions are logically linked with logged data transactions corresponding to said select data based on said at least one said distinctly shared data-related characteristic.

16. The method of claim 1, wherein said prioritizing results in grouped data storage resource allocation prioritization for data grouped from disperse storage locations within the system irrespective of physical data storage proximity and data access recency.

17. A device for prioritizing storage resource allocations in a data storage system having a plurality of networked data storage resources and servicing at least one data consumer, the device comprising:

a plurality of ports for communicatively coupling the device to the plurality of data storage resources and the plurality of data consumers;

a switch to route data transactions for data on the data storage system between the plurality of storage resources and the plurality of data consumers;

a processor; and

a memory, said memory having instructions located thereon that when implemented by said processor cause the device to;

monitor said data transactions to extract at least one respective data-related characteristic therefrom;

log each said data-related characteristic in said memory in association with a respective data transaction identifier;

logically link data transaction identifiers into transaction subsets based on shared data-related characteristics; and

prioritize allocation of the storage resources for data corresponding to at least one of said transaction subsets as a function of said shared data-related characteristics.

18. The device of claim 17, said instructions further causing placement of said corresponding data based on said allocation.

19. The device of claim 17 wherein said data-related characteristic comprises a time-dependent characteristic, and wherein said prioritizing comprises dynamically prioritizing allocation as a function of said time-dependent characteristic.

20. The device of claim 17, wherein the memory further comprises instructions to identify a high data access frequency for a given time period in at least partially defining a given data transaction subset, and wherein said prioritizing further comprises prioritizing allocation of a high performance storage resource during said given time period to data corresponding to said given subset.

21. The device of claim 17, wherein the memory further comprises instructions to identify a low data access frequency for a given time period in at least partially defining a given data transaction subset, and wherein said prioritizing further comprises prioritizing allocation of a lower performance storage resource during said given time period to data corresponding to said given subset.

22. The device of claim 17, wherein said allocation comprises dark storage allocation to storage locations characterized by infrequent use.

23. The device of claim 17, wherein said data-related characteristic comprises a priority of said corresponding data, and wherein said linking comprises analyzing said priority at prior intervals during at least one prior time period.

24. The method of claim 17, wherein said prioritizing results in grouped data storage resource allocation prioritization for data grouped from disperse storage locations within the system irrespective of physical data storage proximity and data access recency.