Automated information lifecycle management system for network data storage

• US 7,693,877 B1
• Filed: 03/23/2007
• Issued: 04/06/2010
• Est. Priority Date: 03/23/2007
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Zasman, Gary Ivor
• Primary Examiner(s): Vital; Pierre M
• Assistant Examiner(s): Nguyen; Loan T
• Application Number:
  US11/726,816
• Time in Patent Office: 1,110 Days
• Field of Search: None
• US Class Current: 707/707
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An automated information lifecycle management (ILM) system for network storage automatically searches a network storage system to identify data items stored therein. The network storage system includes multiple data storage facilities of different types coupled to a network. The ILM system automatically classifies each of the data items into a category of multiple data categories, and automatically optimizes each of the data items to a particular data storage facility of the multiple data storage facilities, according to the category into which the data item was classified. The ILM system may also automatically discover and detect storage devices added to the network and assign them to the proper storage class, for storing data of the corresponding category. The ILM system further may automatically determine or predict when additional storage capacity is needed for any particular class of storage and may alert a user accordingly and/or free up additional storage capacity.

Claims

1. A method comprising:

automatically searching a network storage system to identify a plurality of data items stored in the network storage system, the network storage system including a plurality of data storage facilities of different types coupled to a network;

automatically classifying each of the data items into a category of a plurality of data categories, said automatically classifying each of the data items comprising automatically classifying a first data item;

automatically optimizing each of the data items to a particular data storage facility of the plurality of data storage facilities, according to the category into which the data item was classified, said automatically optimizing each of the data items to a particular data storage facility comprising automatically migrating the first data item to a first data storage facility of the plurality of data storage facilities based on a classification of the first data item;

automatically, by a computer, reclassifying the first data item after the first data item has been migrated to the first data storage facility; and

after automatically reclassifying the first data item, automatically performing one of moving, migrating and deleting the first data item based on a new classification of the first data item.

2. A method as recited in claim 1, wherein the plurality of data storage facilities include a plurality of different storage classes.

3. A method as recited in claim 2, wherein automatically optimizing each of the data items to a particular data storage facility comprises:

moving a first data item from a first storage facility to a second storage facility;

migrating a second data item from the first storage facility to a third storage facility;

deleting a third data item; and

de-duplicating a fourth data item.

4. A method as recited in claim 2, further comprising:

automatically detecting a storage device connected to the network;

automatically determining a storage class of the storage device; and

automatically associating the storage device with a data category of said plurality of data categories, based on the storage class of the storage device.

5. A method as recited in claim 2, further comprising:

automatically detecting that an amount of used storage space in one of said storage classes meets a specified criterion; and

in response to detecting that the amount of used storage space in said one of said storage classes meets the specified criterion, causing an alert to issue to indicate to a user that additional storage space is needed for said one of said storage classes.

6. A method as recited in claim 2, further comprising:

automatically detecting that an amount of used storage space in one of said storage classes exceeds the specified capacity usage criterion; and

in response to automatically detecting that the amount of used storage space in said one of said storage classes exceeds the specified capacity usage criterion, automatically freeing additional storage space in said one of said storage classes.

7. A method as recited in claim 1, wherein the plurality of data categories include:

primary data, confidential data, archivable data, and compliance data.

8. A method as recited in claim 1, wherein the plurality of data storage facilities include:

a primary storage facility; and

an archive storage facility which has an access latency greater than an access latency of the primary storage facility.

9. A method as recited in claim 8, further comprising an encryption engine, wherein the plurality of data storage facilities further include a storage facility to store encrypted data encrypted by the encryption engine.

10. A method as recited in claim 9, wherein the plurality of data storage facilities further include a storage facility to store write-once/read-many (WORM) data.

11. A method comprising:

automatically identifying a plurality of data items stored in a network storage system, the plurality of data items including a plurality of files, the network storage system including a plurality of data storage facilities of different classes coupled to a network;

automatically classifying each of the data items into a category of a plurality of data categories;

automatically, by a computer, optimizing each of the data items to a particular data storage facility of the plurality of data storage facilities, according to the category into which the data item was classified, wherein said automatically optimizing comprises;

moving a first data item from a first storage facility to a second storage facility based on a classification of the first data item,

migrating a second data item from the first storage facility to a third storage facility based on a classification of the second data item,

deleting a third data item based on a classification of the third data item,

replicating a fourth data item based on a classification of the fourth data item, and

de-duplicating a fifth data item;

automatically detecting a storage device connected to the network storage system;

automatically determining a storage class of the storage device;

automatically assigning the storage device to a data category of said plurality of data categories, based on the storage class of the storage device;

automatically detecting that an amount of used storage space in one of said storage classes meets a specified criterion; and

in response to detecting that the amount of used storage space in said one of said storage classes meets the specified criterion, performing at least one of;

causing an alert to issue to indicate to a user that additional storage space is needed for said one of said storage classes; and

automatically freeing additional storage space in said one of said storage classes.

12. A method as recited in claim 11, wherein:

said automatically classifying each of the data items comprises automatically classifying a first data item;

said automatically optimizing each of the data items to a particular data storage facility comprises automatically moving or migrating the first data item to a first data storage facility of the plurality of data storage facilities based on a classification of the first data item; and

wherein the method further comprises;

reclassifying the first data item after the first data item has been moved or migrated to the first data storage facility; and

after the first data item has been reclassified, automatically moving, migrating or deleting the first data item based on a new classification of the first data item.

13. A method as recited in claim 11, wherein the plurality of data categories include:

primary data, confidential data, old data, and compliance data.

14. A method as recited in claim 11, wherein the plurality of storage classes include:

primary storage;

archive storage which has an access latency greater than an access latency of the primary storage;

encrypted data storage; and

write-once/read-many (WORM) storage.

15. An automated information lifetime management (ILM) system for use in a network storage system, the ILM system comprising:

a search and classification module to search the network storage system to identify a plurality of data items stored in the network storage system and to automatically classify each of the data items into a category of a plurality of data categories, the network storage system including a plurality of data storage facilities of different classes, wherein the automatically classifying each of the data items includes automatically classifying a first data item;

an optimization engine to automatically optimize each of the data items to a particular data storage facility of the plurality of data storage facilities, according to the category into which the data item was classified, wherein automatically optimizing each of the data items includes automatically migrating the first data item to a first data storage facility of the plurality of data storage facilities based on a classification of the first data item;

a reclassification module executing on a processor to reclassify the first data item after the first data item has been migrated to the first data storage facility; and

a movement module to automatically perform one of moving, migrating, and deleting the first data item based on a new classification of the first data item, wherein the one of moving, migrating and deleting the first data item occurs after the first data item is reclassified.

16. An automated ILM system as recited in claim 15, wherein the optimization engine is configured to optimize data by moving data, migrating data, deleting data, and de-duplicating data, according to specified policies.

17. An automated ILM system as recited in claim 16, further comprising a discovery and allocation module to:

detect a storage device connected to the network storage system;

determine a storage class to which the storage device belongs; and

associate the storage device with a data category of said plurality of data categories, based on the storage class to which the storage device belongs.

18. An automated ILM system as recited in claim 16, further comprising a capacity management module to:

detect that an amount of used storage space in one of said storage classes meets a specified capacity criterion; and

in response to detecting that the amount of used storage space in said one of said storage classes meets the specified capacity criterion, cause an alert to issue to indicate to a user that additional storage space is needed for said one of said storage classes.

19. An automated ILM system as recited in claim 16, further comprising a capacity management module to:

detect that an amount of used storage space in one of said storage classes meets a specified capacity criterion; and

in response to detecting that the amount of used storage space in said one of said storage classes meets the specified capacity criterion, free additional storage space in said one of said storage classes.

20. An automated ILM system as recited in claim 15, wherein the plurality of data categories include:

primary data, confidential data, old data, and compliance data.

21. An automated ILM system as recited in claim 15, further comprising an encryption engine to encrypt data classified as confidential by the search and classification module.

22. An automated ILM system as recited in claim 15, wherein the plurality of data storage facilities include:

a primary storage facility; and

an archive storage facility which has an access latency greater than an access latency of the primary storage facility.

23. An automated ILM system as recited in claim 22, wherein the plurality of data storage facilities further include a storage facility to store encrypted data.

24. An automated ILM system as recited in claim 22, wherein the plurality of data storage facilities further include a storage facility to store write-once/read-many (WORM) data.

25. A storage server to operate in a network storage system, the storage server comprising:

a storage interface through which to access a mass storage subsystem including plurality of non-volatile mass storage devices;

a network interface through which to communicate with a plurality of storage clients over a network;

a storage manager to process data access requests received from the storage clients directed to data stored in the mass storage subsystem; and

an ILM subsystem including;

logic to search the network storage system to identify a plurality of data items stored in the network storage system, the network storage system including a plurality of data storage facilities of different types;

logic to classify each of the data items into a category of a plurality of data categories, the classifying including classifying a first data item;

logic to optimize each of the data items to a particular data storage facility of the plurality of data storage facilities, according to the category into which the data item was classified, the optimizing including migrating the first data item to a first data storage facility based on a classification of the first data item;

logic to reclassify the first data item after the first data item is migrated to the first data storage facility; and

logic to perform one of migrating, moving and deleting the first data item based on a new classification of the first data item.

26. A storage server as recited in claim 25, wherein the plurality of data storage facilities include a plurality of different storage classes.

27. A storage server as recited in claim 25, wherein the logic to optimize comprises logic to move data, logic to migrate data, logic to delete data, and logic to de-duplicate data, according to specified policies.

28. A storage server as recited in claim 25, wherein the ILM subsystem further includes:

logic to detect a storage device connected to the network;

logic to determine a storage class of the storage device; and

logic to associate the storage device with a data category of said plurality of data categories, based on the storage class of the storage device.

29. A storage server as recited in claim 25, wherein the ILM subsystem further includes:

logic to detect that an amount of used storage space in one of said storage classes meets a specified criterion; and

logic to cause an alert to issue to indicate to a user that additional storage space is needed for said one of said storage classes, in response to detecting that the amount of used storage space in said one of said storage classes meets the specified criterion.

30. A storage server as recited in claim 25, wherein the ILM subsystem further includes:

logic to detect that an amount of used storage space in one of said storage classes meets a specified criterion; and

logic to free additional storage space in said one of said storage classes in response to detecting that an amount of used storage space in one of said storage classes meets a specified capacity usage criterion.

31. A storage server as recited in claim 25, wherein the plurality of data categories include:

primary data, confidential data, old data, and compliance data.

32. A storage server as recited in claim 25, further comprising logic to encrypt data classified as confidential by the ILM module.

33. A storage server as recited in claim 25, wherein the plurality of data storage facilities include:

a primary storage facility; and

an archive storage facility which has an access latency greater than an access latency of the primary storage.

34. A storage server as recited in claim 33, wherein the plurality of data storage facilities further include a storage facility to store encrypted data;

35. A storage server as recited in claim 34, wherein the plurality of data storage facilities further include a storage facility to store write-once/read-many (WORM) data.

36. A system comprising:

a processor coupled to a memory through a bus; and

executable instructions stored in the memory which when executed by the processor cause the processor to automatically;

search a network storage system to identify a first data item stored in the network storage system;

classify the first data item into a first category, the classifying based on one of contents of the first data item and an access history of the first data item;

migrate the first data item to a first data storage facility based on a classification of the first data item into the first category;

reclassify the first data item into a second category after migrating the first data item to the first data storage facility, the reclassifying based on one of the contents of the first data item and the access history of the first data item; and

migrate the first data item based on the reclassification of the first data item into the second category.