Managing data on data storage systems

• US 8,392,370 B1
• Filed: 03/28/2008
• Issued: 03/05/2013
• Est. Priority Date: 03/28/2008
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Whitney, William, Gupta, Uday K., Evans, Michael B., Cotter, Gerald E.
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US12/058,240
• Time in Patent Office: 1,803 Days
• Field of Search: None
• US Class Current: 707/640
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A method is used in managing data on data storage systems. Rules are defined for candidate data unit selection. The rules are automatically executed on a primary data storage system to examine data units on the primary data storage system. The rules are applied to the data units. A data unit is identified that matches the rules and the identified data unit is migrated. The identified data unit is copied from the primary storage system to a secondary storage system. The identified data unit on the primary storage system is replaced with metadata. the data storage space in the primary storage system corresponding to the identified data unit is freed up.

Claims

1. A method for use in managing data on data storage systems, the method comprising:

defining first rules for candidate data unit selection, wherein the first rules are based on characteristics of data units, wherein the characteristics of the data units include age of each data unit, last access time for each data unit, size of each data unit, and file type of each data unit;

automatically executing the first rules on a primary data storage system to examine the data units on the primary data storage system, wherein the first rules identify the data units for migration;

based on the first rules, dividing the data units on the primary data storage system into priority groups, wherein the priority groups are based on usage information of the data units;

identifying a priority group for migration;

migrating each data unit of the identified priority group from the primary storage system to a secondary storage system, wherein each data unit is migrated to the secondary storage system within the same storage domain;

creating an association between the primary storage system and the secondary storage system for maintaining a consistency group for the data units;

replacing each migrated data unit with metadata on the primary storage system, wherein the metadata has a same identifier as the migrated data unit the metadata is replacing and stores priority data associated with the migrated data unit the metadata is replacing;

freeing up data storage space in the primary storage system corresponding to each data unit of the identified priority group;

based on second rules, determining whether an access attempt for one of the migrated data units can be satisfied from the secondary storage system, wherein the second rules indicate how to access the one of the migrated data units

based on the determination, completing the access attempt by accessing the one of the migrated data units from the secondary storage system;

determining whether the access attempt causes the one of the migrated data units to belong to a different priority group by changing usage information of the one of the migrated data units; and

based on the determination, copying a subset of contents of the one of the migrated data units from the secondary storage system to the primary storage system after the access attempt has been satisfied from the secondary storage system.

2. The method of claim 1, wherein the primary data storage system has a first priority group containing data units that are frequently accessed and a second priority group containing data units that are seldom accessed, and the identified data unit belongs to the second priority group.

3. The method of claim 1, wherein the identified data unit is maintained in its original lookup space.

4. The method of claim 1, wherein the rules take into account the data unit'"'"'s last access time.

5. The method of claim 1, wherein the data unit is copied to a respective location in the secondary storage system and the metadata points to the respective location.

6. The method of claim 1, wherein the data unit is moved to optimized storage within the same storage domain.

7. The method of claim 1, wherein a tight linkage is provided between the primary storage system and the secondary storage system to maintain a consistency group.

8. The method of claim 1, wherein the secondary storage system is storage that uses redundant data elimination (RDE).

9. The method of claim 1, wherein at least one of the primary and secondary storage systems uses semiconductor memory.

10. The method of claim 1, wherein at least one of the primary and secondary storage systems uses semiconductor memory exclusively.

11. The method of claim 1, wherein a data unit that is accessed frequently is stored in flash memory.

12. The method of claim 1, wherein a directory listing that includes the identified data unit has the same appearance to the user as a conventional directory listing.

13. The method of claim 1, further comprising:

processing data units on a project basis so that data units belonging to a file system and data storage associated with a particular project are moved to the secondary storage system automatically once the project period has completed.

14. The method of claim 1, wherein the namespace of a file system is unaffected so that on demand needs can be serviced without any manual intervention.

15. The method of claim 1, wherein the migration takes into account the type and size of the data unit.

16. The method of claim 1, wherein the identified data unit is migrated on a schedule.

17. The method of claim 1, further comprising:

invoking a driver when an access attempt is made to the identified data unit that has been moved to the secondary storage system, the driver causing at least some of the contents of the identified data unit to be retrieved from the secondary storage system to service the access attempt.

18. The method of claim 1, further comprising:

giving priority to reducing electrical power consumption by moving the identified data unit to a secondary storage system that uses little or no power when its data units are not being accessed.