MECHANISMS FOR PERFORMING ACCURATE SPACE ACCOUNTING FOR VOLUME FAMILIES

• US 20200133555A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Nanda, Kumari Bijayalaxmi, Marko, Michal, Sen, Soumyadeep, Mathews, Alexander S.
• Granted Patent: US 11,010,082 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

Techniques for performing space accounting for volume families. The techniques include maintaining two counters for each volume family, including a first counter that tracks a first amount of physical space allocated to volumes in the volume family based on the number of pages written to the volumes, and a second counter that tracks a second amount of physical space owned by the volume family, and maintaining a third counter for each volume in each branch of the volume family that tracks a number of pages among a total number of pages written to the volume that were overwritten in the immediate sequential next volume in the same branch as the volume. By maintaining, for each volume family, the first counter and the second counter, and, for each volume in the volume family, the third counter, space accounting metrics can be obtained that allow data storage activities to be performed efficiently.

Claims

1. A method of performing space accounting for volume families in a data storage system, comprising:

maintaining a first count representing a first amount of physical storage space allocated to a plurality of data volumes in a volume family based on a number of data pages written to the respective data volumes, the plurality of data volumes being stored in association with a first data storage appliance;

maintaining a second count representing a second amount of physical storage space allocated to the plurality of data volumes in the volume family, the second amount of physical storage space corresponding to an amount of physical storage space owned by the volume family; and

based on one or more of the first amount of physical storage space and the second amount of physical storage space, performing a migration of the plurality of data volumes in the volume family from the first data storage appliance to a second data storage appliance, the first amount of physical storage space corresponding to an amount of physical storage space required to receive the volume family at the second data storage appliance, and the second amount of physical storage space corresponding to an amount of physical storage space recovered at the first data storage appliance having performed the migration of the volume family.

2. The method of claim 1 wherein the data pages include one or more deduplicated data pages, and wherein the method further comprises:

maintaining a deduplication domain for each deduplicated data page written to the respective data volumes, the deduplication domain consisting of data segments within the volume family.

3. The method of claim 1 wherein the data pages include a deduplicated data page, the deduplicated data page including one or more data segments that are shared among different logical addresses across the same data volume or different data volumes, and wherein the method further comprises:

tracking multiple logical address references to each data segment of the deduplicated data page.

4. The method of claim 3 further comprising:

determining whether one or more of the logical address references to the data segment are from the same data volume or different data volumes of the volume family; and

having determined that one or more of the logical address references to the data segment are from the same data volume or different data volumes of the volume family, increasing the second count by an amount corresponding to a physical storage space allocated to the data segment.

5. The method of claim 3 further comprising:

determining whether one or more of the logical address references to the data segment are from the same data volume or different data volumes of another volume family; and

having determined that one or more of the logical address references to the data segment are from the same data volume or different data volumes of the other volume family, reducing the second count by an amount corresponding to a physical storage space allocated to the data segment.

6. The method of claim 1 wherein the maintaining of the first count representing the first amount of physical storage space allocated to the plurality of data volumes in the volume family includes tracking a logical size of each data volume, the logical size corresponding to a logical committed size of the data volume.

7. The method of claim 1 wherein the volume family has an associated family identifier (ID), wherein the family ID is referred to herein as an 10 key, wherein the volume family includes a writable volume, and wherein the method further comprises:

receiving a write request at the first data storage appliance, the write request requesting that a data page be written to the writable volume of the volume family;

incorporating the IO key into the write request; and

forwarding the write request along a write path to effect writing of the data page to the writable volume of the volume family.

8. The method of claim 7 further comprising:

maintaining the IO key in a namespace index node (Mode) of the first data storage appliance.

9. The method of claim 7 further comprising:

having written the data page to the writable volume of the volume family, increasing the first count by an amount corresponding to a logical committed storage space allocated to the data page.

10. The method of claim 1 wherein the plurality of data volumes in the volume family include one or more snapshot volumes, and wherein the method further comprises:

performing snapshot delete processing to delete at least one of the snapshot volumes in the volume family;

having performed the snapshot delete processing, determining an amount of logical committed storage space recovered from the snapshot delete processing; and

reducing the first count by an amount corresponding to the recovered logical committed storage space.

11. The method of claim 1 wherein the volume family has at least one branch including a sequence of at least some of the plurality of data volumes in the volume family, wherein the volume family has an associated family identifier (ID), each branch of the volume family has an associated branch ID, and each data volume in the volume family has an associated volume ID, wherein the family ID, the branch ID, and the volume ID for each data volume in the volume family are referred to herein collectively as an IO key, wherein a first branch of the volume family includes a writable volume, and wherein the method further comprises:

receiving a write request at the first data storage appliance, the write request requesting that a data page be written to the writable volume of the volume family;

incorporating the IO key into the write request; and

forwarding the write request along a write path to effect writing of the data page to the writable volume of the volume family.

12. The method of claim 11 further comprising:

maintaining the IO key in a namespace index node (inode) of the first data storage appliance.

13. The method of claim 11 further comprising:

maintaining a mapping hierarchy of the sequence of data volumes, the mapping hierarchy including at least a volume level with a plurality of volume nodes corresponding to the sequence of data volumes, respectively, and a leaf level with a plurality of leaf nodes corresponding to a plurality of leaf pages, respectively, each leaf page pointing to a data page of a respective one of the sequence of data volumes.

14. The method of claim 13 wherein each leaf page has an attribute specifying an owner volume ID of an owner of the data page pointed to by the leaf page, and wherein the method further comprises:

having written the data page to the writable volume, setting an owner volume ID in a leaf page of the mapping hierarchy pointing to the data page to a current latest volume ID of the writable volume.

15. The method of claim 14 further comprising:

creating a snapshot volume in the first branch of the volume family based on the writable volume;

assigning the current latest volume ID to the snapshot volume; and

assigning a new latest volume ID to the writable volume.

16. The method of claim 15 further comprising:

having created the snapshot volume in the first branch of the volume family, setting the owner volume ID in the leaf page of the mapping hierarchy pointing to the data page to the new latest volume ID of the writable volume.

17. The method of claim 15 further comprising:

maintaining a third count representing a third amount of physical storage space allocated to the plurality of data volumes in the volume family, the third amount of physical storage space corresponding to a number of data pages among a total number of data pages written to the snapshot volume that were overwritten in an immediate sequential next volume in the first branch of the volume family, taking into account a logical size of the snapshot volume.

18. A data storage appliance configured to perform space accounting for volume families, comprising:

at least one storage device configured to store a plurality of data volumes of a volume family;

a memory configured to store executable instructions; and

storage processing circuitry configured to execute the executable instructions out of the memory;

to maintain a first count representing a first amount of physical storage space allocated to the plurality of data volumes in the volume family based on a number of data pages written to the respective data volumes;

to maintain a second count representing a second amount of physical storage space allocated to the plurality of data volumes in the volume family, the second amount of physical storage space corresponding to an amount of physical storage space owned by the volume family; and

based on one or more of the first amount of physical storage space and the second amount of physical storage space, to perform a migration of the plurality of data volumes in the volume family to a second data storage appliance, the first amount of physical storage space corresponding to an amount of physical storage space required to receive the volume family at the second data storage appliance, and the second amount of physical storage space corresponding to an amount of physical storage space recovered having performed the migration of the volume family.

19. The data storage appliance of claim 18 wherein the data pages include one or more deduplicated data pages, and wherein the storage processing circuitry is further configured to execute the executable instructions out of the memory to maintain a deduplication domain for each deduplicated data page written to the respective data volumes, the deduplication domain consisting of data segments within the volume family.

20. A computer program product including a set of non-transitory, computer-readable media having instructions that, when executed by control circuitry of a computerized apparatus, cause the control circuitry to perform a method of performing space accounting for volume families in a data storage system, the method comprising:

maintaining a first count representing a first amount of physical storage space allocated to a plurality of data volumes in a volume family based on a number of data pages written to the respective data volumes, the plurality of data volumes being stored in association with a first data storage appliance;

maintaining a second count representing a second amount of physical storage space allocated to the plurality of data volumes in the volume family, the second amount of physical storage space corresponding to an amount of physical storage space owned by the volume family; and

based on one or more of the first amount of physical storage space and the second amount of physical storage space, performing a migration of the plurality of data volumes in the volume family from the first data storage appliance to a second data storage appliance, the first amount of physical storage space corresponding to an amount of physical storage space required to receive the volume family at the second data storage appliance, and the second amount of physical storage space corresponding to an amount of physical storage space recovered at the first data storage appliance having performed the migration of the volume family.