Managing metadata and data for a logical volume in a distributed and declustered system

• US 9,104,332 B2
• Filed: 04/16/2013
• Issued: 08/11/2015
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chambliss, David D., Garmiza, Ehood, Shalev, Leah, Weissbrem, Eliyahu
• Primary Examiner(s): Choe, Yong
• Assistant Examiner(s): Waddy, Jr., Edward
• Application Number:
  US13/863,831
• Publication Number:
  US 20140310489A1
• Time in Patent Office: 847 Days
• Field of Search: 711/114, 711/170, 711/162, 711/E12.103, 711/E12.001, 711/E12.002, 711/159, 711/165, 711/202, 711/209, 707/E17.01, 707/999.01, 707/E17.032, 707/827, 707/822, 707/737, 707/770, 707/E17.089, 707/626, 707/644, 707/652, 707/821, 707/823, 707/828, 707/999.204
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 2211/1057 Parity-multiple bits-RAID6, i.e. RAID 6 implementations
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Methods, apparatus and computer program products for a distributed system include dividing logical volume data into data subsets, and defining at least one distributedly storage configuration for the logical volume. Metadata for the logical volume is written to a first set of first metadata tables, and the first set of first metadata tables is divided into metadata subsets having a one-to-one correspondence with the data subsets. The metadata subsets are distributed among the multiple digital information devices, and the metadata is copied from the first set of first metadata tables to a second set of corresponding second metadata tables in a one-to-one correspondence with the first metadata tables. The second metadata tables are distributed among the multiple digital information devices, and upon modifying the metadata in a one of the first metadata tables while processing a storage request, the corresponding second metadata table is revised with the updated metadata.

Claims

1. A method, comprising:

arranging multiple storage devices and multiple digital information devices having respective memories to communicate within a network;

dividing data of a logical volume into data subsets;

defining, for the logical volume, at least one storage configuration for the data subsets distributed among the respective storage devices;

writing metadata for the logical volume to a first set of first metadata tables, each of the first metadata tables configured as a master metadata table;

dividing the first set of first metadata tables into metadata subsets having a one-to-one correspondence with the data subsets;

distributedly storing the metadata subsets among the multiple digital information devices, the storage configuration of the data subsets independent from the storing of the metadata subsets;

copying the metadata from the first set of first metadata tables to a second set of corresponding second metadata tables in a one-to-one correspondence with the first metadata tables, each of the second metadata tables configured as a backup metadata table;

distributedly storing the second metadata tables among the multiple digital information devices so that a given first metadata table and the corresponding second metadata table are stored to a different one of the multiple digital information devices, and so that the corresponding second metadata tables for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices; and

upon modifying the metadata in a one of the first metadata tables while processing a storage request, updating the corresponding second metadata table with the updated metadata.

2. The method according to claim 1, and comprising distributedly storing the second metadata tables so that the corresponding second metadata tables for the first metadata tables on the first of the digital information devices are distributed among at least an additional three of the digital information devices.

3. The method according to claim 1, and comprising upon detecting a failure of a given one of the digital information devices storing one of the first metadata tables, configuring the corresponding second metadata table as a new master metadata table, creating a new second metadata table, and configuring the new second metadata table as a backup metadata table for the new master metadata table.

4. The method according to claim 1, and comprising writing operational data for the logical volume to a first set of first operational data tables having a one-to-one correspondence with the data subsets, each of the first operational data tables configured as a master operational data table, dividing the first set of first operational data tables into operational data subsets, the storage configuration of the data subsets independent from the storing of the operational data subsets, distributedly storing the operational data subsets among the multiple digital information devices, copying the operational data to a second set of corresponding second operational data tables in a one-to-one correspondence with the first operational data tables, each of the second operational data tables configured as a backup operational data table, distributedly storing the first and the second operational data tables among the multiple digital information devices so that a given first operational data table and the corresponding second operational data table are stored to a different one of the multiple digital information devices, and so that the corresponding second operational data tables for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices, and upon modifying the operational data in a one of the first operational data tables while processing a storage request, updating the corresponding operational data table with the updated operational data.

5. The method according to claim 4, and comprising distributedly storing the second operational data tables so that the corresponding second operational data tables for the first operational data tables on the first of the digital information are distributed among at least an additional three of the digital information devices.

6. The method according to claim 4, and comprising upon detecting a failure of a given one of the digital information devices storing one of the first operational data tables, configuring the corresponding second operational data table as a new master cache, creating a new second operational data table, and configuring the new second operational data table as a backup operational data table for the new master operational data table.

7. The method according to claim 1, wherein the digital information device comprises a module of a storage controller, and wherein the storage configuration comprises a redundant array of independent disks (RAID) configuration.

8. The method according to claim 1, wherein the metadata for the logical volume is selected from a list comprising a partition table, a transaction distribution table and a disk distribution table.

9. An apparatus, comprising:

multiple storage devices and multiple digital information devices arranged on a network and having respective memories; and

a separate processor coupled to each of the respective memories and configured to divide data of a logical volume into data subsets, to define, for the logical volume, at least one storage configuration for the data subsets distributed among the respective storage devices, to write metadata for the logical volume to a first set of first metadata tables, each of the first metadata tables configured as a master metadata table, to divide the first set of first metadata tables into metadata subsets having a one-to-one correspondence with the data subsets, to distributedly store the metadata subsets among the multiple digital information devices, the storage configuration of the data subsets independent from the storing of the metadata subsets, to copy the metadata from the first set of first metadata tables to a second set of corresponding second metadata tables in a one-to-one correspondence with the first metadata tables, each of the second metadata tables configured as a backup metadata table, and to distributedly store the second metadata tables among the multiple digital information devices so that a given first metadata table and the corresponding second metadata table are stored to a different one of the multiple digital information devices, and so that the corresponding second metadata tables for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices, and upon modifying the metadata in a one of the first metadata tables while processing a storage request, a given one of the separate processors is configured to update the corresponding second metadata table with the updated metadata.

10. The apparatus according to claim 9, wherein the separate processor is configured to distributedly store the second metadata tables so that the corresponding second metadata tables for the first metadata tables on the first of the digital information devices are distributed among at least an additional three of the digital information devices.

11. The apparatus according to claim 9, wherein upon detecting a failure of a given one of the digital information devices storing one of the first metadata tables, the separate processor is arranged to configure the corresponding second metadata table as a new master metadata table, to create a new second metadata table, and to configure the new second metadata table as a backup metadata table for the new master metadata table.

12. The apparatus according to claim 9, wherein the separate processor is configured to write operational data for the logical volume to a first set of first operational data tables having a one-to-one correspondence with the data subsets, each of the first operational data tables configured as a master operational data table, the storage configuration of the data subsets independent from the storing of the cache subsets, to divide the first set of first operational data tables into operational data subsets, to distributedly store the operational data subsets among the multiple digital information devices, to copy the operational data to a second set of corresponding second operational data tables in a one-to-one correspondence with the first operational data tables, each of the second operational data tables configured as a backup operational data table, to distributedly store the first and the second operational data tables among the multiple digital information devices so that a given first operational data table and the corresponding second operational data table are stored to a different one of the multiple digital information devices, and so that the corresponding second operational data table for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices, and wherein upon modifying the operational data in a one of the first operational data tables while processing a storage request, the separate processor is configured to update the corresponding operational data table with the updated operational data.

13. The apparatus according to claim 12, wherein the separate processor is configured to distributedly store the second operational data tables so that the corresponding second operational data tables for the first operational data tables on the first of the digital information are distributed among at least an additional three of the digital information devices.

14. The apparatus according to claim 12, wherein upon detecting a failure of a given one of the digital information devices storing one of the first operational data tables, the separate processor is arranged to configure the corresponding second operational data table as a new master operational data table, to create a new second operational data table, and to configure the new second operational data tables as a backup operational data table for the new master cache.

15. A computer program product, the computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to arrange multiple storage devices and multiple digital information devices having respective memories to communicate within a network;

computer readable program code configured to divide data of a logical volume into data subsets;

computer readable program code configured to define, for the logical volume, at least one storage configuration for the data subsets distributed among the respective storage devices;

computer readable program code configured to write metadata for the logical volume to a first set of first metadata tables, each of the first metadata tables configured as a master metadata table;

computer readable program code configured to divide the first set of first metadata tables into metadata subsets having a one-to-one correspondence with the data subsets;

computer readable program code configured to distributedly store the metadata subsets among the multiple digital information devices, the storage configuration of the data subsets independent from the storing of the metadata subsets;

computer readable program code configured to copye the metadata from the first set of first metadata tables to a second set of corresponding second metadata tables in a one-to-one correspondence with the first metadata tables, each of the second metadata tables configured as a backup metadata table;

computer readable program code configured to distributedly store the second metadata tables among the multiple digital information devices so that a given first metadata table and the corresponding second metadata table are stored to a different one of the multiple digital information devices, and so that the corresponding second metadata tables for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices; and

computer readable program code configured to update, upon modifying the metadata in a one of the first metadata tables while processing a storage request, the corresponding second metadata table with the updated metadata.

16. The computer program product according to claim 15, and comprising computer readable program code configured to distributedly store the second metadata tables so that the corresponding second metadata tables for the first metadata tables on the first of the digital information devices are distributed among at least an additional three of the digital information devices.

17. The computer program product according to claim 15, and comprising upon detecting a failure of a given one of the digital information devices storing one of the first metadata tables, computer readable program code configured to configure the corresponding second metadata table as a new master metadata table, to create a new second metadata table, and to configure the new second metadata table as a backup metadata table for the new master metadata table.

18. The computer program product according to claim 15, and comprising computer readable program code configured to write operational data for the logical volume to a first set of first operational data tables having a one-to-one correspondence with the data subsets, each of the first operational data tables configured as a master operational data table, to divide the first set of first operational data tables into operational data subsets, the storage configuration of the data subsets independent from the storing of the operational data subsets, to distributedly store the operational data subsets among the multiple digital information devices, to copy the operational data to a second set of corresponding second operational data tables in a one-to-one correspondence with the first operational data tables, each of the second operational data tables configured as a backup operational data table, to distributedly store the first and the second operational data tables among the multiple digital information devices so that a given first operational data table and the corresponding second operational data table are stored to a different one of the multiple digital information devices, and so that the corresponding second operational data tables for the first metadata tables on a first of the digital information devices are distributed among at least an additional two of the digital information devices, and upon modifying the operational data in a one of the first operational data tables while processing a storage request, computer readable program code configured to update the corresponding operational data table with the updated operational data.

19. The computer program product according to claim 18, and comprising computer readable program code configured to distributedly store the second operational data tables so that the corresponding second operational data tables for the first operational data tables on the first of the digital information are distributed among at least an additional three of the digital information devices.

20. The computer program product according to claim 18, and comprising upon detecting a failure of a given one of the digital information devices storing one of the first operational data tables, computer readable program code configured to configure the corresponding second operational data table as a new master cache, to create a new second operational data table, and to configure the new second operational data table as a backup operational data table for the new master operational data table.