Adding Storage Capacity to an Object Storage System

• US 20150205531A1
• Filed: 01/20/2014
• Published: 07/23/2015
• Est. Priority Date: 01/20/2014
• Status: Abandoned Application

• Current Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Original Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Inventors: DeMattio, Christopher J., Cutforth, Craig F., Arnold, Caroline W.
• Application Number:
  US14/159,181
• Publication Number:
  US 20150205531A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Apparatus and method for adding storage capacity to an object storage system. In accordance with some embodiments, a first set of data storage devices store data objects in accordance with a first map structure. A management module detects a second set of data storage devices added to the first set and, in response thereto, generates a second map structure and migrates a portion of the data objects from the first set to the second set based on the second map structure to balance the first and second sets.

Claims

1. An object storage system comprising:

a first set of data storage devices storing data objects in accordance with a first map structure; and

a management module configured to detect a second set of at least one data storage device added to the first set and, in response thereto, generate a second map structure and migrate a portion of the data objects from the first set to the second set based on the second map structure to balance the first and second sets.

2. The object storage system of claim 1, wherein the first set of data storage devices provide a first overall data storage capacity to store and retrieve data objects of users of the object storage system, and the addition of the second set of at least one data storage device provides a larger, second overall data storage capacity to store and retrieve data objects of users of the object storage system.

3. The object storage system of claim 1, wherein the first set of data storage devices are arranged into N subgroups, and wherein a different number of data objects in each of the N subgroups are migrated to the second set of at least one data storage device so that each of the N subgroups and the second set store nominally the same total number of data objects.

4. The object storage system of claim 1, wherein the first map structure correlates the data objects to the respective data storage devices in the first set prior to the migration, and the second map structure correlates the data objects to the respective data storage devices in the first and second set after the migration.

5. The object storage system of claim 1, wherein the management module comprises a new storage detection module adapted to automatically generate the second map structure and a data migration module adapted to automatically migrate data objects to the second set of storage devices to conform the data objects to the second map structure.

6. The object storage system of claim 5, wherein the data migration module comprises a mapping compare block which identifies a set of data objects requiring migration to conform to the second map structure, and a migration sequencing block which issues a succession of data migration commands to a storage controller to migrate the set of data objects to conform to the second map structure, wherein the map builder module transfers a copy of the second map structure responsive to a data migration complete status signal from the data migration module indicating that the set of data objects have been successfully migrated.

7. The object storage system of claim 3, further comprising a transition management block which manages data access operations upon the data objects migrated by the data migration module prior to the issuance of the data migration compete status signal.

8. The object storage system of claim 1, wherein the management module migrates the data objects prior to deployment of the second map structure to a server.

9. The object storage system of claim 1, wherein the management module will receive limited input before generating a second map structure or migrating a portion of the data objects to balance the first and second sets.

10. The object storage system of claim 1, wherein the first set of data storage devices are arranged into a plurality of storage nodes each having an associated storage controller, and the second set of at least one data storage device is added to a selected storage node to expand a total storage capacity of the selected storage node.

11. The object storage system of claim 10, wherein the management module migrates the data objects within the selected node prior to deployment of the second map structure to the storage controllers of the remaining storage nodes.

12. The object storage system of claim 1, further comprising a server adapted to communicate between users of the object storage system and the first set of data storage devices along a primary communication path, and wherein the management module migrates the data objects from the first set of data storage devices to a second set of at least one data storage device along a secondary communication path in parallel to the primary communication path.

13. An object storage system comprising:

a storage controller having a processor and memory;

a plurality of storage devices connected to the storage controller to form a storage node, the storage devices arranged into N subgroups, wherein data objects are stored in the N subgroups based on a first map structure stored in the storage controller memory; and

a management module which automatically detects an additional plurality of storage devices connected to the storage controller to provide N+1 subgroups, migrates a portion of the data objects in each of the N subgroups to the additional plurality of storage devices, generates a second map structure to describe the data objects stored in the N+1 subgroups, and stores the second map structure in the storage controller memory.

14. The object storage system of claim 13 further comprising a plurality of additional storage nodes each comprising a storage controller with a processor and memory and a plurality of storage devices associated with the storage controller, wherein the management module further operates to transfer a copy of the second map structure to each of the additional storage nodes for storage in the associated storage controller memory of the node.

15. The object storage system of claim 13, wherein the management module comprises a new storage detection module adapted to generate the second map structure and a data migration module adapted to migrate data objects to the second set of storage devices to conform the data objects to the second map structure.

16. A computer implemented method comprising:

storing data objects in a first set of data storage devices of an object storage system in accordance with a first map structure stored in a memory;

connecting a second set of data storage devices to the first set; and

detecting the connection of the second set to the first set, and in response thereto, generating a second map structure and migrating a portion of the data objects from the first set to the second set based on the second map structure to rebalance the first and second sets.

17. The method of claim 16, wherein the first set of data storage devices provide a first overall data storage capacity to store and retrieve data objects of users of the object storage system, and the addition of the second set of data storage devices provides a larger, second overall data storage capacity to store and retrieve data objects of users of the object storage system.

18. The method of claim 16, wherein the first set of data storage devices are arranged into N subgroups, and wherein a different number of data objects in each of the N subgroups are migrated to the second set of data storage devices so that each of the N subgroups and the second set store nominally the same total number of data objects.

19. The method of claim 16, wherein the first set of data storage devices are associated with a selected storage node in the object storage system having a storage controller, wherein the detecting step is carried out by a management module having a new storage detection module which detects the connection of the second set, a new map generation module which generates the second map structure, and a data migration module which migrates the data objects from the first set to the second set.

20. The method of claim 16, wherein the object storage system further comprises a proxy server which processes data transfers between the selected storage node and users of the object storage system, and wherein the management module further operates to detect an available bandwidth of the proxy server and to transfer the data objects to the second set so as to fall within the detected available bandwidth.