Merging containers in a multi-container system

• US 8,370,401 B2
• Filed: 10/26/2010
• Issued: 02/05/2013
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Gordon, Colin Stebbins, Singh, Pratap Vikram, Trimmer, Donald Alvin
• Primary Examiner(s): Btit, Jacob F
• Assistant Examiner(s): Shanmugasundaram, Kannan
• Application Number:
  US12/912,436
• Publication Number:
  US 20110040795A1
• Time in Patent Office: 833 Days
• Field of Search: 707/808, 707999001-999005
• US Class Current: 707/808
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/955 Object-oriented

Abstract

An apparatus and method for merging the data of a first container and a second container into a target container that includes at least one block having a plurality of extents to store data objects. The apparatus may include a storage sever coupled to a plurality of storage devices. The storage server is configured to merge multiple data objects of the plurality of containers that store a data object per each container into a target container that stores multiple data objects within the target container.

Claims

1. A method, comprising:

receiving a merge request to merge a first logical container and a second logical container, wherein the first logical container includes a first data object and the second logical container includes a second data object, wherein each logical container comprises at least one block wherein the first and second data objects are stored at a block level in the first and second logical containers respectively in an existing file system whose internal block fragmentation dictates the size of each block;

merging the first and second data objects of the first and second logical containers into a target container, wherein the target container comprises at least one block having a plurality of extents to store data objects, wherein the first and second data objects are stored at an extent level within the target container after said merging, wherein the extent is a smaller unit of storage space than the block, and wherein merging the data comprises separately referencing the first logical container, the second logical container, the target container, the first data object, and the second data object in response to the merge request;

updating a container map to indicate that the first data object and the second data object are stored in the target container, wherein the container map includes a plurality of mapping entries that map filenames to data object identifiers, wherein each of the data object identifiers identifies a corresponding data object and includes;

a container identifier that specifies a logical container in which that corresponding data object is stored; and

an object index value used to identify which of one or more data objects stored in that logical container is that corresponding data object; and

updating an object map stored within the target container to indicate extent locations within the target container of the first data object and the second data object, wherein the object map comprises a listing of a plurality of data objects stored in the target container and extent locations within the target container where the plurality of data objects are stored.

2. The method of claim 1, wherein the target container is one of the first and second logical containers.

3. The method of claim 1, wherein the target container is a third logical container.

4. The method of claim 1, wherein merging the data of the first and second containers comprises:

migrating the first data object within the first logical container to the target container; and

migrating the second data object within the second logical container to the target container.

5. The method of claim 4, wherein migrating the first data object comprises:

reading the first data object from a physical location of the first data object within the first logical container using a first identifier;

writing the first data object into a physical location of the target container; and

generating a second identifier that points to the first data object that is written in the target container.

6. The method of claim 5, further comprising marking the first identifier of the first data object within the first logical container as being migrated to the target container.

7. The method of claim 5, wherein reading the first object from the physical location within the first logical container comprises decoding the first identifier of the first data object to reference the physical location of the first data object within the first logical container.

8. The method of claim 7, wherein decoding the first identifier of the first data object comprises:

decoding a container identifier of the first identifier to locate the physical location of the first logical container within which the first data object is stored; and

decoding an object index value of the first identifier to locate a start of the first data object within the first logical container.

9. The method of claim 5, wherein writing the first data object to the physical location within the target container comprises decoding the second identifier of the first data object to reference the physical location where the first data object is to be written within the target container.

10. The method of claim 9, wherein decoding the second identifier of the first data object comprises:

decoding a container identifier of the second identifier to locate the physical location of the target container within which the first data object is to be written; and

decoding an object index value of the second identifier to locate a start of the first data object within the second logical container.

11. The method of claim 4, wherein migrating the second data object comprises:

reading the second data object from a physical location of the second data object within the second logical container using a first identifier;

writing the second data object into a physical location of the target container; and

generating a second identifier that points to the second data object that is written in the target container.

12. The method of claim 11, further comprising marking the first identifier of the second data object within the second logical container as being migrated to the target container.

13. The method of claim 1, wherein merging the first and second logical containers further comprises making a dual entry in the container map to point to a physical location of the first logical container and a physical location of the target container, and wherein all write access requests to the identifier of the first data object that is located in the first logical container is mirrored in the physical locations of the first data object within the first logical container and within the target container.

14. The method of claim 1, further comprising backing up the target container and not the first and second logical containers after merging the data of the first and second logical containers into the target container.

15. A storage server, comprising:

a processor;

a first communication interface through which to communicate with a plurality of storage devices;

a second communication interface through which to communicate with a client of the storage server;

a file system; and

a memory storing instructions which configure the processor to merge data of a first logical container and a second logical container into a target container that includes at least one block that includes a plurality of extents to store data objects, by;

migrating a first data object within the first logical container to a first physical location of the target container;

migrating a second data object within the second logical container to a second physical location of the target container, wherein, before the merge, the first and second data objects are stored at a block level in the first and second logical containers respectively in an existing file system whose internal block fragmentation dictates the size of each block, and wherein, after the merge, the first and second data objects are stored at an extent level within the target container after said merging, wherein the extent is a smaller unit of storage space than the block;

updating a container map to indicate that the first data object and the second data object are stored in the target container, wherein the container map includes a plurality of mapping entries that map filenames to data object identifiers, wherein each of the data object identifiers identifies a corresponding data object and includes;

a container identifier to specify a logical container in which that corresponding data object is stored; and

an object index value used to identify which of one or more data objects stored in that logical container is that corresponding data object; and

updating an object map stored within the target container to indicate extent locations within the target container of the first data object and the second data object, wherein the object map comprises a listing of a plurality of data objects stored in the target container and extent locations within the target container where the plurality of data objects are stored.

16. The storage server of claim 15, wherein the target container is one of the first and second logical containers.

17. The storage server of claim 15, wherein the target container is a third logical container.

18. The storage server of claim 15, wherein migrating the first data object comprises:

reading the first data object from a physical location of the first data object within the first logical container using a first identifier;

writing the first data object into a physical location of the target container; and

generating a second identifier that points to the first data object that is written in the target container.

19. The storage server of claim 18, wherein reading the first object from the physical location within the first logical container comprises:

decoding a container identifier of the first identifier to locate the physical location of the first logical container within which the first data object is stored; and

decoding an object index value of the first identifier to locate a start of the first data object within the first logical container.

20. The storage server of claim 15, wherein migrating the second data object comprises:

reading the second data object from a physical location of the second data object within the second logical container using a first identifier;

writing the second data object into a physical location of the target container; and

generating a second identifier that points to the second data object that is written in the target container.

21. The storage server of claim 20, wherein writing the first data object to the physical location within the target container comprises:

decoding a container identifier of the second identifier to locate the physical location of the target container within which the first data object is to be written; and

decoding an object index value of the second identifier to locate a start of the first data object within the second logical container.

22. An article of manufacture comprising one or more non-transitory storage media storing instructions which, when executed by a system, cause the system to perform operations comprising:

identifying within a file system a first logical container and a second logical container for merging, wherein the second logical container includes at least one block and has some empty storage space, and wherein each block comprises a plurality of extents to store data objects;

referencing the physical location of data objects stored at a block level in the first and second logical containers in the file system whose internal block fragmentation dictates the size of each block; and

migrating the data object stored in the first logical container into the empty storage space of the second logical container, wherein the data objects stored in the second logical container are stored at an extent level within the second logical container after said migrating, wherein the extent is a smaller unit of storage space than the block, and wherein migrating further comprises;

updating an object map stored within the second logical container, wherein the object map comprises a listing of a plurality of data objects stored in the second logical container and extent locations within the second logical container where the plurality of data objects are stored; and

updating a container map to indicate that the data object is stored in the second logical container, wherein the container map includes a plurality of mapping entries that map filenames to data object identifiers, wherein each of the data object identifiers identifies a corresponding data object and includes;

a container identifier that specifies a logical container in which that corresponding data object is stored; and

an object index value used to identify which of the one or more data objects stored in that logical container is that corresponding data object.

23. The article of claim 22, wherein the one or more migrated data objects each comprise an object index value represented in an object map in the second logical container and a physical location within the second logical container.

24. The article of claim 22, wherein migrating the one or more data objects of the first logical container into empty space of the second logical container comprises:

reading a first data object from a physical location of the first logical container using a container identifier in the container map; and

writing the first data object to a physical location within the second logical container that corresponds to the empty storage space.

25. An apparatus, comprising:

a storage server to store a plurality of data objects in a file system comprising a processor coupled to a memory, wherein the plurality of data objects are stored at a block level in the file system whose internal block fragmentation dictates the size of each block, wherein the storage server is configured to merge the plurality of data objects into a target container from a plurality of logical containers that store the plurality of data objects, one data object per each logical container, wherein the target container comprises at least one block, and wherein each block of the target container comprises a plurality of extents to store data objects, wherein the plurality of data objects are stored at an extent level after the merge, wherein the extent is a smaller unit of storage space than the block, wherein the target container contains an object map, wherein the object map comprises a listing of a plurality of data objects stored in the target container and extent locations within the target container where the plurality of data objects are stored, and wherein the storage server is further configured to update a container map to indicate that the merged plurality of data objects are stored in the target container, wherein the container map includes a plurality of mapping entries that map filenames to data object identifiers, wherein each of the data object identifiers identifies a corresponding data object and includes;

a container identifier that specifies a logical container in which that corresponding data object is stored, and

an object index value used to identify which of the one or more data objects stored in that logical container is that corresponding data object.

26. The apparatus of claim 25, wherein the target container is one block in size.

27. The apparatus of claim 25, wherein the container map includes a plurality of mapping entries with different filenames mapped to data object identifiers including a same container identifier.