Data containerization for reducing unused space in a file system

• US 7,739,312 B2
• Filed: 04/27/2007
• Issued: 06/15/2010
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Gordon, Colin Stebbins, Singh, Pratap Vikram, Trimmer, Donald Alvin
• Primary Examiner(s): Alam; Shahid A
• Assistant Examiner(s): Nguyen; Phong
• Application Number:
  US11/796,268
• Publication Number:
  US 20080270461A1
• Time in Patent Office: 1,145 Days
• Field of Search: 707/103.R, 707/204, 707/205, 707/999.103, 707/999.204, 707/999.205, 711/173, 711/152, 711/170
• US Class Current: 707/802
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

An apparatus and method for containerization of multiple data objects within a block of a single container. The apparatus and method may pack multiple data objects together in a block of a logical container in a file system. The method may include receiving data in the form of multiple data objects to be stored in a file system, and collectively data packing the multiple data objects together in at least one block of a logical container in the file system. The block is a fundamental unit of storage space of the file system, and each block of the logical container includes multiple extents to store data from at least one data object of the multiple objects. The apparatus may include a plurality of storage devices coupled to a storage server. The storage server is configured to store the multiple data objects in at least one block of the logical container in the file system. Some data objects may be stored in multiple extents of one or more blocks depending on the size of the data object.

Claims

1. A machine-readable storage medium including instructions that when executed by a machine perform a method, comprising:

receiving data to be stored in a file system, wherein the data comprises a plurality of data objects; and

collectively data packing at least two of the plurality of data objects together in at least one block of a logical container in the file system, wherein the block is a fundamental unit of storage space of the file system, and wherein each block of the logical container comprises a plurality of extents to store data from at least one data object of the plurality of data objects, wherein the extent is a smaller unit of storage space than the block, wherein receiving the data to be stored comprises;

receiving a first data object of the plurality of data objects to be stored in a first block of the logical container; and

receiving a second data object of the plurality of data objects to be stored in the first block of the logical container, and wherein collectively data packing comprises;

storing the first data object in a portion of the plurality of extents of the first block, wherein storing the first data object comprises storing the first data object in the portion of the plurality of extents of the first block as if the portion of the plurality of extents of the first block were a single container in the file system having a current block size that is smaller than the block size according to an internal block fragmentation of the file system; and

storing the second data object in any remaining extents of the first block of the logical container that do not contain data of the first data object.

2. The machine-readable storage medium of claim 1, wherein collectively data packing comprises storing the plurality of data objects in the first block of the logical container.

3. The machine-readable storage medium of claim 1, wherein collectively data packing comprises storing the plurality of data objects in the first block and a second block of the logical container.

4. A method of operating a storage server, the method comprising:

receiving data to be stored in a file system, wherein the data comprises a plurality of data objects; and

collectively data packing the plurality of data objects together in at least one block of a logical container in the file system, wherein the block is a fundamental unit of storage space of the file system, and wherein each block of the logical container comprises a plurality of extents to store data from at least one data object of the plurality of data objects, wherein the extent is a smaller unit of storage space than the block, wherein receiving the data to be stored comprises;

receiving a first data object of the plurality of data objects to be stored in a first block of the logical container in the file system;

receiving a second data object of the plurality of data objects to be stored in the first block of the logical container, and wherein collectively data packing comprises;

storing the first data object in a portion of the plurality of extents of the first block, wherein storing the first data object comprises storing the first data object in the portion of the plurality of extents of the first block as if the portion of the plurality of extents of the first block were a single container in the file system having a current block size that is smaller than the block size according to an internal block fragmentation of the file system; and

storing the second data object in any remaining extents of the first block of the logical container that do not contain data of the first data object.

5. The method of claim 4, further comprising:

tracking a block size of each block of the logical containers in the file system; and

dividing each block of the logical containers into the plurality of extents of a given size to store the plurality of data objects, wherein the size of each extent is smaller than the block size.

6. The method of claim 4, wherein storing the first data

object further comprises tracking a physical location of the first data object in the first block of the logical container using one or more metadata files.

7. The method of claim 4, wherein storing the second data object further comprises storing a remaining portion of the data of the second data object that is not stored in the remaining extents of the first block into a portion of the plurality of extents of a second block of the logical container.

8. The method of claim 4, wherein storing the second data object further comprises tracking a physical location of the second data object in the first block of the logical container using one or more metadata files.

9. The method of claim 4, wherein storing the second data object further comprises tracking a physical location of the second data object in the first block and the second block of the logical container using one or more metadata files.

10. The method of claim 4, further comprising:

receiving a plurality of additional data objects to be stored in the logical container in the file system;

storing the plurality of additional data objects in any remaining extents of the first block of the logical container that do not contain data of the first data object; and

storing any remaining data of the plurality of additional data objects that is not data packed into the first block of the logical container together in one or more additional blocks of the logical container.

11. The method of claim 10, further comprising tracking physical locations of the additional data objects in the first block and the one or more additional blocks of the logical container using one or more metadata files.

12. The method of claim 4, wherein the block size is 4 Kbytes, and the extent size is less than the block size.

13. An apparatus, comprising

a storage server to store a plurality of data objects, and

wherein the storage server is configured to store at least two of the plurality of data objects in a block of a logical container in the file system, wherein the block is a fundamental unit of storage space of the file system, wherein each block comprises a plurality of extents to store data from at least one data objects of the plurality of data objects, wherein the extent is a smaller unit of storage space than the block, wherein the storage server is further configured to receive a first data object of the plurality of data objects to be stored in a first block of the logical container and to store the first data object in a portion of the plurality of extents of the first block as if the portion of the plurality of extents of the first block were a single container in the file system having a current block size that is smaller than the block size according to an internal block fragmentation of the file system, and

wherein the storage server is further configured to receive a second data object of the plurality of data objects to be stored in the first block and to store the second data object in any remaining extents of the first block that do not contain data of the first data object.

14. The apparatus of claim 13, wherein the logical container comprises one or more blocks having a block size according to the internal block fragmentation of the file system.

15. The apparatus of claim 13, wherein the storage server is configured to receive the plurality of data objects from a client.

16. The apparatus of claim 13, wherein the storage server is configured to read already existing containers each having zero or more data objects and to store the zero or more data objects of the already existing containers in the logical container.

17. The apparatus of claim 13, further comprising a client coupled to the storage server, and wherein the client comprises an application that is configured to access a data object of the plurality of data object in the file system using a Portable Operating System Interface for UNIX (POSIX) file input-output (I/O) call, and a library linked to the application to convert the POSIX file I/O call into a containerization call to access the data object that is stored in the block of the logical container in the file system.

18. The apparatus of claim 17, wherein the library comprises a container map, wherein the container map comprises a mapping between the data object and a physical location of the logical container that stores the data object.

19. The apparatus of claim 18, wherein the container comprises an object map, wherein the object map comprises an object index that represents a physical location of the data object within the logical container.

20. The apparatus of claim 19, wherein the object index includes a value that represents an offset in the logical container of where the data object starts and a value that represents a size of the data object.

21. The apparatus of claim 17, wherein the library comprises a container map, wherein the container map comprises an identifier that represents a physical location of the logical container.

22. The apparatus of claim 17, wherein the container map includes mappings to a plurality of logical containers in the file system, and wherein the plurality of logical containers each comprise an object map that includes mappings to a plurality of data objects stored with each of the plurality of logical containers.

23. The apparatus of claim 22, wherein the container map comprises a plurality of identifiers that represents physical locations of the plurality of logical containers, and wherein each of the object maps comprises a plurality of object indexes that represent physical locations of the plurality of data objects stored within each of the plurality of logical containers.

24. The apparatus of claim 13, wherein the storage server is configured to store the plurality of data objects on an extent-size basis, wherein the extent-size basis is smaller than a block-size basis.

25. The apparatus of claim 13, wherein the block size is 4 Kbytes, and the extent size is less than the block size.

26. A storage server, comprising:

a processor;

a first communication interface through which to communicate with a plurality of storage devices;

a second communication interface through which to communicate with a client of the storage server;

a file system;

a memory storing instructions which configure the processor to collectively data pack a first data object and a second data object in a logical container in the file system by;

receiving the first and second data objects to be stored in the logical container, wherein the logical container comprises at least one block, wherein the block is a fundamental unit of storage space of the file system, and each block of the logical container includes a plurality of extents and a fixed block size, and wherein an extent size of each of the plurality of extents is less than the fixed block size;

storing the first data object in a first set of extents of the plurality of extents in the logical container, wherein storing the first data object comprises storing the first data object in the first set of extents as if the first set of extents of the first block were a single container in the file system having a current block size that is smaller than the block size according to an internal block fragmentation of the file system; and

storing the second data object in a second set of extents of the plurality of extents in the logical container.

27. The storage server of claim 26, wherein storing the first and second data object comprises:

tracking a physical location of the first data object within the logical container using a metadata file; and

tracking a physical location of the second data object within the logical container using a metadata file.

28. The storage server of claim 26, wherein the logical container comprises a plurality of blocks, and wherein storing the first and second data objects comprises:

receiving one or more additional data objects;

dividing each of the plurality of blocks into a plurality of extents of a given size to store data; and

filling the plurality of extents of a first block of the plurality of blocks with data of the first and second data objects and the one or more additional data objects until the first block is full.

29. The storage server of claim 26, wherein storing the first and second data object further comprises filling the plurality of extents of a second block of the plurality of blocks with data of the one or more additional data objects when the first block is full.

30. The storage server of claim 26, wherein data packing further comprises tracking physical locations of the first data object, the second data object, and the one or more data object in the first and second blocks using one or more metadata files.

31. The storage server of claim 26, further comprising means for reducing an amount of unused storage space in the file system due to the internal block size when the sizes of the first and second data object are smaller than a block size of the internal block size of the file system.

32. The storage server of claim 26, further comprising means for reducing an amount of storage space in the file system used for storing metadata of the file system.