Virtual block mapping for relocating compressed and/or encrypted file data block blocks

• US 8,190,850 B1
• Filed: 10/01/2009
• Issued: 05/29/2012
• Est. Priority Date: 10/01/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Armangau, Philippe, Veeraswamy, Sairam, Bono, Jean-Pierre, Davenport, William C., Wang, Yubing
• Primary Examiner(s): Farrokh, Hashem
• Application Number:
  US12/571,007
• Time in Patent Office: 971 Days
• Field of Search: 711/202, 711/170
• US Class Current: 711/202
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

This invention is a system and a method for operating a storage server to provide read or write access to a data in a data network using a new architecture. The method of creating virtual block mapping pointer in response to a request by a client of the storage server to de-duplicate the file system data block or to allow compression of one or more file system data blocks into one or more physical data blocks. Further, the method relocates one or more file system data blocks from one part of the file system address space to another by using one or more virtual block mapping pointers that provides the mapping information for the one or more file system data blocks that are being relocated. The virtual block mapping pointer allows relocating of file system data blocks by same number of metadata operations regardless of number of files sharing the block that are being relocated and the state of those blocks (compressed or not).

Claims

1. A method of operating a storage server in a data network for relocating one or more data blocks of a file, the file including multiple data blocks containing data of the file, and the file further being organized as a hierarchy of file system blocks including inodes, indirect blocks, and one or more data blocks, the file stored in a data storage including a production file having a first data block in the hierarchy of file system blocks, the first data block being a child of the production file inode in the hierarchy of file system blocks, the method comprising the steps of:

creating a virtual block mapping pointer in response to a request by a client of the storage server to de-duplicate the file system data block or to create a snapshot copy of the production file, wherein the virtual block mapping pointer provides a mapping information to a logical block storing data of the file system block of the production file, wherein metadata of the file system data block includes a pointer pointing back to the metadata of the virtual block mapping pointer; and

relocating one or more file system data blocks from one part of the file system address space to another by using the virtual block mapping pointer providing the mapping information for the one or more file system data blocks containing compressed, encrypted or de-duplicated data;

wherein the metadata of the virtual block mapping pointer includes a total distributed weight, wherein the total distributed weight indicates the total reference count of all the file system data blocks that points to the virtual block mapping pointer.

2. The method of claim 1, wherein the storage server is represented by a file server, wherein the file server presents a logical view of a file to a client of the file server, wherein the data of the file can be compressed or encrypted.

3. The method of claim 1, wherein the metadata of the virtual block mapping pointer includes a total distributed weight, wherein the total distributed weight indicates the total reference count of all the file system data blocks that points to the virtual block mapping pointer.

4. The method of claim 1, wherein the metadata of the virtual block mapping pointer further includes a mapping bit wherein the mapping bit indicates whether the virtual block mapping pointer points to another virtual block mapping pointer or a file system data block.

5. The method of claim 1, wherein the metadata of the virtual block mapping pointer further includes a data bit wherein the data bit indicates whether the data stored on the file system data block is compressed or encrypted.

6. The method of claim 1, wherein the metadata of the virtual block mapping pointer further includes a network IP address of a host machine, wherein the host machine stores the file system data block that is referenced by the virtual block mapping pointer.

7. The method of claim 1, wherein the metadata of the virtual block mapping pointer further includes a block type field wherein the block type indicates whether the virtual block mapping pointer is a regular virtual block mapping pointer, a compressed virtual block mapping pointer, a regular extended virtual block mapping pointer or a network virtual block mapping pointer.

8. The method of claim 1, wherein creating a virtual block mapping pointer comprises the steps of:

allocating a virtual block mapping pointer;

changing the mapping information of a de-duplicated file system data block to point to the virtual block mapping pointer;

incrementing the total distributed weight of the virtual block mapping pointer by the weight of the file system data block that points to the virtual block mapping pointer;

changing the mapping bit of the virtual block mapping pointer to indicate the type of the block;

changing the data bit of the virtual block mapping pointer to indicate if the data is compressed or encrypted; and

changing the metadata of the virtual block mapping pointer to include the mapping information to the file system block of the production file.

9. The method of claim 1, wherein de-duplication of a file system data block further comprises the steps of:

checking whether metadata of the file system data block indicates the block type as a virtual block mapping pointer and upon finding that the metadata of the file system data block does not point to a virtual block mapping pointer, performing the look-up operation to find the virtual block mapping pointer;

creating a new virtual block mapping block upon not finding the virtual block mapping pointer during the look-up operation and adding the virtual block mapping block to an encryption list, wherein, metadata of the file system data block is updated to point to the new virtual block mapping block; and

upon finding the virtual block mapping pointer during the look-up operation, updating the metadata of the file system data block to point to the virtual block mapping pointer and updating the metadata of the virtual block mapping pointer to point to the file system data block being de-duplicated.

10. A method of claim 1, wherein relocating one or more file system data block comprises the steps of:

locating the metadata of the one or more file system data block that is being relocated to find a pointer referring back to the virtual block mapping pointer providing mapping information to the file system data block that is being relocated;

copying the content of the file system data block from a source logical address to a target block address;

changing the metadata of the virtual block mapping pointer to point to the file system data block at the target block address; and

freeing the file system data block at the source address.

11. The system of claim 1, wherein the metadata of the virtual block mapping pointer further includes a mapping bit wherein the mapping bit indicates whether the virtual block mapping pointer points to another virtual block mapping pointer or a file system data block.

12. The system of claim 1, wherein the metadata of the virtual block mapping pointer further includes a data bit wherein the data bit indicates whether the data stored on the file system data block is compressed or encrypted.

13. The system of claim 1, wherein the metadata of the virtual block mapping pointer further includes a block type field wherein the block type indicates whether the virtual block mapping pointer is a regular virtual block mapping pointer, a compressed virtual block mapping pointer, a regular extended virtual block mapping pointer or a network virtual block mapping pointer.

14. A method of operating a storage server in a data network for read or write access to data of a file, the file including multiple data blocks containing data of the file, and the file further being organized as a hierarchy of file system blocks including inodes, indirect blocks, and data blocks, the file stored in a data storage including a production file having a first data block in the hierarchy of file system blocks, the first data block being a child of the production file inode in the hierarchy of file system blocks, the method comprising the steps of:

creating a first virtual block mapping pointer in response to a request by a client of the storage server to de-duplicate the file system data block or to create a snapshot copy of the production file, wherein the first virtual block mapping pointer provides a mapping information to a logical block storing data of the file system block of the production file, wherein metadata of the file system data block includes a pointer pointing back to the metadata of the first virtual block mapping pointer; and

compressing one or more file system data block into one or more physical data block by creating a second virtual block mapping pointer wherein the second virtual block mapping pointer provides the mapping information to the physical data block storing compressed data of the file system block of the production file and first virtual block mapping pointer provides the mapping information to the second virtual block mapping pointer, wherein metadata of the physical data block includes a pointer pointing back to the metadata of the second virtual block mapping pointer;

wherein the metadata of the virtual block mapping pointer includes a total distributed weight, wherein the total distributed weight indicates the total reference count of all the file system data blocks that points to the virtual block mapping pointer.

15. The method of claim 14, wherein the storage server is represented by a file server, wherein the file server presents a logical view of a file to a client of the file server, wherein the data of the file can be compressed or encrypted.

16. The method of claim 14, wherein the metadata of the virtual block mapping pointer further includes a mapping bit wherein the mapping bit indicates whether the virtual block mapping pointer points to another virtual block mapping pointer or a file system data block.

17. The method of claim 14, wherein the metadata of the virtual block mapping pointer further includes a data bit wherein the data bit indicates whether the data stored on the file system data block is compressed or encrypted.

18. The method of claim 14, wherein the metadata of the virtual block mapping pointer further includes a network IP address of a host machine, wherein the host machine stores the file system data block that is referenced by the virtual block mapping pointer.

19. The method of claim 14, wherein the metadata of the virtual block mapping pointer further includes a block type field wherein the block type indicates whether the virtual block mapping pointer is a regular virtual block mapping pointer, a compressed virtual block mapping pointer, a regular extended virtual block mapping pointer or a network virtual block mapping pointer.

20. The method of claim 14, wherein creating a virtual block mapping pointer comprises the steps of:

allocating a virtual block mapping pointer;

changing the mapping information of a de-duplicated file system data block to point to the virtual block mapping pointer;

incrementing the total distributed weight of the virtual block mapping pointer by the weight of the file system data block that points to the virtual block mapping pointer;

changing the mapping bit of the virtual block mapping pointer to indicate the type of the block;

changing the data bit of the virtual block mapping pointer to indicate if the data is compressed or encrypted; and

changing the metadata of the virtual block mapping pointer to include the mapping information to the file system block of the production file.

21. The method of claim 14, wherein de-duplication of a file system data block further comprises the steps of:

checking whether metadata of the file system data block indicates the block type as a virtual block mapping pointer and upon finding that the metadata of the file system data block does not point to a virtual block mapping pointer, performing the look-up operation to find the virtual block mapping pointer;

creating a new virtual block mapping block upon not finding the virtual block mapping pointer during the look-up operation and adding the virtual block mapping pointer to an encryption list, wherein, metadata of the file system data block is updated to point to the new virtual block mapping block; and

upon finding the virtual block mapping pointer during the look-up operation, updating the metadata of the file system data block to point to the virtual block mapping pointer and updating the metadata of the virtual block mapping pointer to point to the file system data block being de-duplicated.

22. A system for accessing a data of a file in a data storage environment, the system comprising:

a data storage storing files, wherein the files stored on the data storage includes a production file having a first data block in the hierarchy of file system blocks, the first data block being a child of the production file inode in the hierarchy of file system blocks;

a storage server, wherein the storage server access the files stored on the data storage, wherein the file includes multiple data blocks containing data of the file, and the file further being organized as a hierarchy of file system blocks including inodes, indirect blocks, and data blocks; and

a program logic in communication with the data storage and the storage server for carrying out the steps of;

creating a first virtual block mapping pointer in response to a request by a client of the storage server to de-duplicate the file system data block or to create a snapshot copy of the production file, wherein the first virtual block mapping pointer provides a mapping information to a logical block storing data of the file system block of the production file, wherein metadata of the file system data block includes a pointer pointing back to the metadata of the first virtual block mapping pointer;

relocating one or more file system data blocks from one part of the file system address space to another by using the virtual block mapping pointer providing the mapping information for the one or more file system data blocks containing compressed, encrypted or de-duplicated data; and

compressing one or more file system data block into one or more physical data block by creating a second virtual block mapping pointer wherein the second virtual block mapping pointer provides the mapping information to the physical data block storing compressed data of the file system block of the production file and first virtual block mapping pointer provides the mapping information to the second virtual block mapping pointer, wherein metadata of the physical data block includes a pointer pointing back to the metadata of the second virtual block mapping pointer;

wherein the metadata of the virtual block mapping pointer includes a total distributed weight, wherein the total distributed weight indicates the total reference count of all the file system data blocks that points to the virtual block mapping pointer.

23. A program product for accessing data of a file, the program product operating in a data storage environment that includes a data storage system in communication with a storage server providing logical disk storage to the storage server for storing files that includes multiple data blocks containing data of the files, and the file further being organized as a hierarchy of file system blocks including inodes, indirect blocks, and data blocks, wherein the program product includes computer-executable logic encoded on a computer-readable medium for executing the following steps:

creating a first virtual block mapping pointer in response to a request by a client of the storage server to de-duplicate the file system data block or to create a snapshot copy of the production file, wherein the first virtual block mapping pointer provides a mapping information to a logical block storing data of the file system block of the production file, wherein metadata of the file system data block includes a pointer pointing back to the metadata of the first virtual block mapping pointer;

relocating one or more file system data blocks from one part of the file system address space to another by using the virtual block mapping pointer providing the mapping information for the one or more file system data blocks containing compressed, encrypted or de-duplicated data; and

compressing one or more file system data block into one or more physical data block by creating a second virtual block mapping pointer wherein the second virtual block mapping pointer provides the mapping information to the physical data block storing compressed data of the file system block of the production file and first virtual block mapping pointer provides the mapping information to the second virtual block mapping pointer, wherein metadata of the physical data block includes a pointer pointing back to the metadata of the second virtual block mapping pointer;

wherein the metadata of the virtual block mapping pointer includes a total distributed weight, wherein the total distributed weight indicates the total reference count of all the file system data blocks that points to the virtual block mapping pointer.