Method and system supporting per-file and per-block replication

• US 7,873,799 B2
• Filed: 04/19/2006
• Issued: 01/18/2011
• Est. Priority Date: 11/04/2005
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Oracle America, Inc. (Oracle Corporation)
• Inventors: Bonwick, Jeffrey S., Moore, William H.
• Primary Examiner(s): Kim; Matt
• Assistant Examiner(s): Chrzanowski; Matthew R
• Application Number:
  US11/406,850
• Publication Number:
  US 20070106851A1
• Time in Patent Office: 1,735 Days
• Field of Search: None
• US Class Current: 711/154
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 2211/105 On the fly coding, e.g. using XOR accumulators
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A method for writing data to a storage pool, involving receiving a first write operation to write a first block to the storage pool, determining a first replication type for the first block using a first replication policy associated with the first block, determining a first number of physical blocks required to write the first block to the storage pool using a size of the first block and the first replication type, and writing the first block, in accordance with the first replication policy, to the storage pool by filling in the first number of physical blocks.

Claims

1. A method for writing data to a storage pool, comprising:

receiving a first write operation to write a first data block to the storage pool, wherein the storage pool is organized in a hierarchical tree structure comprising data blocks and indirect blocks located hierarchically above the data blocks;

determining a first replication type for the first data block using a first replication policy associated with the first data block;

determining a first number of physical blocks required to write the first data block to the storage pool using a size of the first data block and the first replication type;

writing the first data block, in accordance with the first replication policy, to the storage pool by filling in the first number of physical blocks,

wherein the first replication type for the first data block is stored in a block pointer of a first indirect block referencing the first data block in the hierarchical tree structure;

receiving a second write operation to write the first indirect block;

determining a second replication type for the first indirect block using the first replication policy associated with the indirect block;

determining a second number of physical blocks required to write the first indirect block using a size of the first indirect block and the second replication type;

writing the first indirect block, in accordance with the first replication policy by filling in the second number of physical blocks in the storage pool,

wherein the second replication type and the size for the first indirect block is stored in a block pointer of a second indirect block referencing the first indirect block.

2. The method of claim 1, further comprising:

modifying the first replication policy associated with the first block to obtain a second replication policy, wherein the second replication policy specifies a third replication type.

3. The method of claim 2, further comprising:

receiving a third write operation to write a modified version of the first data block to the storage pool;

determining the third replication type for the modified version of first data block using the second replication policy associated with the first data block;

determining a third number of physical blocks required to write the modified version of the first data block to the storage pool using a size of the modified version of the first data block and the third replication type; and

writing the modified version of the first data block, in accordance with the second replication policy, to the storage pool by filling in the third number of physical blocks.

4. The method of claim 2, further comprising:

initiating a background process in response to modifying the first replication policy, wherein the background process is configured to;

obtain the first data block from the storage pool, and

re-write the first data block to the storage pool using the second replication policy.

5. The method of claim 1, wherein the first replication policy corresponds to one selected from the group consisting of:

a block level replication policy, a file level replication policy, and a file system replication policy.

18. The method of claim 1, wherein the first number of physical blocks required to write the first data block to the storage pool is a multiple of the maximum supported replication level of a file system associated with the storage pool.

6. A system for writing data, comprising:

a storage pool comprising;

a plurality of child blocks, wherein each of the plurality of child blocks comprises one selected from the group consisting of a data block and an indirect block, wherein each of the indirect blocks reference at least one of the plurality of child blocks and is hierarchically above the at least one of the plurality of child blocks referenced by each indirect block;

a parent block referencing at least one child block that is hierarchically above the at least one child block; and

a file system configured to;

write a first block to the storage pool, wherein the first block is one of the plurality of child blocks, wherein the file system writes the first block to the storage pool using the following method steps;

determining a first replication type for the first block using a first replication policy associated with the first block,

determining a first number of physical blocks required to write the first block to the storage pool using a size of the first block and the first replication type,

writing the first block, in accordance with the first replication policy, to the storage pool by filling in the first number of physical blocks, wherein the first replication type for the first data block is stored in a block pointer of a first indirect block referencing the first block; and

write the first indirect block to the storage pool by;

determining a second replication type for the first indirect block using the first replication policy associated with the first indirect block,

determining a second number of physical blocks required to write the first indirect block using a size of the first indirect block and the second replication type,

writing the first indirect block, in accordance with the first replication policy by filling in the second number of physical blocks in the storage pool,

wherein the second replication type and the size for the first indirect block is stored in a block pointer of a second indirect block referencing the first indirect block.

7. The system of claim 6, wherein the file system is further configured to:

modify the first replication policy associated with the first block to obtain a second replication policy, wherein the second replication policy specifies a third replication type.

8. The system of claim 7, wherein the file system is further configured to:

receive a second write operation to write a modified version of the first block to the storage pool;

determine the third replication type for the modified version of first block using the second replication policy associated with the first block;

determine a third number of physical blocks required to write the modified version of the first block to the storage pool using a size of the modified version of the first block and the third replication type; and

write the modified version of the first block, in accordance with the second replication policy, to the storage pool by filling in the third number of physical blocks.

9. The system of claim 7, wherein the file system is further configured to:

receive a second write operation to write a second block to the storage pool;

determine the third replication type for the second block using the second replication policy, wherein the second block is associated with the second replication policy;

determine a third number of physical blocks required to write the second block to the storage pool using a size of the second block and the third replication type; and

write the second block, in accordance with the second replication policy, to the storage pool by filling in the third number of physical blocks.

10. The system of claim 7, wherein the file system is further configured to:

initiate a background process in response to modifying the first replication policy, wherein

the background process is configured to;

obtain the first block from the storage pool, and

re-write the first block to the storage pool using the second replication policy.

11. The system of claim 6, wherein the first replication policy corresponds to one selected from the group consisting of:

a block level replication policy, a file level replication policy, and a file system replication policy.

19. The system of claim 6, wherein the first number of physical blocks required to write the first data block to the storage pool is a multiple of the maximum supported replication level of the file system.

12. A computer readable medium comprising computer readable program code embodied therein for causing a computer system to:

determine a first replication type for the first block using a first replication policy associated with the first block, wherein the first block is a data block;

determine a first number of physical blocks required to write the first block to the storage pool using a size of the first block and the first replication type;

write the first block, in accordance with the first replication policy, to the storage pool by filling in the first number of physical blocks,

wherein the storage pool is organized in a hierarchical tree structure comprising data blocks and indirect blocks located hierarchically above the data blocks, and

wherein the first replication type for the first data block is stored in a block pointer of a first indirect block referencing the first data block in the hierarchical tree structure;

determining a second replication type for the first indirect block using the first replication policy associated with the first indirect block;

determining a second number of physical blocks required to write the first indirect block using a size of the first indirect block and the second replication type; and

writing the first indirect block, in accordance with the first replication policy by filling in the second number of physical blocks in the storage pool,

wherein the second replication type and the size of the first indirect block is stored in a block pointer of a second indirect block referencing the first indirect block.

13. The computer readable medium of claim 12, further comprising computer readable program code embodied therein for causing the computer system to:

modify the first replication policy associated with the first block to obtain a second replication policy, wherein the second replication policy specifies a third replication type.

14. The computer readable medium of claim 13, further comprising computer readable program code embodied therein for causing the computer system to:

receive a second write operation to write a modified version of the first block to the storage pool;

determine the third replication type for the modified version of first block using the second replication policy associated with the first block;

determine a third number of physical blocks required to write the modified version of the first block to the storage pool using a size of the modified version of the first block and the third replication type; and

write the modified version of the first block, in accordance with the second replication policy, to the storage pool by filling in the third number of physical blocks.

15. The computer readable medium of claim 13, further comprising computer readable program code embodied therein for causing the computer system to:

receive a second write operation to write a second block to the storage pool;

determine the third replication type for the second block using the second replication policy, wherein the second block is associated with the second replication policy;

determine a third number of physical blocks required to write the second block to the storage pool using a size of the second block and the third replication type; and

write the second block, in accordance with the second replication policy, to the storage pool by filling in the third number of physical blocks.

16. The computer readable medium of claim 13, further comprising computer readable program code embodied therein for causing the computer system to:

initiate a background process in response to modifying the first replication policy, wherein the background process is configured to;

obtain the first block from the storage pool, and

re-write the first block to the storage pool using the second replication policy.

17. The computer readable medium of claim 12, wherein the first replication policy corresponds to one selected from the group consisting of:

a block level replication policy, a file level replication policy, and a file system replication policy.