Method and apparatus for vectored block-level checksum for file system data integrity

• US 7,415,653 B1
• Filed: 04/21/2004
• Issued: 08/19/2008
• Est. Priority Date: 04/21/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Ahrens, Matthew A., Bonwick, Jeffrey S.
• Primary Examiner(s): Lamarre; Guy J.
• Application Number:
  US10/828,715
• Time in Patent Office: 1,581 Days
• Field of Search: 714/763, 714/773, 714769-771, 711/162
• US Class Current: 714/763
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00

Abstract

A method for storing a data block, involving storing the data block in a storage pool, obtaining a data block location, determining a checksum function for the data block, calculating a data block checksum using the checksum function for the data block, and storing a first indirect block in the storage pool, wherein the first indirect block comprises the data block location, the data block checksum, and a checksum function ID corresponding to the checksum function for the data block.

Claims

1. A method for storing a data block, comprising:

storing the data block in a storage pool;

obtaining a data block location associated with the data block;

determining a checksum function for the data block;

calculating a data block checksum using the checksum function for the data block;

storing a first indirect block in the storage pool, wherein the first indirect block comprises the data block location, the data block checksum, and a checksum function ID corresponding to the checksum function for the data block;

determining a checksum function for the first indirect block, wherein the checksum function for the first indirect block is associated with a checksum function ID corresponding to the checksum function for the first indirect block;

calculating a first indirect block checksum using the checksum function for the first indirect block;

obtaining a first indirect block location associated with the first indirect block; and

storing a second indirect block in the storage pool, wherein the second indirect block comprises the first indirect block location, the first indirect block checksum, and the checksum function ID corresponding to the checksum function for the first indirect block.

2. The method of claim 1, further comprising:

assembling the first indirect block, wherein assembling the first indirect block comprises populating a block pointer in the first indirect block.

3. The method of claim 2, wherein populating the block pointer comprises:

storing the data block checksum in a checksum field in the block pointer,

storing the checksum function ID for the data block in a checksum function ID field in the block pointer; and

storing the data block location in the block pointer, wherein storing the data block location comprises storing a metaslab ID and offset.

4. The method of claim 3, further comprising:

storing a birth value in a birth field in the block pointer.

5. The method of claim 2, wherein the first indirect block is assembled using a data management unit.

6. The method of claim 1, wherein the storage pool comprises at least one storage device.

7. The method of claim 1, wherein the storage pool is divided into a plurality of metaslabs.

8. The method of claim 7, wherein each of the plurality of metaslabs is associated with a metaslab ID.

9. The method of claim 7, wherein the data block location comprises the metaslab ID and an offset.

10. The method of claim 1, wherein storing the data block comprises using a storage pool allocator.

11. A method for storing a first data block and a second data block, comprising:

storing the first data block and the second data block in a storage pool;

obtaining a first data block location and a second data block location;

selecting a first checksum function and a second checksum function from a plurality of checksum functions wherein the first checksum function is associated with a first checksum function ID and the second checksum function is associated with a second checksum function ID;

calculating a first data block checksum for the first data block using the first checksum function, wherein the first checksum function is associated with the first checksum function ID;

calculating a second data block checksum for the second data block using the second checksum function, wherein the second checksum function is associated with the second checksum function ID; and

storing an array of block pointers in an indirect block, wherein the array of block pointers comprises;

a first block pointer comprising the first data block location, the first data block checksum, and the first checksum function ID, and

a second block pointer comprising the second data block location, the second data block checksum, and the second checksum function ID.

12. The method of claim 11, wherein the indirect block is assembled using a data management unit.

13. The method of claim 11, wherein the indirect block is stored using a storage pool allocator.

14. A method for retrieving data in a data block, comprising:

obtaining a first indirect block comprising a first stored checksum, a first stored checksum function ID, and a second indirect block location;

obtaining a second indirect block using the second indirect block location, wherein the second indirect block comprises a second stored checksum, a second stored checksum function ID, and a data block location;

obtaining the data block using the data block location;

calculating the checksum for the data block using a checksum function corresponding to the second stored checksum ID to obtain a calculated checksum;

retrieving the data from the data block, if the second stored checksum equals the calculated checksum; and

performing an appropriate action, if the second stored checksum is not equal to the calculated checksum.

15. The method of claim 14, wherein the indirect block is assembled using a data management unit.

16. The method of claim 14, wherein the calculated checksum is calculated using a storage pool allocator.

17. A method for storing and retrieving a data block, comprising:

storing the data block in a storage pool;

obtaining a data block location associated with the data block;

determining a checksum function for the data block;

calculating a data block checksum using the checksum function for the data block;

storing a first indirect block in the storage pool, wherein the first indirect block comprises the data block location, the data block checksum, and a data block;

checksum function ID corresponding to the checksum function for the data block;

determining a checksum function for the first indirect block, wherein the checksum function for the first indirect block is associated with a first indirect block checksum function ID corresponding to the checksum function for the first indirect block;

calculating a first indirect block checksum using the checksum function for the first indirect block;

obtaining a first indirect block location; and

storing a second indirect block in the storage pool, wherein the second indirect block comprises the first indirect block location, the first indirect block checksum, and the first indirect block checksum function ID;

obtaining the first indirect block comprising the data block checksum, the data block checksum function ID, and the data block location;

obtaining the data block using the data block location;

calculating the checksum for the data block using the checksum function for the data block corresponding to the data block checksum function ID to obtain a calculated checksum;

retrieving the data from the data block, if the data block checksum equals the calculated checksum; and

performing an appropriate action, if the data block checksum is not equal to the calculated checksum.

18. A system for storing a data block, comprising:

a storage pool comprising;

the data block;

a first indirect block, wherein the first indirect block comprises a data block checksum, a data block checksum function ID, and a data block location, wherein the data block checksum is calculated using a data block checksum function, wherein the data block checksum function corresponds to the data block checksum function ID;

a second indirect block, comprising a first indirect block checksum, a first indirect block checksum function ID, and a first indirect block location, wherein the first indirect block checksum is calculated using a first indirect block checksum function, wherein the first indirect checksum function corresponds to the first indirect block checksum function ID;

wherein the data block checksum function ID and the first indirect block checksum function ID correspond to checksum functions selected from a plurality of checksum functions implemented by the system; and

a storage pool allocator configured to store the data block, the first indirect block, and the second indirect block in the storage pool.

19. The system of claim of claim 18, further comprising:

a data management unit configured to assemble the first indirect block and request the storage pool allocator store the first indirect block.

20. The system of claim 18, wherein the storage pool comprises at least one storage disk.

21. The system of claim 18, wherein the storage pool is divided into a plurality of metaslabs.

22. The system of claim 21, wherein each of the plurality of metaslabs is associated with a metaslab ID.

23. The system of claim 22, wherein the data block location comprises the metaslab ID and an offset.

24. A computer system for storing a data block, comprising:

a processor;

a memory;

a storage device; and

software instructions stored in the memory for enabling the computer system under control of the processor, to;

store the data block in a storage pool;

obtain a data block location;

determine a checksum function for the data block;

calculate a data block checksum using the checksum function for the data block;

store a first indirect block in the storage pool, wherein the first indirect block comprises the data block location, the data block checksum, and a data block checksum function ID corresponding to the checksum function for the data block;

determine a checksum function for the first indirect block, wherein the checksum function for the first indirect block is associated with a first indirect block checksum function ID corresponding to the checksum function for the first indirect block;

calculate a first indirect block checksum using the checksum function for the first indirect block;

obtain a first indirect block location; and

store a second indirect block in the storage pool, wherein the second indirect block comprises the first indirect block location, the first indirect block checksum, and the first indirect block checksum function ID.

25. A network system having a plurality of nodes, comprising:

a storage pool comprising;

a data block;

a first indirect block, wherein the first indirect block comprises the data block checksum, the data block checksum function ID, and the data block location, wherein the data block checksum is calculated using the data block checksum function, wherein the data block checksum function corresponds to the data block checksum function ID;

a second indirect block, wherein the second indirect block comprises the first indirect block checksum, the first indirect block checksum function ID, and the first indirect block location, wherein the first indirect block checksum is calculated using the first indirect block checksum function, wherein the first indirect block checksum function corresponds to the first indirect block checksum function ID;

wherein the data block checksum function ID and the first indirect block checksum function ID correspond to checksum functions selected from a plurality of checksum functions implemented by the system; and

a storage pool allocator configured to store the data block, the first indirect block, and the second indirect block in the storage pool;

wherein the storage pool is located on any one of the plurality of nodes; and

wherein the storage pool allocator is located on any one of the plurality of nodes.