System and method for parallelized replay of an NVRAM log in a storage appliance

• US 7,698,306 B2
• Filed: 04/27/2007
• Issued: 04/13/2010
• Est. Priority Date: 07/03/2001
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Watanabe, Steven S., Lewis, Blake H., Edwards, John K.
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US11/796,687
• Publication Number:
  US 20070245095A1
• Time in Patent Office: 1,082 Days
• Field of Search: 707/2, 707/200, 709/203
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1441 Resetting or repowering
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery

Abstract

A system and method for enabling parallel replay of a backup memory log of client transaction request entries to a network storage appliance file system is provided. The backup memory is typically implemented as a non-volatile random access memory (NVRAM). An initiator establishes a swarm of messages with a plurality of transaction blocks pointing to logged request entries and related states associated therewith. The states represent the various phases of file system recovery and disk storage including a retrieval of disk information (data and meta-data), typically in the form of a LOAD, and a subsequent modify phase. The swarm is transferred to the file system for parallel disk information-retrieval in an interleaved process. Any transactions that cannot be performed due to a required prerequisite action (e.g. a prior file-create) are returned to the initiator for reloading once the prerequisite action has occurred.

Claims

1. A method for replay of a backup memory in a storage system having a file system for managing transfer of data to and from an attached disk array, the method comprising:

accumulating, in a log in the backup memory, storage system transaction request entries after a consistency point at which time results of the transaction request entries are committed to the disk array;

establishing a swarm of messages with respect to the transaction request entries and delivering the swarm to the file system; and

performing a parallel disk information-retrieval process of the file system that is carried out in an arbitrary order on the swarm of messages in parallel by a processor within the storage system, wherein the processor commingles steps of each message to process the messages in parallel.

2. The method as set forth in claim 1 further comprising establishing, for each of the messages of the swarm, a transaction block including a pointer to one of the transaction request entries in the log, respectively, and a state that indicates whether each of the messages is one of (a) newly transferred to the file system, (b) subject to completion of a LOAD phase thereon by the disk information-retrieval process, (c) subject to completion of a MODIFY phase thereon by a MODIFY process of the file system or (d) incapable of being subject to the LOAD phase until a prerequisite event occurs.

3. The method as set forth in claim 2 wherein the prerequisite event is completion of the LOAD phase and a MODIFY phase with respect to another of the messages.

4. The method as set forth in claim 3 further comprising retransferring each of the messages incapable of being subject to a load phase until the prerequisite event occurs to the file system for completion of the LOAD phase after the prerequisite event occurs, respectively.

5. The method as set forth in claim 1 wherein the storage system comprises a network storage appliance.

6. A computer-readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that accumulate, in a log in a backup memory, storage system transaction request entries after a consistency point at which results of the transaction request entries are committed to an attached disk array;

program instructions that establish a swarm of messages with respect to the transaction request entries and delivering the swarm to a file system; and

program instructions that perform parallel a disk information-retrieval process of the file system that is carried out in an arbitrary order on the swarm of messages in parallel by a processor within a storage system, wherein the processor commingles steps of each message to process the messages in parallel.

7. The computer-readable storage medium as set forth in claim 6 further comprising program instructions that establish, for each of the messages of the swarm, a transaction block including a pointer to one of the transaction request entries in the log, respectively, in the log and a state that indicates whether each of the messages is one of (a) newly transferred to the file system, (b) subject to completion of the LOAD phase thereon by the disk information-retrieval process, (c) subject to completion of a MODIFY phase thereon by a MODIFY process of the file system or (d) incapable of being subject to the LOAD phase until a prerequisite event occurs.

8. The computer-readable storage medium as set forth in claim 7 wherein the prerequisite event is completion of the LOAD phase and a MODIFY phase with respect to another of the messages.

9. The computer-readable storage medium as set forth in claim 8 further comprising program instructions that retransfer each of the messages incapable of being subject to a load phase until the prerequisite event occurs to the file system for completion of the LOAD phase after the prerequisite event occurs, respectively.

10. The computer-readable storage medium as set forth in claim 6 wherein the storage system comprises a network storage appliance.

11. A method for replay of a backup memory in a storage system having a file system for managing transfer of data to and from an attached disk array, the method comprising:

accumulating one or more transaction request entries after a consistency point, said consistency point is a time at which results of the transaction request entries are committed to the disk array;

establishing a plurality of messages with respect to the transaction request entries, said plurality of messages being referred to as a swarm of messages and delivering the swarm to the file system; and

executing a parallel disk information-retrieval process on the swarm of messages in parallel in an arbitrary order by a processor within the storage system, wherein the processor commingles steps of each message to process the messages in parallel.

12. The method as set forth in claim 11, further comprising:

establishing, for each of the messages of the swarm, a transaction block including a pointer to one of the transaction request entries in the log.

13. The method as set forth in claim 12, further comprising:

establishing a state that indicates whether each of the messages is one of

(a) newly transferred to the file system,

(b) subject to completion of a LOAD phase thereon by the disk information-retrieval process,

(c) subject to completion of a MODIFY phase thereon by a MODIFY process of the file system, or

(d) incapable of being subject to the LOAD phase until a prerequisite event occurs.

14. The method as set forth in claim 13, further comprising:

using as the prerequisite event completion of the LOAD phase and a MODIFY phase with respect to another of the messages.

15. The method as set forth in claim 14, further comprising:

retransferring each of the messages incapable of being subject to a load phase until the prerequisite event occurs to the file system for completion of the LOAD phase after the prerequisite event occurs.

16. The method of claim 11, further comprising:

modifying at least some messages in the swarm of messages based on the order in which transaction request entries were accumulated in the log.

17. A file system, comprising:

a backup memory storing a plurality of file system transaction entries;

a first process that establishes a swarm of messages with respect to the file system transaction entries and delivers the swarm of messages to the file system;

a second process that performs a parallel LOAD phase for a plurality of messages in the swarm of messages where the LOAD phase is processed in an arbitrary order by commingling one or more steps of the LOAD phase applied to each message of the swarm of messages; and

a third process that performs a MODIFY phase for at least some messages in the swarm of messages, the MODIFY phase operating on messages based on the order in which file system transaction entries were stored in the backup memory.

18. The file system of claim 17, further comprising:

a fourth process that determines whether a file system transaction entry corresponds to a file system transaction that can be performed right away.

19. The file system of claim 18, wherein the fourth process, in response to determining that the file system transaction can not be performed right away, associates the file system transaction entry with a LOAD RETRY state until a prior prerequisite transaction is performed.

20. A method, comprising:

storing a plurality of file system transaction entries in a backup memory;

establishing a swarm of messages with respect to the file system transaction entries;

delivering the swarm of messages to a file system;

performing a parallel LOAD phase for a plurality of messages in the swarm of messages where the LOAD phase is processed in an arbitrary order by commingling one or more steps of the LOAD phase applied to each message of the swarm of messages; and

performing a MODIFY phase for at least some messages in the swarm of messages, the MODIFY phase operating on messages based on the order in which file system transaction entries were stored in the backup memory.

21. A method for operating a storage system, comprising:

storing a plurality of file system transaction entries in a backup memory;

establishing a swarm of messages with respect to the plurality of file system transaction entries;

delivering the swarm of messages to a file system; and

performing a parallel retrieval process by a processor within the storage system, for a plurality of messages in the swarm of messages by processing the messages in an arbitrary order, where the processor executes the messages in the arbitrary order by commingling the processing of messages and steps of the retrieval process.

22. An operating system configured with a storage system, comprising:

a backup memory storing a plurality of transaction entries;

a first process that establishes a swarm of messages with respect to the transaction entries and delivers the swarm of messages to the operating system;

a second process that performs a parallel LOAD phase for a plurality of messages in the swarm of messages where the LOAD phase is processed in an arbitrary order by a processor commingling one or more steps of the LOAD phase applied to each message of the swarm of messages; and

a third process that performs a MODIFY phase for at least some messages in the swarm of messages, the MODIFY phase operating on messages based on the order in which the transaction entries were stored in the backup memory.

23. A system for replay of a backup memory in a storage system having an operating system for managing transfer of data to and from an attached disk array, the system comprising:

a log in the backup memory containing the storage system transaction entries accumulated after a consistency point at which time results of the storage system transaction entries are committed to the disk array;

an initiator process that establishes a plurality of messages with respect to the storage system transaction entries and delivers the plurality of messages to the operating system; and

a parallel disk information-retrieval process in the operating system that is carried out on the plurality of messages in an arbitrary order by the processor commingling steps of each message of the plurality of messages.