System and method for distributing volume status information in a storage system

• US 8,359,429 B1
• Filed: 11/08/2004
• Issued: 01/22/2013
• Est. Priority Date: 11/08/2004
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Sharma, Gopal, Rao, Santosh S., Gorby, Richard, Asthana, Aseem
• Primary Examiner(s): Campos, Yaima
• Assistant Examiner(s): Ayash, Marwan
• Application Number:
  US10/983,876
• Time in Patent Office: 2,997 Days
• Field of Search: None
• US Class Current: 711/111
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2082 Data synchronisation

Abstract

System and method for distributing volume status information in a storage system. According to one embodiment, a system may include a plurality of volumes configured to store data, where the volumes are configured as mirrors of one another, and a plurality of hosts configured to access the plurality of volumes. A first one of the plurality of hosts may be configured to execute a mirror recovery process and to maintain a progress indication of the mirror recovery process, and the first host may be further configured to distribute the progress indication to another one or more of the plurality of hosts.

Claims

1. A system, comprising:

a plurality of volumes configured to store data, wherein said volumes are configured as mirrors of one another; and

a plurality of hosts configured to access said plurality of volumes;

wherein a first one of said plurality of hosts is configured to execute a mirror recovery process with respect to said plurality of volumes and to maintain a progress indication of said mirror recovery process;

wherein said first host is further configured to distribute said progress indication to another one or more of said plurality of hosts including a second host; and

wherein said second host is configured to;

detect a read operation directed to a particular data block stored by said plurality of volumes;

dependent upon said progress indication distributed by said first host, determine whether said particular data block has been recovered by said mirror recovery process; and

in response to determining that said particular data block has not been recovered by said mirror recovery process, convert said read operation into a read-writeback operation, wherein said read-writeback operation is operable both to read a data value corresponding to said particular data block from a given one of said plurality of volumes and to write said data value to each other one of said plurality of volumes.

2. The system as recited in claim 1, wherein to execute said mirror recovery process, said first host is further configured to:

read a data block at a current mirror recovery offset from a given one of said plurality of volumes; and

in response to reading said data block, write said data block to each of said plurality of volumes other than said given volume;

wherein said current mirror recovery offset is monotonic; and

wherein said progress indication comprises said current mirror recovery offset.

3. The system as recited in claim 2, wherein said first host is further configured to grant a distributed lock to said second one of said plurality of hosts, and wherein said first host is further configured to both grant said distributed lock and distribute said progress indication to said second host in response to receiving a distributed lock request corresponding to said particular block from said second host.

4. The system as recited in claim 3, wherein said second host is further configured to:

in response to conveying said distributed lock request to said first host, receive an indication of said distributed lock and said progress indication distributed from said first host; and

in response to determining that said particular data block has been recovered by said mirror recovery process, release said distributed lock and perform said read operation;

wherein to determine whether said particular data block has been recovered, said second host is further configured to compare an offset specified by said read operation to said current mirror recovery offset included in said progress indication.

5. The system as recited in claim 3, wherein said second host is further configured to:

maintain a local indication of said current mirror recovery offset; and

convey said distributed lock request to said first host dependent upon whether said particular data block has been recovered by said mirror recovery process, wherein to determine whether said particular data block has been recovered, said second host is further configured to compare an offset specified by said read operation to said local indication of said current mirror recovery.

6. The system as recited in claim 3, wherein said second host is further configured to:

maintain a local indication of said current mirror recovery offset;

subsequent to detecting a write operation to a given data block stored by said plurality of volumes, determine whether said given data block has been recovered by said mirror recovery process, wherein to determine whether said given data block has been recovered, said second host is further configured to compare an offset specified by said write operation to said local indication of said current mirror recovery offset; and

in response to determining that said given data block has not yet been recovered, convey a distributed lock request corresponding to said given data block to said first host.

7. The system as recited in claim 6, wherein said second host is further configured to proceed with said write operation using a local lock instead of a distributed lock in response to determining that said given data block has been recovered.

8. The system as recited in claim 1, wherein said first host is further configured to atomically distribute said progress indication to each other one of said plurality of hosts, wherein to atomically distribute said progress indication, said first host is further configured to quiesce accesses to said plurality of volumes originating from said plurality of hosts until distribution of said progress indication is complete.

9. A method, comprising:

storing data on a plurality of volumes, wherein said volumes are configured as mirrors of one another;

a first one of a plurality of hosts executing a mirror recovery process with respect to said plurality of volumes;

said first host maintaining a progress indication of said mirror recovery process;

said first host distributing said progress indication to one or more of said plurality of hosts, including a second host, configured to access said plurality of volumes;

said second host detecting a read operation directed to a particular data block stored by said plurality of volumes;

dependent upon said progress indication distributed by said first host, said second host determining whether said particular data block has been recovered by said mirror recovery process; and

in response to determining that said particular data block has not been recovered by said mirror recovery process, said second host converting said read operation into a read-writeback operation, wherein said read-writeback operation is operable both to read a data value corresponding to said particular data block from a given one of said plurality of volumes and to write said data value to each other one of said plurality of volumes.

10. The method as recited in claim 9, wherein said first host executing said mirror recovery process comprises:

reading a data block at a current mirror recovery offset from a given one of said plurality of volumes; and

in response to said reading, writing said data block to each of said plurality of volumes other than said given volume;

wherein said current mirror recovery offset is monotonic; and

wherein said progress indication comprises said current mirror recovery offset.

11. The method as recited in claim 10, further comprising:

said first host granting a distributed lock to a second one of said plurality of hosts and distributing said progress indication to said second host, wherein both granting said distributed lock and distributing said progress indication occur in response to receiving a distributed lock request corresponding to said particular block from said second host.

12. The method as recited in claim 11, further comprising:

in response to conveying said distributed lock request to said first host, said second host receiving an indication of said distributed lock and said progress indication distributed from said first host; and

in response to determining that said particular data block has been recovered by said mirror recovery process, said second host releasing said distributed lock and performing said read operation, wherein determining whether said particular data block has been recovered comprises said second host comparing an offset specified by said read operation to said current mirror recovery offset included in said progress indication.

13. The method as recited in claim 11, further comprising:

said second host maintaining a local indication of said current mirror recovery offset; and

said second host conveying said distributed lock request to said first host dependent upon determining whether said particular data block has been recovered by said mirror recovery process, wherein determining whether said particular data block has been recovered comprises said second host comparing an offset specified by said read operation to said local indication of said current mirror recovery.

14. The method as recited in claim 11, further comprising:

said second host maintaining a local indication of said current mirror recovery offset;

subsequent to detecting a write operation to a given data block stored by said plurality of volumes, said second host determining whether said given data block has been recovered by said mirror recovery process, wherein determining whether said given data block has been recovered comprises said second host comparing said offset specified by said write operation to said local indication of said current mirror recovery offset; and

in response to determining that said given data block has not yet been recovered, said second host conveying a distributed lock request corresponding to said given data block to said first host.

15. The method as recited in claim 14, further comprising said second host proceeding with said write operation using a local lock instead of a distributed lock in response to determining that said given data block has been recovered.

16. The method as recited in claim 9, further comprising said first host atomically distributing said progress indication to each other one of said plurality of hosts, wherein atomically distributing said progress indication comprises quiescing accesses to said plurality of volumes originating from said plurality of hosts until distribution of said progress indication is complete.

17. A computer-accessible storage medium, comprising program instructions, wherein the program instructions are computer-executable to implement:

storing data on a plurality of volumes, wherein said volumes are configured as mirrors of one another;

a first one of a plurality of hosts executing a mirror recovery process respect to said plurality of volumes;

said first host maintaining a progress indication of said mirror recovery process;

said first host distributing said progress indication to one or more of a plurality of hosts, including a second host, configured to access said plurality of volumes;

said second host detecting a read operation directed to a particular data block stored by said plurality of volumes;

dependent upon said progress indication distributed by said first host, said second host determining whether said particular data block has been recovered by said mirror recovery process; and

in response to determining that said particular data block has not been recovered by said mirror recovery process, said second host converting said read operation into a read-writeback operation, wherein said read-writeback operation is operable both to read a data value corresponding to said particular data block from a given one of said plurality of volumes and to write said data value to each other one of said plurality of volumes.

18. The computer-accessible storage medium as recited in claim 17, wherein to implement said first host executing said mirror recovery process, the program instructions are further computer-executable to implement:

reading a data block at a current mirror recovery offset from a given one of said plurality of volumes; and

in response to said reading, writing said data block to each of said plurality of volumes other than said given volume;

wherein said current mirror recovery offset is monotonic; and

wherein said progress indication comprises said current mirror recovery offset.

19. The computer-accessible medium as recited in claim 18, wherein the program instructions are further computer-executable to implement said first host granting a distributed lock to a given one of said plurality of hosts and to distributing said progress indication to said given host, and wherein both granting said distributed lock and distributing said progress indication occur in response to receiving a distributed lock request corresponding to said particular block from said given host.

20. A system, comprising:

a plurality of volumes configured to store data, wherein said volumes are configured as mirrors of one another; and

a plurality of hosts configured to access said plurality of volumes;

wherein a first one of said plurality of hosts is configured to distribute a progress indication of a mirror recovery process in addition to a lock indication corresponding to a particular data block stored by said plurality of volumes to a second one of said plurality of hosts in response to receiving a particular distributed lock request corresponding to said particular data block from said second host;

wherein said progress indication comprises an indication of data blocks stored by said plurality of volumes that have already been recovered by said mirror recovery process; and

wherein said second host is configured to generate one or more additional distributed lock requests for data blocks stored by said plurality of volumes dependent upon said progress indication returned in response to said particular distributed lock request.

21. The system as recited in claim 20,

wherein said progress indication comprises a current mirror recovery offset.

22. The system as recited in claim 20, wherein said second host is further configured to:

subsequent to receiving said progress indication, detect a read operation to a given data block stored by said plurality of volumes;

dependent upon said progress indication, determine whether said given data block has been recovered by said mirror recovery process; and

in response to determining that said given data block has not been recovered by said mirror recovery process, convert said read operation into a read-writeback operation.

23. The system as recited in claim 20, wherein said second host is further configured to:

subsequent to receiving said progress indication, detect a write operation to a given data block stored by said plurality of volumes;

dependent upon said progress indication, determine whether said given data block has been recovered by said mirror recovery process; and

in response to determining that said given data block has not yet been recovered by said mirror recovery process, convey a distributed lock request corresponding to said given data block to said first host.