Data storage system and method for uninterrupted read-only access to a consistent dataset by one host processor concurrent with read-write access by another host processor

• US 6,823,336 B1
• Filed: 09/26/2000
• Issued: 11/23/2004
• Est. Priority Date: 09/26/2000
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Srinivasan, Sudhir, Gupta, Uday K., Armangau, Philippe, Vahalia, Uresh
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US09/669,939
• Time in Patent Office: 1,519 Days
• Field of Search: 707/201, 707/203, 707/8
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A data storage system receives sets of the revisions such that each set of revisions changes the dataset from one consistent state to another. Each set of revisions is processed in a write-selected phase followed by a read-selected phase. In the write-selected phase, the revisions in each set are written to a buffer and processed to produce a directory of the set of revisions. In the read-selected phase, the revisions are read from the buffer and integrated into the dataset. When one set of revisions is in the read-selected phase, the next set of revisions is in the write-selected phase. To permit uninterrupted read-only access to a consistent state of the dataset, the data storage system responds to a request for reading specified data on a priority basis by first accessing the directory of the read-selected revisions to determine whether the specified data are in the read-selected set of revisions, and if so, the specified data are read from the read-selected set of revisions, and if not, the specified data are read from the dataset. In a preferred implementation, the sets of revisions are alternately written to either a first buffer or a second buffer, and processing of each set of revisions is switched from the write-selected phase to the read-selected phase by switching between the two buffers.

Claims

1. A method of operating a data storage system to provide uninterrupted read access to a consistent dataset concurrent with performing a series of revisions upon the dataset, the series of revisions including a first set of revisions followed by a second set of revisions, the dataset being in a consistent state after performing each set of revisions upon the dataset, said method comprising:

a) processing the first set of revisions to build up a directory of the first set of revisions; and

then

b) processing the second set of revisions to build up a directory of the second set of revisions, and concurrently performing the first set of revisions upon the dataset, and concurrently performing read access to specified data in the dataset by accessing the directory of the first set of revisions to determine whether the specified data are in the first set of revisions, and upon finding that the specified data are in the first set of revisions, obtaining the specified data from the first set of revisions, and upon finding that the specified data are not in the first set of revisions, obtaining the specified data from the dataset.

2. The method as claimed in claim 1, wherein the performing of the first set of revisions upon the dataset is suspended to give priority to the read access to the dataset.

3. The method as claimed in claim 1, which further includes, upon completion of the performance of the first set of revision upon the dataset, performing the second set of revisions upon the dataset, and concurrently processing read access to the dataset.

4. The method as claimed in claim 3, wherein the first set of revisions are stored in a first buffer, and the second set of revisions are stored in a second buffer, and wherein the method includes storing a third set of revisions in the first buffer, and upon completion of the performance of the first set of revisions upon the dataset, processing the third set of revisions to build up a directory of the third set of revisions.

5. The method as claimed in claim 1, wherein the revisions are data to be written to blocks of storage, and the directory of the first set of revisions is an index of addresses of blocks of storage to be written to for the first set of revisions, and the directory of the second set of revisions is an index of addresses of blocks of storage to be written to for the second set of revisions.

6. The method as claimed in claim 1, wherein the dataset is a file system, and at least one of the revisions is a change of a file name.

7. The method as claimed in claim 1, which includes selecting a number of revisions to be included in each set of revisions in order to obtain a limited frequency of updating of the dataset.

8. The method as claimed in claim 7, which includes buffering the first set of revisions and the second set of revisions, and permitting the frequency of updating of the dataset to exceed the limited frequency when required to avoid buffer overflow.

9. A method of read-write access by a first host processor to a dataset in a first data storage system concurrent with uninterrupted read-only access by a second host processor to a consistent state of a copy of the dataset in a second data storage system, revisions to the dataset in the first data storage system from the read-write access also being made to the copy of the dataset in the second data storage system, the revisions including a first set of revisions followed by a second set of revisions, the dataset being in a consistent state after performing each set of revisions upon the dataset, the method comprising:

a) processing the first set of revisions to build up a first directory of dataset revisions in the first set of revisions; and

then

b) processing the second set of revisions to build up a second directory of dataset revisions in the second set of revisions, and concurrently performing the first set of revisions upon the copy of the dataset, and concurrently performing read access on a priority basis to specified data in the dataset by accessing the first directory of dataset revisions to determine whether the specified data are in the first set of revisions, and upon finding that the specified data are in the first set of revisions, obtaining the specified data from the first set of revisions, and upon finding that the specified data are not in the first set of revisions, obtaining the data from the copy of the dataset.

10. The method as claimed in claim 9, wherein the first set of revisions are stored in a first buffer, and the second set of revisions are stored in a second buffer, and wherein the method includes storing a third set of revisions in the first buffer, and upon completion of the performance of the first set of revisions upon the copy of the dataset, processing the third set of revisions to build up a directory of the third set of revisions.

11. The method as claimed in claim 9, wherein the revisions are data to be written to blocks of storage, and the first directory of dataset revisions is an index of addresses of blocks of storage to be written to for the first set of revisions, and the second directory of dataset revisions is an index of addresses of blocks of storage to be written to for the second set of revisions.

12. The method as claimed in claim 9, wherein the dataset is a file system, the revisions include a change in the name of a file, and the revisions are specified by file access commands transmitted to the second data storage system from a remote mirroring facility in the first host processor.

13. The method as claimed in claim 9, wherein the method includes selecting a number of revisions to be included in each set of revisions in order to obtain a limited frequency of updating of the copy of the dataset.

14. The method as claimed in claim 13, wherein the method includes buffering the first set of revisions and the second set of revisions, and permitting the updating of the dataset to occur more frequently that the limited frequency when required to avoid buffer overflow.

15. The method as claimed in claim 9, wherein the first data storage system groups the revisions of the dataset into the first set of revisions and into the second set of revisions in response to transaction commit commands from the first host processor, and the first data storage system transmits the first set of revisions and the second set of revisions to the second data storage system.

16. The method as claimed in claim 15, wherein the first data storage system transmits the first set of revisions to the second data storage system by placing the first set of revisions in a first volume of storage in the first data storage system, mirroring the first volume of storage in the first data storage system to a second volume of storage in the second data storage system, transmitting the second set of revisions to the second data storage system by placing the second set of revisions in a third volume of storage in the first data storage system, and mirroring the third volume of storage to a fourth volume of storage in the second data storage system.

17. The method as claimed in claim 16, which includes the first data storage system flushing the first set of revisions to the first volume of storage in response to a transaction commit command from the first host processor, and then forwarding the transaction commit command to the second data storage system.

18. The method as claimed in claim 16, which includes the first data storage system assigning a unique sequence number to each set of revisions, and storing the sequence number for said each set of revisions when said each set of revisions is stored in one of the storage volumes.

19. The method as claimed in claim 18, wherein each storage volume has a plurality of regions, and each sequence number is mapped to a respective one of regions for storage of the set of data revisions to which said each sequence number is assigned.

20. A data storage system comprising, in combination,

data storage, and

a storage controller responsive to read and write commands for accessing specified data of a dataset in the data storage, wherein each set of write commands modifies the dataset from one consistent state to another, and the storage controller is programmed to respond to each set of write commands by first operating upon revisions of said each set of write commands in a write-selected phase and then operating upon the revisions of said each set of write commands in a read-selected phase, the storage controller building up a directory of the revisions of said each set of write commands in the write-selected phase, and the storage controller accessing the directory of the revisions of said each set of write commands in the read-selected phase, the storage controller performing the revisions of said each set of write commands in the read-selected phase upon the dataset, and concurrently responding to the read commands on a priority basis by accessing the directory of the revisions of said each set of write commands in the read-selected phase to obtain specified data from the revisions of said each set of write commands in the read-selected phase when the specified data are in the revisions of said each set of write commands in the read-selected phase, and when the specified data are not in the revisions of said each set of write commands in the read-selected phase, obtaining the specified data from the dataset.

21. The data storage system as claimed in claim 20, wherein the data storage system further includes a first buffer and a second buffer, and the storage controller is programmed to place the revisions of said each set of write commands alternately in either the first buffer or the second buffer and to select one of the buffers to access the revisions for one of the sets of write commands currently in the read-selected phase and to select the other of the buffers to access one of the sets of write commands currently in the write-selected phase, and to switch between the buffers to switch from the read-selected phase to the write-selected phase for the revisions of each of the sets of the write commands.

22. The data storage system as claimed in claim 21, wherein the storage controller is programmed to switch between the buffers in response to a transaction commit command.

23. The data storage system as claimed in claim 20, wherein the data storage system is a file server, the dataset is a file system, the read commands are file access commands, and the write commands are file access commands.

24. The data storage system as claimed in claim 23, wherein the directory of the revisions of said each set of write commands is a hierarchical data structure including at least a root directory, a subdirectory, and at least one file entry in the subdirectory.

25. A data storage system comprising, in combination:

data storage, and

a storage controller responsive to read and write commands for accessing specified data of a dataset in the data storage, the write commands specifying sets of revisions to be made to the dataset, wherein

the storage controller is programmed to respond to transaction commit commands by alternately writing the sets of revisions to a first volume of the data storage and to a second volume of the data storage, each set of revisions to the dataset including revisions from a set of transactions defined by the transaction commit commands so that each set of revisions changes the dataset from one consistent state to another, and

the storage controller is programmed with a remote mirroring facility for mirroring the first and second volumes to corresponding volumes in a remote data storage system.

26. The data storage system as claimed in claim 25, wherein the storage controller is programmed to respond to a transaction commit command by flushing a current set of revisions to one of the first or second volumes, the flushing operation including the remote mirroring facility transmitting the current set of revisions from said one of the first or second volumes to the remote data storage system, and then transmitting a transaction commit command to the remote data storage system.

27. The data storage system as claimed in claim 26, wherein the storage controller is programmed to select a number of revisions to be included in each set of revisions in order to obtain a limited frequency of transmission of transaction commit commands to the remote data storage system.

28. The data storage system as claimed in claim 27, wherein the storage controller is programmed to permit the frequency of transmission of transaction commit commands to the remote data storage system to exceed the limited frequency when required to avoid overflow of the first and second volumes.

29. The data storage system as claimed in claim 25, wherein the storage controller is programmed to assign a unique sequence number to each set of revisions and to store the sequence number with each set of revisions when each set of revisions is stored in one of the storage volumes.

30. The data storage system as claimed in claim 29, wherein each of said volumes has a plurality of regions, and the storage controller is programmed to map each sequence number to a respective one of the regions for storage of the set of revisions to which said each sequence number is assigned.

31. A program storage device comprising:

a program for a storage controller of a data storage system, the program being executable by the storage controller for responding to read and write commands for accessing specified data of a dataset in data storage of the data storage system, wherein each set of write commands modifies the dataset from one consistent state to another, and the program is executable by the storage controller for responding to each set of write commands by first operating upon revisions of said each set of write commands in a write-selected phase and then operating upon the revisions of said each set of write commands in a read-selected phase, the storage controller building up a directory of the revisions of said each set of write commands in the write-selected phase, and the storage controller accessing the directory of the revisions of said each set of write commands in the read-selected phase, the storage controller performing the revisions of said each set of write commands in the read-selected phase upon the dataset, and concurrently responding to the read commands on a priority basis by accessing the directory of the revisions of said each set of write commands in the read-selected phase to obtain specified data from the revisions of said each set of write commands in the read-selected phase when the specified data are in the revisions of said each set of write commands in the read-selected phase, and when the specified data are not in the revisions of said each set of write commands in the read-selected phase, obtaining the specified data from the dataset.

32. The program storage device as claimed in claim 31, wherein the data storage system further includes a first buffer and a second buffer, and the program is executable by the storage controller for placing the revisions of said each set of write commands alternately in either the first buffer or the second buffer and to select one of the buffers to access the revisions for one of the sets of write commands currently in the read-selected phase and to select the other of the buffers to access one of the sets of write commands currently in the write-selected phase, and to switch between the buffers to switch from the read-selected phase to the write-selected phase for the revisions of each of the sets of the write commands.

33. The program storage device as claimed in claim 32, wherein the program is executable by the storage controller for switching between the buffers in response to a transaction commit command.