DISTRIBUTED MAINTENANCE OF SNAPSHOT COPIES BY A PRIMARY PROCESSOR MANAGING METADATA AND A SECONDARY PROCESSOR PROVIDING READ-WRITE ACCESS TO A PRODUCTION DATASET

• US 20070260830A1
• Filed: 05/08/2006
• Published: 11/08/2007
• Est. Priority Date: 05/08/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Faibish, Sorin, Jiang, Xiaoye, Fridella, Stephen, Gupta, Uday
• Granted Patent: US 7,676,514 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A primary processor manages metadata of a production dataset and a snapshot copy, while a secondary processor provides concurrent read-write access to the primary dataset. The secondary processor determines when a first write is being made to a data block of the production dataset, and in this case sends a metadata change request to the primary data processor. The primary data processor commits the metadata change to the production dataset and maintains the snapshot copy while the secondary data processor continues to service other read-write requests. The secondary processor logs metadata changes so that the secondary processor may return a “write completed” message before the primary processor commits the metadata change. The primary data processor pre-allocates data storage blocks in such a way that the “write anywhere” method does not result in a gradual degradation in I/O performance.

Claims

1. A method of operating a primary data processor and a secondary data processor for access to a production dataset in data storage while maintaining a snapshot copy of the production dataset, comprising:

the primary data processor pre-allocating blocks of the data storage for use as new storage blocks in the production dataset when data blocks in the production dataset are first written to after creation of the snapshot copy of the production dataset;

the secondary data processor responding to a request from an application for a first write to a data block of the production dataset after creation of the snapshot copy of the production dataset by writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production dataset; and

the primary data processor responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production dataset by committing said one of the pre-allocated blocks of the data storage to the production dataset.

2. The method as claimed in claim 1, wherein the primary data processor maintains block mapping information of blocks of the data storage in the snapshot copy of the production dataset.

3. The method as claimed in claim 1, wherein the pre-allocation of the blocks of the data storage by the primary data processor includes the primary data processor pre-allocating a block of the data storage for a specified block offset in the production dataset by selecting a free storage location for storage co-locality with neighboring blocks of the production dataset.

4. The method as claimed in claim 1, wherein the primary data processor maintains a local map of pre-allocated blocks of the data storage, and responds to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production dataset by finding such a pre-allocated block of the data storage in the local map of pre-allocated blocks of the data storage.

5. The method as claimed in claim 1, wherein the primary data processor responds to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production dataset by selecting a free storage location and pre-allocating the selected free storage location to the specified block offset in the production dataset.

6. The method as claimed in claim 1, which further includes the secondary data processor responding to the request from the application for the first write to the data block of the production dataset by logging the first write to the data block of the production dataset and returning a “

write completed”

message to the application before the primary data processor commits said one of the pre-allocated blocks of the data storage to the production dataset.

7. The method as claimed in claim 1, which further includes the secondary data processor maintaining a local cache of production dataset mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production dataset mapping information includes storage addresses for at least some of the data blocks of the production dataset and block state information indicating whether each of said at least some of the data blocks of the production dataset has been written to after creation of the snapshot copy of the production dataset.

8. The method as claimed in claim 7, which further includes creating a new snapshot copy of the production dataset by the secondary data processor changing the block state information to indicate that all of said at least some of the data blocks of the production dataset have not been written to after creation of the new snapshot copy of the production dataset.

9. The method as claimed in claim 1, which further includes the secondary data processor sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production dataset, and the secondary data processor responding to the request for mapping information needed for writing to the specified block offset range of the production dataset by returning to the secondary data processor mapping information for blocks of the production dataset that have been written to in the specified offset range of the production dataset after creation of the snapshot copy of the production dataset, and by returning to the secondary data processor mapping information for pre-allocated blocks of the data storage for receiving new data for blocks of the production dataset that have not yet been written to in the specified offset range of the production dataset after creation of the snapshot copy of the production dataset.

10. A method of operating a primary data processor and a secondary data processor for access to a production file in data storage while maintaining a snapshot copy of the production file, comprising:

the secondary data processor maintaining a local cache of production file mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production file mapping information includes storage addresses for at least some of the data blocks of the production file and block state information indicating whether each of said at least some of the data blocks of the production file has been written to after creation of the snapshot copy of the production file;

the secondary data processor responding to a request from an application for writing to a specified data block of the production file by accessing the local cache of production file mapping information to determine that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, and upon determining that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production file; and

the primary data processor maintaining block mapping information of blocks of the data storage in the snapshot copy of the production file, and the primary data processor responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production file by committing said one of the pre-allocated blocks of the data storage to the production file.

11. The method as claimed in claim 10, which includes the primary data processor pre-allocating blocks of the data storage for use as new storage blocks in the production file when data blocks in the production file are first written to after creation of the snapshot copy of the production file.

12. The method as claimed in claim 10, which further includes the secondary data processor responding to the request from the application for the first write to the specified data block of the production file by logging the first write to the specified data block of the production file and returning a “

write completed”

message to the application before the primary data processor commits said one of the pre-allocated blocks of the data storage to the production file.

13. The method as claimed in claim 10, which further includes the secondary data processor sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production file, and the secondary data processor responding to the request for mapping information needed for writing to the specified block offset range of the production file by returning to the secondary data processor mapping information for file blocks that have been written to in the specified offset range of the production file after creation of the snapshot copy of the production file, and by returning to the secondary data processor mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file.

14. A data processing system comprising:

a primary data processor programmed for maintaining metadata of a production file and metadata of a snapshot copy of the production file; and

a secondary data processor being coupled to the primary data processor for sending metadata changes to the primary data processor, and being programmed for storing data of the production file and data of the snapshot copy of the production file in data storage;

wherein the secondary data processor is programmed for maintaining a local cache of production file mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production file mapping information includes storage addresses for at least some of the data blocks of the production file and block state information indicating whether each of said at least some of the data blocks of the production file has been written to after creation of the snapshot copy of the production file;

wherein the secondary data processor is programmed for responding to a request from an application for writing to a specified data block of the production file by accessing the local cache of production file mapping information to determine that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, and upon determining that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production file; and

wherein the primary data processor is programmed for maintaining block mapping information of blocks of the data storage in the snapshot copy of the production file, and for responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production file by committing said one of the pre-allocated blocks of the data storage to the production file.

15. The data processing system as claimed in claim 14, wherein the primary data processor is programmed for pre-allocating blocks of the data storage for storing data of the production file for use as new storage blocks in the production file when data blocks in the production file are first written to after creation of the snapshot copy of the production file.

16. The data processing system as claimed in claim 14, wherein the secondary data processor is further programmed for responding to the request from the application for the first write to the specified data block of the production file by logging the first write to the specified data block of the production file and returning a “

write completed”

message to the application before the primary data processor commits said one of the pre-allocated blocks of the data storage to the production file.

17. The data processing system as claimed in claim 14, which further includes the secondary data processor sending to the primary data processor a request for mapping information needed for reading from a specified block offset range of the production file, and the secondary data processor responding to the request for mapping information needed for reading from the specified block offset range of the production file by returning to the secondary data processor mapping information for the specified block offset range of the production file, the mapping information for the specified block offset range of the production file including storage addresses for the data blocks in the specified block offset range of the production file and block state information indicating whether each of the data blocks in the specified block offset range of the production file has been written to after creation of the snapshot copy of the production file.

18. The data processing system as claimed in claim 14, which further includes the secondary data processor sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production file, and the secondary data processor responding to the request for mapping information needed for writing to the specified block offset range of the production file by returning to the secondary data processor mapping information for file blocks that have been written to in the specified offset range of the production file after creation of the snapshot copy of the production file, and by returning to the secondary data processor mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file.

19. The data processor as claimed in claim 14, wherein the primary data processor is programmed for maintaining a local map of pre-allocated blocks of the data storage, and for responding to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production file by finding such a pre-allocated block of data storage in the local map of pre-allocated blocks of the data storage.

20. The data processor as claimed in claim 14, wherein the primary data processor is programmed for responding to a request from the secondary data processor for a pre-allocated block of storage for a specified block offset in the production file by selecting a free storage location and pre-allocating the selected free storage location to the specified block offset in the production file.