Distributed maintenance of snapshot copies by a primary processor managing metadata and a secondary processor providing read-write access to a production dataset

• US 7,676,514 B2
• Filed: 05/08/2006
• Issued: 03/09/2010
• Est. Priority Date: 05/08/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Faibish, Sorin, Jiang, Xiaoye, Fridella, Stephen A., Gupta, Uday K.
• Primary Examiner(s): Woo; Isaac M
• Assistant Examiner(s): Ahluwalia; Navneet K
• Application Number:
  US11/382,193
• Publication Number:
  US 20070260830A1
• Time in Patent Office: 1,401 Days
• Field of Search: 707 1- 10, 707200-204
• US Class Current: 707/646
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A primary processor manages metadata of a production dataset and a snapshot copy, while a secondary processor provides concurrent read-write access to the primary dataset. The secondary processor determines when a first write is being made to a data block of the production dataset, and in this case sends a metadata change request to the primary data processor. The primary data processor commits the metadata change to the production dataset and maintains the snapshot copy while the secondary data processor continues to service other read-write requests. The secondary processor logs metadata changes so that the secondary processor may return a “write completed” message before the primary processor commits the metadata change. The primary data processor pre-allocates data storage blocks in such a way that the “write anywhere” method does not result in a gradual degradation in I/O performance.

Claims

1. A method of operating a primary data processor and a secondary data processor for access to a production file in data storage while maintaining a snapshot copy of the production file, comprising:

the secondary data processor maintaining a local cache of production file mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production file mapping information includes storage addresses for at least some of the data blocks of the production file and block state information indicating whether each of said at least some of the data blocks of the production file has been written to after creation of the snapshot copy of the production file;

the secondary data processor responding to a request from an application for writing to a specified data block of the production file by accessing the local cache of production file mapping information to determine that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, and upon determining that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production file; and

the primary data processor maintaining block mapping information of blocks of the data storage in the snapshot copy of the production file, and the primary data processor responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production file by committing said one of the pre-allocated blocks of the data storage to the production file.

2. The method as claimed in claim 1, wherein the primary data processor maintains a local map of pre-allocated blocks of the data storage, and responds to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production dataset by finding such a pre-allocated block of the data storage in the local map of pre-allocated blocks of the data storage.

3. The method as claimed in claim 1, wherein the primary data processor responds to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production dataset by selecting a free storage location and pre-allocating the selected free storage location to the specified block offset in the production dataset.

4. The method as claimed in claim 1, which further includes creating a new snapshot copy of the production dataset by the secondary data processor changing the block state information to indicate that all of said at least some of the data blocks of the production dataset have not been written to after creation of the new snapshot copy of the production dataset.

5. The method as claimed in claim 1, which includes the primary data processor pre-allocating blocks of the data storage for use as new storage blocks in the production file when data blocks in the production file are first written to after creation of the snapshot copy of the production file.

6. The method as claimed in claim 1, which further includes the secondary data processor responding to the request from the application for the first write to the specified data block of the production file by logging the first write to the specified data block of the production file and returning a “

write completed”

message to the application before the primary data processor commits said one of the pre-allocated blocks of the data storage to the production file.

7. The method as claimed in claim 1, which further includes the secondary data processor sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production file, and the primary data processor responding to the request for mapping information needed for writing to the specified block offset range of the production file by returning to the secondary data processor mapping information for file blocks that have been written to in the specified offset range of the production file after creation of the snapshot copy of the production file, and by returning to the secondary data processor mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file.

8. A data processing system comprising:

a primary data processor programmed for maintaining metadata of a production file and metadata of a snapshot copy of the production file; and

a secondary data processor being coupled to the primary data processor for sending metadata changes to the primary data processor, and being programmed for storing data of the production file and data of the snapshot copy of the production file in data storage;

wherein the secondary data processor is programmed for maintaining a local cache of production file mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production file mapping information includes storage addresses for at least some of the data blocks of the production file and block state information indicating whether each of said at least some of the data blocks of the production file has been written to after creation of the snapshot copy of the production file;

wherein the secondary data processor is programmed for responding to a request from an application for writing to a specified data block of the production file by accessing the local cache of production file mapping information to determine that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, and upon determining that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production file; and

wherein the primary data processor is programmed for maintaining block mapping information of blocks of the data storage in the snapshot copy of the production file, and for responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production file by committing said one of the pre-allocated blocks of the data storage to the production file.

9. The data processing system as claimed in claim 8, wherein the primary data processor is programmed for pre-allocating blocks of the data storage for storing data of the production file for use as new storage blocks in the production file when data blocks in the production file are first written to after creation of the snapshot copy of the production file.

10. The data processing system as claimed in claim 8, wherein the secondary data processor is further programmed for responding to the request from the application for the first write to the specified data block of the production file by logging the first write to the specified data block of the production file and returning a “

write completed”

message to the application before the primary data processor commits said one of the pre-allocated blocks of the data storage to the production file.

11. The data processing system as claimed in claim 8, wherein the secondary data processor is further programmed for sending to the primary data processor a request for mapping information needed for reading from a specified block offset range of the production file, and the primary data processor is further programmed for responding to the request for mapping information needed for reading from the specified block offset range of the production file by returning to the secondary data processor mapping information for the specified block offset range of the production file, the mapping information for the specified block offset range of the production file including storage addresses for the data blocks in the specified block offset range of the production file and block state information indicating whether each of the data blocks in the specified block offset range of the production file has been written to after creation of the snapshot copy of the production file.

12. The data processing system as claimed in claim 8, wherein the secondary data processor is further programmed for sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production file, and the primary data processor is further programmed for responding to the request for mapping information needed for writing to the specified block offset range of the production file by returning to the secondary data processor mapping information for file blocks that have been written to in the specified offset range of the production file after creation of the snapshot copy of the production file, and by returning to the secondary data processor mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file.

13. The data processing system as claimed in claim 8, wherein the primary data processor is programmed for maintaining a local map of pre-allocated blocks of the data storage, and for responding to a request from the secondary data processor for a pre-allocated block of the data storage for a specified block offset in the production file by finding such a pre-allocated block of data storage in the local map of pre-allocated blocks of the data storage.

14. The data processing system as claimed in claim 8, wherein the primary data processor is programmed for responding to a request from the secondary data processor for a pre-allocated block of storage for a specified block offset in the production file by selecting a free storage location and pre-allocating the selected free storage location to the specified block offset in the production file.

15. A data processing system comprising:

a primary data processor programmed for maintaining metadata of a production file and metadata of a snapshot copy of the production file; and

a secondary data processor being coupled to the primary data processor for sending metadata changes to the primary data processor, and being programmed for storing data of the production file and data of the snapshot copy of the production file in data storage;

wherein the secondary data processor is programmed for maintaining a local cache of production file mapping information and a local write map of pre-allocated blocks of the data storage, wherein the local cache of production file mapping information includes storage addresses for at least some of the data blocks of the production file and block state information indicating whether each of said at least some of the data blocks of the production file has been written to after creation of the snapshot copy of the production file;

wherein the secondary data processor is programmed for responding to a request from an application for writing to a specified data block of the production file by accessing the local cache of production file mapping information to determine that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, and upon determining that the specified data block of the production file has not yet been written to after creation of the snapshot copy of the production file, writing to one of the pre-allocated blocks of the data storage and sending a request to the primary data processor to commit said one of the pre-allocated blocks of the data storage to the production file;

wherein the primary data processor is programmed for maintaining block mapping information of blocks of the data storage in the snapshot copy of the production file, and for responding to the request from the secondary data processor to commit said one of the pre-allocated blocks of the data storage to the production file by committing said one of the pre-allocated blocks of the data storage to the production file;

the secondary data processor is programmed for sending to the primary data processor a request for mapping information needed for reading from a specified block offset range of the production file, and the primary data processor is programmed for responding to the request for mapping information needed for reading from the specified block offset range of the production file by returning to the secondary data processor mapping information for the specified block offset range of the production file, the mapping information for the specified block offset range of the production file including storage addresses for the data blocks in the specified block offset range of the production file and block state information indicating whether each of the data blocks in the specified block offset range of the production file has been written to after creation of the snapshot copy of the production file;

the secondary data processor is programmed for sending to the primary data processor a request for mapping information needed for writing to a specified block offset range of the production file, and the primary data processor is programmed for responding to the request for mapping information needed for writing to the specified block offset range of the production file by returning to the secondary data processor mapping information for file blocks that have been written to in the specified offset range of the production file after creation of the snapshot copy of the production file, and by returning to the secondary data processor mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file; and

the primary data processor is programmed for maintaining a local map of pre-allocated blocks of the data storage, and for searching the local map of pre-allocated blocks of the data storage for finding the mapping information for pre-allocated blocks of storage for receiving new data for file blocks that have not yet been written to in the specified offset range of the production file after creation of the snapshot copy of the production file.