Replication of snapshot using a file system copy differential

• US 20040267836A1
• Filed: 06/25/2003
• Published: 12/30/2004
• Est. Priority Date: 06/25/2003
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bergant, Milena, Armangau, Philippe
• Granted Patent: US 7,567,991 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A snapshot copy facility stores snapshot copies of a production, file system. The snapshot copy facility receives a request for the difference between a specified older snapshot,copy and a specified younger snapshot copy, and responds by returning the difference between the older snapshot copy and the younger snapshot copy. In a preferred implementation, the snapshot copy facility has an index for each snapshot copy for indicating blocks of data in the production file system that have changed between the snapshot copy and a next snapshot copy of the production file system. The indices are scanned for a sequence of the snapshot copies to determine the blocks that have changed, and the snapshot copy facility returns the block numbers and data in the younger snapshot copy for the blocks that have changed.

Claims

1. A method of operating a snapshot copy facility that stores a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point,in time, said method comprising:

the snapshot copy facility receiving a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies; and

the snapshot copy facility responding to the request by returning the difference between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

2. The method as claimed in claim 1, wherein the production file system includes blocks of data, and the snapshot copy facility returns an identification of each block that has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies, and the snapshot copy facility returns the data in the specified younger one of the snapshot copies for said each block that has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

3. The method as claimed in claim 2, wherein the identifications of the changed blocks and the data of the changed blocks are returned in a sequential block number order.

4. The method as claimed in claim 1, wherein the snapshot copy facility has an index for each snapshot copy for indicating changes between said each snapshot copy and a next snapshot copy of the production file system, and the method includes scanning the index for the specified older one of the snapshot copies.

5. The method as claimed in claim 4, wherein the index for at least one of the snapshot copies is a bit map.

6. The method as claimed in claim 4, wherein the index for at least one of the snapshot copies includes a hash table.

7. The method as claimed in claim 4, which includes scanning the indices for a sequence of the snapshot copies including the index for the specified older one of the snapshot copies and a respective index for each snapshot copy of the production file system that is both younger than the specified older one snapshot copies and older than the specified younger one of the snapshot copies.

8. The method as claimed in claim 7, wherein the indices for the sequence of the snapshot copies are scanned by a program routine having an outer loop indexing blocks of data in the file system, and an inner loop indexing the snapshot copies in the sequence of the snapshot copies.

9. The method as claimed in claim 1, wherein the snapshot copy facility has an index for each snapshot copy for indicating blocks of data that are known to be invalid in said each snapshot copy, and the method includes scanning the index for the specified younger one of the snapshot copies, and when the index indicates that a block is not known to be invalid, then determining whether the block has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

10. A method of operating a snapshot copy facility that stores a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, the snapshot copy facility having an index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system, wherein the method comprises:

scanning the indices for a sequence of the snapshot copies to determine the blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

11. The method as claimed in claim 10, wherein at least one of the indices is a bit map.

12. The method as claimed in claim 10, wherein at least one of the indices includes a hash table.

13. The method as claimed in claim 10, which includes responding to a request for the difference between the older one of the snapshot copies and a younger one of the snapshot copies by:

returning a sequence of block numbers of the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies, and

returning the data in the younger one of the snapshot copies for the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies.

14. The method as claimed in claim 13, wherein the block numbers of the changed blocks and the data of the changed blocks are returned in a sequential block number order.

15. The method as claimed in claim 10, wherein the indices for the sequence of the snapshot copies are scanned by a program routine having an outer loop indexing respective blocks, and an inner loop indexing snapshot copies in the sequence of the snapshot copies.

16. The method as claimed in claim 15, wherein the snapshot copy facility has a meta bit map for each snapshot copy for indicating blocks of data that are known to be invalid in said each snapshot copy, and the method includes scanning the meta bit map for the specified younger one of the snapshot copies, and when the meta bit map is found to indicate that a block is not known to be invalid, then determining whether the block has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies by scanning the indices for the sequence of the snapshot copies.

17. A method of operating a snapshot copy facility that stores a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective, point in time, the snapshot copy facility having a first index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system and that have a “

before image”

saved for said each snapshot copy, the snapshot copy facility having a second index for said each snapshot copy for indicating blocks of data that are not in use in said each snapshot copy;

said method comprising;

responding to a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies by accessing the second index for the specified younger one of the snapshot copies to determine blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, and for blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, accessing at least one of the first indices for a sequence of the snapshot copies to determine blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

18. The method as claimed in claim 17, which also includes accessing at least one of the second indices for the snapshot copies in the sequence of the snapshot copies and finding that at least one of the blocks is not in use in at least one of the snapshot copies in the sequence of the snapshot copies to determine that said at least one of the blocks has changed between the older one of the snapshot copies and the younger one of the snapshot copies not changed.

19. A method of operating a network file server, the network file server having a snapshot copy facility for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, said method comprising:

the network file server receiving a request for an update to a specified snapshot copy of the production file system;

the network file server responding to the request by checking whether the snapshot copy facility contains the specified snapshot copy of the production file system, and upon finding that the snapshot copy facility contains the specified snapshot copy of the production file system, the network file server returning the difference between the specified snapshot copy of the production file system and a more recent snapshot copy, of the production file system.

20. The network file server as claimed in claim 19, wherein the more recent snapshot copy of the production file system is the most recent one of the snapshot copies of the production file system that are stored in the snapshot copy facility.

21. The network file server as claimed in claim 19, wherein, the request specifies the more recent snapshot copy of the production file system.

22. The network file server as claimed in claim 19, wherein the network file server returns the difference between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system by returning a series of block numbers for blocks of the production file system that have changed between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system, and returning the data in the more recent snapshot copy of the production file system for said each block that has changed between the specified one of the snapshot copies of the production file system and the more recent snapshot copy of the production file system.

23. In a data processing network having a client and a network file server, the network file server storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, the client having a local version of an older one of the snapshot copies, a method of providing the client with a younger one of the snapshot copies, the method comprising:

the network file server determining the difference between the younger one of the snapshot copies and the older one of the snapshot copies;

the network file server transmitting the difference between the younger one of the snapshot copies and the older one of the snapshot copies to the local version of the older one of the snapshot copies; and

writing the difference between the younger one of the snapshot copies and the older one of the snapshot copies into the local version of the older one of the snapshot copies to produce a local version of the younger one of the snapshot copies.

24. The method as claimed in claim 23, wherein the network file server determines the difference between the younger one of the snapshot copies and the older one of the snapshot copies in response to an update request from the client, the update request specifying the older one of the snapshot copies.

25. The method as claimed in claim 23, wherein the network file server determines the difference between the younger one of the snapshot copies and the older one of the snapshot copies by determining blocks of the production file system that have changed between the younger one of the snapshot copies and the older one of the snapshot copies, and the network file server transmits the difference between the younger one of the snapshot copies and the older one of the snapshot copies to the local version of the younger one of the snapshot copies by transmitting block identifiers for the blocks of the production file system that have changed between the younger one of the snapshot copies and the older one of the snapshot copies, and by transmitting the data in the younger one of the snapshot copies for the blocks of the production file system that have changed between the younger one of the snapshot copies and the older one of the snapshot copies.

26. A snapshot copy facility comprising:

storage for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time; and

at least one processor programmed for receiving a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies; and

for responding to the request by returning the difference between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

27. The snapshot copy facility as claimed in claim 26, wherein the production file system includes blocks of data, and said at least one processor is programmed for returning an identification of each block that has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies, and the snapshot copy facility returns the data in the specified younger one of the snapshot copies for said each block that has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

28. The snapshot copy facility as claimed in claim 27, wherein said at least one processor is programmed to return the identifications of the changed blocks and the data of the changed blocks in a sequential block number order.

29. The snapshot copy facility as claimed in claim 26, wherein the snapshot copy facility has an index for each snapshot copy for indicating changes between said each snapshot copy and a next snapshot copy of the production file system, and said at least one processor is programmed for scanning the index for the specified older one of the snapshot copies.

30. The snapshot copy facility as claimed in claim 29, wherein the index for at least one of the snapshot copies is a bit map.

31. The snapshot copy facility as claimed in claim 29, wherein the index for at least one of the snapshot copies includes a hash table.

32. The snapshot copy facility as claimed in claim 29, wherein said at least one processor is programmed for scanning the indices for a sequence of the snapshot copies including the index for the specified older one of the snapshot copies and a respective index for each snapshot copy of the production file system that is both younger than the specified older one snapshot copies and older than the specified younger one of the snapshot copies.

33. The snapshot copy facility as claimed in claim 32, wherein said at least one processor is programmed for scanning the indices for the sequence of the snapshot copies by a program routine having an outer loop indexing the blocks, and an inner loop indexing the snapshot copies in the sequence of the snapshot copies.

34. The snapshot copy facility as claimed in claim 26, wherein the snapshot copy facility has an index for each snapshot copy for indicating blocks of data that are known to be invalid in said each snapshot copy, and said at least one processor is programmed for scanning the index for the specified younger one of the snapshot copies, and when the index indicates that a block is not known to be invalid, then determining whether the block has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

35. A snapshot copy facility comprising:

storage for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time;

an index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system, and

at least one processor programmed for scanning the indices for a sequence of the snapshot copies to determine the blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

36. The snapshot copy facility as claimed in claim 35, wherein at least one of the indices is a bit map.

37. The snapshot copy facility as claimed in claim 35, wherein at least one of the indices includes a hash table.

38. The snapshot copy facility as claimed in claim 35, wherein the production file system includes blocks of data, and said at least one processor is programmed to respond to a request for the difference between the older one of the snapshot copies and a younger one of the snapshot copies by:

returning a sequence of block numbers of the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies, and

returning the data in the younger one of the snapshot copies for the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies.

39. The snapshot copy facility as claimed in claim 38, wherein said at least one processor is programmed to return the, block numbers of the changed blocks and the data of the changed blocks in a sequential block number order.

40. The snapshot copy facility as, claimed in claim 35, wherein said at least one processor is programmed for scanning the indices for the sequence of the snapshot copies by a program routine having an outer,loop indexing the blocks, and an inner loop indexing the snapshot copies in the sequence of the snapshot copies.

41. The snapshot copy facility as claimed in claim 35, which includes a meta bit map for each snapshot copy for indicating blocks of data that are known to be invalid in said each snapshot copy, and wherein said at least one processor is programmed for scanning the meta bit map for the specified younger one of the snapshot copies, and when the meta bit map is found to indicate that a block is not known to be invalid, then determining whether the block has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies by scanning the indices for the sequence of the snapshot copies.

42. A snapshot copy facility comprising:

storage for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time;

a first index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system and that have a “

before image”

for said each snapshot copy stored in the storage,

a second index for each snapshot copy for indicating blocks of data that are not in use in said each snapshot copy, and

at least one processor programmed for responding to a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies by accessing the second index for the specified younger one of the snapshot copies to determine blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, and for blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, accessing at least one of the first indices for a sequence of the snapshot copies to determine blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

43. The snapshot copy facility as claimed in claim 42, wherein said at least one processor is also programmed for accessing at least one of the second indices for the snapshot copies in the sequence of the snapshot copies and finding that at least one of the blocks is not in use in at least one of the snapshot copies in the sequence of the snapshot copies to determine that said at least one of the blocks has changed between the older one of the snapshot copies and the younger one of the snapshot copies not-changed.

44. A network file server comprising a snapshot copy facility for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time,

wherein the network file server is programmed for receiving a request, for an update to a specified snapshot copy of the production file system, and responding to the request by checking whether the snapshot copy facility contains the specified snapshot copy of the production file system, and upon finding that the snapshot copy facility contains the specified snapshot copy of the production file system, returning the difference between the specified snapshot copy of the production file system and a more recent snapshot copy of the production file system.

45. The network file server as claimed in claim 44, wherein the more recent snapshot copy of the production file system is the most recent one of the snapshot copies of the production file system that are stored in the snapshot copy facility.

46. The network file server as claimed in claim 44, wherein the request specifies the more recent snapshot copy of the production file system.

47. The network file server as claimed in claim 44, wherein the network file server is programmed to return the difference between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system by returning a series of block numbers for blocks of the production file system that have changed between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system, and the data in the more recent snapshot copy of the production file system for said each block that has changed between the specified one of the snapshot copies of the production file system and the more recent snapshot copy of the production file system.

48. The network file server as claimed in claim 44, wherein the network file server is programmed to return the more recent snapshot copy of the production file system upon finding that the snapshot copy facility does not contain the specified snapshot copy of the production file system.

49. A program storage device containing a program for a snapshot copy facility, the snapshot copy facility storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, wherein the program is executable for responding to a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies by returning the difference between the specified older one of the snapshot copies and the specified younger one of the snapshot copies.

50. The program storage device as claimed in claim 49, wherein the program is executable for returning an identification of each block that has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies, and for returning the data in the specified younger one of the snapshot copies for said each block that has changed between the specified older one of the-,snapshot copies and the specified younger one of the snapshot copies.

51. The program storage device as claimed in claim 50, wherein the program is executable for returning the identifications of the changed blocks and the data of the changed blocks in a sequential block number order.

52. The program storage device as claimed in claim 49, wherein the snapshot copy facility has an index for each snapshot copy for indicating changes between said each snapshot copy and a next snapshot copy of the production file system, and the program is executable for scanning the index for the specified older one of the snapshot copies.

53. The program storage device as claimed in claim 52, wherein the program is executable for scanning the indices for a sequence of the snapshot copies including the index for the specified older one of the snapshot copies and a respective index for each snapshot copy of the production file system that is both younger than the specified older one snapshot copies and older than the specified younger one of the snapshot copies.

54. The program storage device as claimed in claim 53, wherein the program is executable for scanning the indices for the sequence of the snapshot copies by a program routine having an outer loop indexing the blocks, and an inner loop indexing the snapshot copies in the sequence of the snapshot copies.

55. A program storage device containing a program for a snapshot copy facility, the snapshot copy facility having a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, and an index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system, wherein the program is executable for scanning the indices for a sequence of the snapshot copies to determine the blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

56. The program storage device as claimed in claim 55, wherein the program is executable for responding to a request for the difference between the older one of the snapshot copies and a younger one of the snapshot copies by:

returning a sequence of block numbers of the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies, and

returning the data in the younger one of the snapshot copies for the blocks that have changed between the older one of the snapshot copies and the younger one of the snapshot copies.

57. The program storage device as claimed in claim 56, wherein the program is executable for returning the block numbers of the changed blocks and the data of the changed blocks in a sequential block number order.

58. The program storage device as claimed in claim 55, wherein the program is executable for scanning the indices for the sequence of the snapshot copies by a program routine having an outer loop indexing the blocks, and an inner loop indexing the snapshot copies in the sequence of the snapshot copies.

59. The program storage device as claimed in claim 55, wherein the snapshot copy facility has a meta bit map for each snapshot copy for indicating blocks of data that are known to be invalid in said each snapshot copy, and wherein the program storage device is executable for scanning the meta bit map for the specified younger one of the snapshot copies, and when the meta bit map is found to indicate that a block is not known to be invalid, then determining whether the block has changed between the specified older one of the snapshot copies and the specified younger one of the snapshot copies by scanning the indices for the sequence of the snapshot copies.

60. A program storage device containing a program for a snapshot copy facility, the snapshot copy facility having a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point in time, a first index for each snapshot copy for indicating blocks of data in the production file system that have changed between said each snapshot copy and a next snapshot copy of the production file system and that have a “

before image”

for said each snapshot copy stored in the snapshot copy facility, and a second index for each snapshot copy for indicating blocks of data that are not in use in said each snapshot copy, wherein the program is executable for responding to a request for the difference between a specified older one of the snapshot copies and a specified younger one of the snapshot copies by accessing the second index for the specified younger one of the snapshot copies to determine blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, and for blocks of data in the production file system that are in use in the specified younger one of the snapshot copies, accessing at least one of the first indices for a sequence of the snapshot copies to determine blocks that have changed between an older one of the snapshot copies and a younger one of the snapshot copies, the sequence of the snapshot copies including the older one of the snapshot copies and each of the snapshot copies that is both younger than the older one of the snapshot copies and older than the younger one of the snapshot copies.

61. The program storage device as claimed in claim 60, wherein the program is executable for accessing at least one of the second indices for the snapshot copies in the sequence of the snapshot copies and finding that at least one of the blocks is not in use in at least one of the snapshot copies in the sequence of the snapshot copies to determine that said at least one of the blocks has changed between the older one of the snapshot copies and the younger one of the snapshot copies not changed.

62. A program storage device containing a program for a network file server, the network file server including a snapshot copy facility for storing a plurality of snapshot copies of a production file system, each of the snapshot copies being a prior state of the production file system at a respective point,in time,

wherein the program is executable for receiving a request for an update to a specified snapshot copy of the production file system, and responding to the request by checking whether the snapshot copy facility contains the specified snapshot copy of the production file system, and upon finding that the snapshot copy facility contains the specified snapshot copy of the production file system, returning the difference between the specified snapshot copy of the production file system and a more recent snapshot copy of the production file system.

63. The program storage device as claimed in claim 62, wherein the more recent snapshot copy of the production file system is the most recent one of the snapshot copies of the production file system that are stored in the snapshot copy facility.

64. The program storage device as claimed in claim 62, wherein the request specifies the more recent snapshot copy of the production file system.

65. The program storage device as claimed in claim 62, wherein the program is executable for returning the difference between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system by returning a series of block numbers for blocks of the production file system that have changed between the specified snapshot copy of the production file system and the more recent snapshot copy of the production file system, and returning the data in the more recent snapshot copy of the production file system for said each block that has changed between the specified one of the snapshot copies of the production file system and the more recent snapshot copy of the production file system.

66. The program storage device as claimed in claim 62, wherein the program is executable for returning the more recent snapshot copy of the production file system upon finding that the snapshot copy facility does not contain the specified snapshot copy of the production file system.