METHOD AND APPARATUS FOR MONITORING THE STORAGE OF DATA IN A COMPUTER SYSTEM

• US 20040080558A1
• Filed: 10/28/2002
• Published: 04/29/2004
• Est. Priority Date: 10/28/2002
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Blumenau, Steven M., Fisher, Michel
• Granted Patent: US 7,546,482 B2
• Time in Patent Office: Days
• US Class Current: 347/19
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 2201/88 Monitoring involving counting

Abstract

One aspect is directed to a method for storing, in a persistent manner, a state of the storage process such that the state of the storage operation may be recovered if the storage operation is interrupted. Another aspect is directed to a method and apparatus that stores the state of a storage operation in a non-volatile memory of a computer that stores data on a storage system. Another aspect is directed to a method for recovering the state of the storage process without having to re-perform storage operations. Another aspect is directed to maintaining state information for other purposes such as monitoring access to data storage locations.

Claims

1. A method for use in a computer system including at least one computer and at least one storage system that is coupled to the at least one computer to store data written thereby, the method comprising acts of:

performing, by the at least one computer, at least one storage operation on a first storage location in the at least one storage system; and

storing, in a non-volatile memory of the at least one computer, information identifying a state of the at least one storage operation performed by the at least one computer.

2. The method according to claim 1, wherein the non-volatile memory includes a non-volatile random access memory (NVRAM) device of the at least one computer.

3. The method according to claim 1, wherein the act of storing further comprises an act of allocating a portion of the non-volatile memory to store the information identifying the state of the at least one storage operation before the at least one storage operation is begun, and maintaining the allocation of the portion of the non-volatile memory to store the information identifying the state of the at least one storage operation until the at least one storage operation is completed successfully.

4. The method according to claim 1, wherein the at least one computer is a server, and wherein the act of storing includes an act of storing the information identifying the state of the at least one storage operation in the non-volatile memory of the server.

5. The method according to claim 4, wherein the server is coupled to the storage system by a communication network, and wherein the act of performing at least one storage operation is performed over the communication network.

6. The method according to claim 1, wherein the at least one computer is a network communication system, and wherein the act of storing includes an act of storing the information identifying the state of the at least one storage operation in the non-volatile memory of the network communication system.

7. The method according to claim 1, wherein the act of storing further comprises an act of maintaining a count of one or more I/O operations performed on the first storage location.

8. The method according to claim 7, further comprising an act of maintaining a count of one or more write operations performed on the first storage location.

9. The method according to claim 7, further comprising an act of maintaining a count of one or more read operations performed on the first storage location.

10. The method according to claim 8, further comprising an act of maintaining, separate from the count of one or more write operations, a count of one or more read operations performed on the first storage location.

11. The method according to claim 1, wherein the act of storing further comprises an act of maintaining a log of I/O operations performed on the first storage location.

12. The method according to claim 1, wherein the storage operation performed on the first location is a first write operation performed on the first storage location, and wherein the method further comprises an act of storing a status of a second write operation performed on a second storage location, wherein the second write operation performed on the second storage location corresponds to the first write operation performed on the first storage location, and wherein the method further comprises an act of determining whether:

only one of the first and second write operations completed;

both of the first and second write operations completed; and

neither of the first and second write operations completed.

13. The method according to claim 12, further comprising acts of maintaining a first count of storage operations performed on the first storage location and maintaining a second count of storage operations performed on the second storage location, and wherein the act of determining includes comparing the first count to the second count.

14. The method according to claim 12, wherein the act of storing further comprises an act of maintaining a log of I/O operations performed on the first storage location.

15. The method according to claim 11, wherein the log of I/O operations comprises one or more entries each having at least one of:

a time at which data stored in the first storage location was written; and

a time at which data stored in the first storage location was read.

16. The method according to claim 3, wherein the act of storing further comprises an act of reallocating, after the at least one storage operation is completed successfully, the portion of the non-volatile memory in which the information identifying the state of the operation was stored to store state information for at least one other storage operation on a different storage location.

17. The method according to claim 1, wherein the at least one storage operation is part of a mirror operation performed to at least two volumes of the at least one storage system, and wherein the first storage location is located in one of the at least two volumes.

18. The method according to claim 1, wherein the at least one storage operation is part of a migration process that migrates data between at least two volumes of the at least one storage system, and wherein the first storage location is located in one of the at least two volumes.

19. The method according to claim 1, further comprising an act of determining, when the act of performing the at least one storage operation on the first storage location is completed, another storage location on which at least one other storage operation is to be performed, and storing a state of the at least one other storage operation in the non-volatile memory of the at least one computer.

41. The computer readable medium according to claim 30, wherein the storage operation on the first location is a first write operation performed on the first storage location, and wherein the method according to claim 1, wherein the storage operation performed on the first location is a first write operation performed on the first storage location, and wherein the method further comprises an act of storing a status of a second write operation performed on a second storage location, wherein the second write operation performed on the second storage location corresponds to the first write operation performed on the first storage location, and wherein the method further comprises an act of determining whether:

only one of the first and second write operations completed;

both of the first and second write operations completed; and

neither of the first and second write operations completed.

42. The computer readable medium according to claim 41, further comprising acts of maintaining a first count of storage operations performed on the first storage location and maintaining a second count of storage operations performed on the second storage location, and wherein the act of determining includes comparing the first count to the second count.

43. The computer readable medium according to claim 41, wherein the act of storing further comprises an act of maintaining a log of I/O operations performed on the first storage location.

44. The computer readable medium according to claim 43, wherein the log of I/O operations comprises one or more entries each having at least one of:

a time at which data stored in the first storage location was written; and

a time at which data stored in the first storage location was read.

20. A method for recovering a state of at least one storage process in a computer system including at least one storage system that stores data written by at least one computer in the computer system, the storage process including a write operation that writes the same data to first and second data storage locations in the at least one storage system, the storage process being interrupted during the write operation, the method comprising acts of:

determining whether data stored in the first data storage location and the second data storage location is consistent; and

resuming the at least one storage process in response to the act of determining.

21. The method according to claim 20, wherein the computer system maintains a first count of one or more storage operations performed on the first data storage location and maintains a second count of one or more storage operations performed on the second data storage location, and wherein the act of determining comprises an act of comparing the first count and the second count to determine whether the data is consistent.

22. The method according to claim 20, wherein the computer system maintains first time stamp information identifying when one or more storage operations were performed on the first data storage location and maintains second time stamp information identifying when one or more storage operations were performed on the second data storage location, and wherein the act of determining comprises an act of comparing the first time stamp information and the second time stamp information to determine whether the data is consistent.

23. The method according to claim 20, further comprising an act of, when it is determined that the data stored in the first and second data storage locations are not consistent, making the data stored in the first data storage location and the second data storage location consistent.

24. The method according to claim 23, wherein the act of making further comprises acts of determining which one of the first and second data storage locations stores more recent data, and copying the more recent data to the other data storage location.

25. The method according to claim 23, wherein the act of making further comprises an act of invalidating the data stored in the first and second data storage locations.

26. The method according to claim 20, wherein the at least one storage process is a migration process that migrates data from the first data storage location to the second data storage location, and the act of resuming includes resuming the migration process.

27. The method according to claim 20, wherein the act of determining includes an act of retrieving an indication of a state of the first and second storage locations.

28. The method according to claim 27, wherein the act of retrieving includes an act of retrieving the indication from a non-volatile memory of the at least one host.

29. The method according to claim 27, wherein the at least one host is a server and wherein the act of retrieving includes retrieving the indication from a non-volatile memory of the server.

30. A computer readable medium encoded with instructions for execution on a computer system, the instructions when executed, perform a method for use in a computer system including at least one computer and at least one storage system that is coupled to the at least one computer to store data written thereby, the method comprising acts of:

performing, by the at least one computer, at least one storage operation on a first storage location in the at least one storage system; and

storing, in a non-volatile memory of the at least one computer, information identifying a state of the at least one storage operation performed by the at least one computer.

31. The computer readable medium according to claim 30, wherein the non-volatile memory includes a non-volatile random access memory (NVRAM) memory device of the at least one computer.

32. The computer readable medium according to claim 30, wherein the act of storing further comprises an act of allocating a portion of the non-volatile memory to store the information identifying the state of the at least one storage operation before the at least one storage operation is begun, and maintaining the allocation of the portion of the non-volatile memory to store the information identifying the state of the at least one storage operation until the at least one storage operation is completed successfully.

33. The computer readable medium according to claim 30, wherein the at least one computer is a server, and wherein the act of storing includes an act of storing the information identifying the state of the at least one storage operation in the non-volatile memory of the server.

34. The computer readable medium according to claim 33, wherein the server is coupled to the storage system by a communication network, and wherein the act of performing at least one storage operation is performed over the communication network.

35. The computer readable medium according to claim 30, wherein the at least one computer is a network communication system, and wherein the act of storing includes an act of storing the information identifying the state of the at least one storage operation in the non-volatile memory of the network communication system.

36. The computer readable medium according to claim 30, wherein the act of storing further comprises an act of maintaining a count of one or more I/O operations performed on the first storage location.

37. The computer readable medium according to claim 36, further comprising an act of maintaining a count of one or more write operations performed on the first storage location.

38. The computer readable medium according to claim 36, further comprising an act of maintaining a count of one or more read operations performed on the first storage location.

39. The computer readable medium according to claim 37, further comprising an act of maintaining, separate from the count of one or more write operations, a count of one or more read operations performed on the first storage location.

40. The computer readable medium according to claim 30, wherein the act of storing further comprises an act of maintaining a log of I/O operations performed on the first storage location.

45. The computer readable medium according to claim 32, wherein the act of storing further comprises an act of reallocating, after the at least one storage operation is completed successfully, the portion of the non-volatile memory in which the information identifying the state of the operation was stored to store state information for at least one other storage operation on a different storage location.

46. The computer readable medium according to claim 30, wherein the at least one storage operation is part of a mirror operation performed to at least two volumes of the at least one storage system, and wherein the first storage location is located in one of the at least two volumes.

47. The computer readable medium according to claim 30, wherein the at least one storage operation is part of a migration process that migrates data between at least two volumes of the at least one storage system, and wherein the first storage location is located in one of the at least two volumes.

48. The computer readable medium according to claim 30, further comprising an act of determining, when the act of performing the at least one storage operation on the first storage location is completed, another storage location on which at least one other storage operation is to be performed, and storing a state of the at least one other storage operation in the non-volatile memory of the at least one computer.

49. A computer readable medium encoded with instructions for execution on a computer system, the instructions, when executed, perform a method for recovering a state of at least one storage process in a computer system including at least one storage system that stores data written by at least one computer in the computer system, the storage process including a write operation that writes the same data to first and second data storage locations in the at least one storage system, the storage process being interrupted during the write operation, the method comprising acts of:

determining whether data stored in the first data storage location and the second data storage location is consistent; and

resuming the at least one storage process in response to the act of determining.

50. The computer readable medium according to claim 49, wherein the computer system maintains a first count of one or more storage operations performed on the first data storage location and maintains a second count of one or more storage operations performed on the second data storage location, and wherein the act of determining comprises an act of comparing the first count and the second count to determine whether the data is consistent.

51. The computer readable medium according to claim 50, wherein the computer system maintains first time stamp information identifying when one or more storage operations were performed on the first data storage location and maintains second time stamp information identifying when one or more storage operations were performed on the second data storage location, and wherein the act of determining comprises an act of comparing the first time stamp information and the second time stamp information to determine whether the data is consistent.

52. The computer readable medium according to claim 51, further comprising an act of, when if it is determined that the data stored in the first and second data storage locations are not consistent, making the data stored in the first data storage location and the second data storage location consistent.

53. The computer readable medium according to claim 52, wherein the act of making further comprises acts of determining which of the first and second data storage locations stores more recent data, and copying the more recent data to the other data storage location.

54. The computer readable medium according to claim 52, wherein the act of making further comprises an act of invalidating the data stored in the first and second data storage locations.

55. The computer readable medium according to claim 49, wherein the at least one storage process is a migration process that migrates data from the first data storage location to the second data storage location, and the act of resuming includes resuming the migration process.

56. The computer readable medium according to claim 49, wherein the act of determining includes act of retrieving an indication of a state of the first and second storage locations.

57. The computer readable medium according to claim 56, wherein the act of retrieving includes an act of retrieving the indication from a non-volatile memory of the at least one host.

58. The computer readable medium according to claim 56, wherein the at least one host is a server and wherein the act of retrieving includes retrieving the indication from a non-volatile memory of the server.

59. A method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising an act of:

maintaining, for at least one data storage location of the at least one storage system, a count of a number of input/output (I/O) operations performed on the at least one data storage location.

60. The method according to claim 59, wherein the act of maintaining comprises an act of maintaining a count of a number of read operations performed on the at the least one data storage location.

61. The method according to claim 59, wherein the act of maintaining comprises an act of maintaining a count of one or more write operations performed on the at least one data storage location.

62. The method according to claim 60, wherein the act of maintaining comprises an act of maintaining, separate from the count of the number of read operations, a count of a number of write operations performed on the at least one data storage location.

63. The method according to claim 59, wherein the act of maintaining comprises an act of storing the count in non-volatile storage device of the computer system.

64. The method according to claim 59, wherein the act of maintaining comprises an act of storing the count in the at least one computer that performs the I/O operations.

65. The method according to claim 59, wherein the act of maintaining comprises an act of storing the count in the storage system.

66. A method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising an act of:

maintaining, for at least one data storage location of the at least one storage system, at least one timestamp that identifies a time at which at least one data storage operation was performed on the at least one data storage location.

67. The method according to claim 66, wherein the act of maintaining further comprises an act of updating the at least one timestamp when an operation subsequent to the at least one data storage operation is performed on the data storage location.

68. The method according to claim 66, wherein the act of maintaining comprises maintaining at least one timestamp that identifies a time that a read operation was performed on the at least one data storage location.

69. The method according to claim 66, wherein the act of maintaining comprises maintaining at least one timestamp that identifies a time that a write operation was performed on the at least one data storage location.

70. The method according to claim 69, wherein the act of maintaining comprises an act of maintaining separate timestamps identifying times at which a read operation and a write operation were performed on the at least one data storage location.

71. The method according to claim 66, wherein the act of maintaining comprises an act of storing the at least one timestamp in non-volatile storage device of the computer system.

72. The method according to claim 66, wherein the act of maintaining comprises an act of storing the at least one timestamp in the at least one computer that performs the I/O operations.

73. The method according to claim 66, wherein the act of maintaining comprises an act of storing the at least one timestamp in the storage system.

74. A method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising acts of:

creating a first set of state information that identifies one or more accesses to a storage location of the storage system;

storing the first set of state information in a non-volatile memory location;

and, upon a failure of the computer system, recovering a state of an application based on the first set of state information.

75. The method according to claim 74, further comprising an act of deleting the first set of state information upon a completion of the at least one data storage operation.

76. The method according to claim 74, wherein the data storage operation is a mirroring operation.

77. The method according to claim 74, wherein the data storage operation is a migration operation.

78. The method according to claim 74, wherein the first set of state information is created in response to a write operation being performed on the data storage location.

79. A computer readable medium encoded with instructions for execution in a computer system, the instructions when executed, perform a method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising acts of:

maintaining, for at least one data storage location of the at least one storage system, a count of a number of input/output (I/O) operations performed on the at least one data storage location.

80. The computer readable medium according to claim 79, wherein the act of maintaining comprises an act of maintaining a count of a number of read operations performed on the at least one data storage location.

81. The computer readable medium according to claim 79, wherein the act of maintaining comprises an act of maintaining a count of one or more write operations performed on the at least one data storage location.

82. The computer readable medium according to claim 80, wherein the act of maintaining comprises an act of maintaining, separate from the count of the number of read operations, a count of a number of write operations performed on the at least one data storage location.

83. The computer readable medium according to claim 81, wherein the act of maintaining comprises an act of storing the count in non-volatile storage device of the computer system.

84. The computer readable medium according to claim 79, wherein the act of maintaining comprises an act of storing the count in the at least one computer that performs the I/O operations.

85. The computer readable medium according to claim 79, wherein the act of maintaining comprises an act of storing the count in the storage system.

86. A computer readable medium encoded with instructions for execution in a computer system, the instructions when executed, perform a method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising an act of:

maintaining, for at least one data storage location of the at least one storage system, at least one timestamp that identifies a time at which at least one data storage operation was performed on the at least one data storage location.

87. The computer readable medium according to claim 86, wherein the act of maintaining further comprises an act of updating the at least one timestamp when an operation subsequent to the at least one data storage operation is performed on the data storage location.

88. The computer readable medium according to claim 86, wherein the act of maintaining comprises maintaining at least one timestamp that identifies a time that a read operation was performed on the at least one data storage location.

89. The computer readable medium according to claim 86, wherein the act of maintaining comprises maintaining at least one timestamp that identifies a time that a write operation was performed on the at least one data storage location.

90. The computer readable medium according to claim 89, wherein the act of maintaining comprises an act of maintaining separate timestamps identifying times at which a read operation and a write operation were performed on the at least one data storage location.

91. The computer readable medium according to claim 86, wherein the act of maintaining comprises an act of storing the at least one timestamp in non-volatile storage device of the computer system.

92. The computer readable medium according to claim 86, wherein the act of maintaining comprises an act of storing the at least one timestamp in the at least one computer that performs the I/O operations.

93. The computer readable medium according to claim 86, wherein the act of maintaining comprises an act of storing the at least one timestamp in the storage system.

94. A computer readable medium encoded with instructions for execution in a computer system, the instructions when executed, perform a method for monitoring storage of data in a computer system including at least one computer and at least one storage system that stores data in response to at least one data storage operation executed by the at least one computer, the method comprising acts of:

creating a first set of state information that identifies one or more accesses to a storage location of the storage system;

storing the first set of state information in a non-volatile memory location;

and, upon a failure of the computer system, recovering a state of an application based on the first set of state information.

95. The computer readable medium according to claim 94, further comprising an act of deleting the first set of state information upon a completion of the at least one data storage operation.

96. The computer readable medium according to claim 94, wherein the data storage operation is a mirroring operation.

97. The computer readable medium according to claim 94, wherein the data storage operation is a migration operation.

98. The computer readable medium according to claim 94, wherein the first set of state information is created in response to a write operation being performed on the data storage location.