Method and apparatus for monitoring update activity in a data storage facility

• US 6,662,197 B1
• Filed: 06/25/1999
• Issued: 12/09/2003
• Est. Priority Date: 06/25/1999
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Hall, Denis, Murphy, Daniel A., LeCrone, Douglas E.
• Primary Examiner(s): Von Buhr, Maria N.
• Application Number:
  US09/344,999
• Time in Patent Office: 1,628 Days
• Field of Search: 707/10, 707/200-204, 711/133-146, 711/159-162, 709/200-203, 709/212-219
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method and apparatus for monitoring update activity, particularly in the form of write requests, to a data storage facility, over at least one cycle to identify any write operation or update that occurs to that data storage facility on a track-by-track basis. At the end of each cycle a list of changed tracks is stored in a data set. After information has been accumulated by a collector application, a reporter application manipulates the data to obtain information concerning the track changes and to estimate communication path requirements.

Claims

1. A method for obtaining information about the number of updates to data blocks in data block locations in a disk storage facility over a time interval wherein said disk storage facility stores data groups as a plurality of data blocks, said method comprising the steps of:

A) defining a data group set of at least one of the data blocks,

B) defining an interval corresponding to the time period for which update information is desired,

C) recording, during the defined interval, only a first update to each data block location for data in the data group set,

D) after the defined interval transferring the information obtained by said recording to the data group set,

E) determining, from the information in the data group set, the total number of data block locations in the data group set that were updated at least one time during the defined interval, and

F) converting the total number of data block locations changed during into bandwidth based information about all the transfers involved in updating during the defined interval.

2. A method as recited in claim 1 wherein said step of recording includes defining a flag for each data block and altering the state of each flag upon receiving a first update to a corresponding data block.

3. A method as recited in claim 2 wherein said step of recording additionally includes:

i) establishing a first state for each flag at the beginning of the defined interval, and

ii) establishing a second state for an individual flag when the corresponding data block is updated a first time, said recording counting the total number of flags in the second state at the end of the defined interval.

4. A method as recited in claim 2 wherein said recording step includes storing in an other dataset an identification of the time interval, data group and the defined flags.

5. A method as recited in claim 2 wherein the defined interval is divided into subintervals and said recording step includes storing, for each data group in the data group set, identifications of the subinterval and data group and the defined flags.

6. A method as recited in claim 5 wherein said step of recording includes:

i) establishing a first state for each flag at the beginning of each sample subinterval, and

ii) establishing a second state for an individual flag when the corresponding data block is updated for a first time, said recording counting the total number of flags in the second state at the end of the sample subinterval.

7. A method as recited in claim 1 wherein updates to the data storage facility are copied over a data communications path having predetermined characteristics to a data facility, said method additionally comprising the step of determining from information obtained during said recording in combination with a first characteristic of the data communications path to obtain a second characteristic of the data communications path.

8. A method as recited in claim 7 wherein the data facility operates as a mirror for the data storage facility for maintaining a synchronized copy of the data therein and wherein the first and second characteristics are, respectively, bandwidth and time for the mirror to achieve synchronism for the recorded update activity for that given bandwidth.

9. A method as recited in claim 7 wherein the data facility operates as a mirror for the data storage facility for maintaining a synchronized copy of the data therein and wherein the first and second characteristics are, respectively, the time required for the mirror to achieve synchronism for the recorded update activity and the bandwidth required to achieve that resynchronization time, respectively.

10. A method for determining, from a local site with a local data storage facility, bandwidth related characteristics for a communications path that transfers data, produced by update operations, between first and second remote data storage facilities over a communications path wherein each of the data storage facilities stores data in datasets on disk tracks, said method comprising the steps of:

A) defining a set of disk tracks in the local site data storage facility,

B) defining a time interval,

C) recording an identification of each disk track in the local data storage facility that is updated during the interval,

D) upon completion of the time interval determining the number of tracks in the defined set of disk tracks that were updated during the time interval, and

E) converting the information based upon the number of disk tracks that were changed during the defined time interval into bandwidth related properties of the communications oath.

11. A method as recited in claim 10 wherein said recording includes:

i) defining a table with a position corresponding to each of the defined disk tracks and each position being set to a first state, and

ii) responding to a first update operation to a defined disk track during the defined time interval to a defined disk track by setting the corresponding position to a second state.

12. A method as recited in claim 11 wherein said recording occurs over a plurality of defined time intervals and said recording sets all the table positions to the first state at the beginning of each of the time intervals.

13. A method as recited in claim 12 additionally comprising the step of storing the contents of the table after each of the time intervals with a date-time stamp corresponding to the interval.

14. A method as recited in claim 13 wherein said determining step includes:

i) defining a report to be generated including a time frame including at least one time interval for the report,

ii) processing the stored contents of the table with date-time stamps within the time frame, and

iii) generating a report with the numbers of track changes recorded in the processed contents.

15. A method as recited in claim 14 wherein the local disk storage facility includes a controller including a set of logical volumes and the defined disk tracks include all the disk tracks in the controller, said step of defining a report including generating a corresponding reporter command, said processing including combining all the track changes for the controller for all of the time intervals.

16. A method as recited in claim 15 wherein said processing includes combining the date-time stamped contents of each table in the report time frame in a logical OR operation to obtain a final table and counting the total number of positions in the final table set to the second state.

17. A method as recited in claim 16 additionally comprising the step of generating the reporter command with one parameter specifying a resynchronization time in which corresponding updates should be completed between the first and second remote data storage facilities, said processing providing the required bandwidth of the communications path to provide that resynchronization time.

18. A method as recited in claim 17 wherein the communications path includes at least one path with a characteristic bandwidth, said bandwidth being further determined by dividing the required bandwidth by the number of paths to obtain the characteristic bandwidth.

19. A method as recited in claim 17 wherein the communications path will include at least one path with a characteristic bandwidth, said required bandwidth being further determined by dividing the required bandwidth by the characteristic bandwidth to determine the number of required paths.

20. A method as recited in claim 16 additionally comprising the step of generating the reporter command with one parameter specifying an available bandwidth of the communications path, said processing providing a resynchronization time during which all updates to the first remote data storage facility will be transferred over the communications path to the second remote data storage facility.

21. A method as recited in claim 20 wherein the communications path includes at least one path having a characteristic bandwidth, the bandwidth provided with the command being the product of the number of paths and the characteristic bandwidth.

22. A method as recited in claim 14 wherein the local data storage facility includes a plurality logical volumes and the defined disk tracks include all the disk tracks in the controller, said step of generating the reporter command including defining a logical volume report for a set of at least one logical volume, said processing combining all the track changes for each of the defined logical volumes.

23. A method as recited in claim 22 wherein said processing includes combining all the contents of the table for a logical volume in a logical OR operation to obtain a final table and summing the positions in the final table that are set to the second state.

24. A method as recited in claim 16 wherein the local disk storage facility includes a plurality logical volumes and each logical volume includes at least one dataset, said step of generating the reporter command defining a dataset report with a set of at least one dataset, said processing combining all the track changes for each of the defined datasets.

25. A method as recited in claim 24 wherein said processing additionally includes defining a mask of the tracks forming the defined datasets, generating a final table that is the logical OR of all the tables in the logical volume containing the defined datasets and thereafter combining the mask and the final table in a logical AND operation.

26. Apparatus for obtaining information about the number of updates to data blocks stored in data block locations in a disk storage facility over a time interval wherein said disk storage facility stores data groups as a plurality of data blocks, said apparatus comprising:

A) means for defining a data group set of at least one of the data blocks,

B) means for defining an interval corresponding to the time period for which update information is desired,

C) a collector application that records, during the defined interval, only a first update to each data block location for data in the data group set,

D) means responsive for transferring the information obtained by said collector application to the data group set, and

E) a reporter application, activated after the defined interval, that determines, for converting the information in the data group set obtained by said transfer means, the total number of data data block locations in the data group set that were updated at least one time during the defined interval and bandwidth based information about all the transfers involved in the updating during the defined period.

27. Apparatus as recited in claim 26 wherein said collector application includes means for defining a flag for each data block, the state of each flag being altered upon receiving a first update to a corresponding data block.

28. Apparatus as recited in claim 27 wherein said collector application includes:

i) means for establishing a first state for each flag at the beginning of the defined interval, and

ii) means for establishing a second state for an individual flag when the corresponding data block is updated a first time, said recording counting the total number of flags in the second state at the end of the defined interval.

29. Apparatus as recited in claim 27 wherein said collector application includes means for storing in an other dataset an identification of the time interval, data group and all the defined flags.

30. Apparatus as recited in claim 27 wherein the defined interval is divided into subintervals and said collector application includes means for storing, for each data group in the data group set, identifications of the subinterval and data group and the defined flags.

31. Apparatus as recited in claim 30 wherein said collector application includes means for establishing a first state for each flag at the beginning of each sample subinterval.

32. Apparatus as recited in claim 26 wherein updates to the data storage facility are copied over a data communications path having predetermined characteristics to a data facility, said apparatus additionally comprising a reporter application that processes the dataset produced by said collector application in combination with a first characteristic of the communications path to obtain a second characteristic of the communications path.

33. Apparatus as recited in claim 26 wherein the data facility operates as a mirror for the data storage facility for maintaining a synchronized copy of the data therein, said apparatus additionally comprising a reporter application that process the dataset produced by said collector application, first and second characteristics are, respectively, bandwidth and time for the mirror to achieve synchronism for the recorded update activity.

34. Apparatus as recited in claim 26 wherein the data facility operates as a mirror for the data storage facility for maintaining a synchronized copy of the data therein and wherein the first and second characteristics are, respectively, the time required for the mirror to achieve synchronism for the recorded update activity and the bandwidth required to achieve that resynchronization time, respectively.

35. Apparatus for determining, from a local site with a local data storage facility, bandwidth related characteristics for a communications path that transfers data, produced by update operations, between first and second remote data storage facilities over a communications path wherein each of the data storage facilities stores data in datasets on disk tracks, said method comprising the steps of:

A) means for defining a set of disk tracks in the local site data storage facility,

B) means for defining a time interval,

C) a collector application for recording an identification of each disk track in the local data storage facility that is updated during the interval, and

D) a reporter application for converting the number of disk tracks in the defined set of disk tracks that were updated during the time interval into bandwidth related properties of the communications path.

36. Apparatus as recited in claim 35 wherein said collector application includes means for defining a table with a position corresponding to each of the defined disk tracks and each position being set to a first state, and said apparatus includes means for responding to a first update operation during the defined time interval to a defined disk track by setting the corresponding position to a second state.

37. Apparatus as recited in claim 36 wherein said collector application operates over a plurality of defined time intervals and includes means for setting all the table positions to the first state at the beginning of each of the time intervals.

38. Apparatus as recited in claim 37 additionally comprising means for storing the contents of the table after each of the time intervals with a date-time stamp corresponding to the interval.

39. Apparatus as recited in claim 38 wherein said reported application operates in response to a command defining a report to be generated including a time frame for the report and includes:

i) means for processing the stored contents of the table with date-time stamps within a time frame including at least one time interval, and

ii) means for generating a report with the numbers of track changes recorded in the processed contents.

40. Apparatus as recited in claim 39 wherein the local disk storage facility includes a controller including a set of logical volumes and the defined disk tracks include all the disk tracks in the controller, said reported application being responsive to a reporter command specifying a controller report and including means for combining all the track changes for the controller for all the time intervals.

41. Apparatus as recited in claim 40 wherein said combining means includes means for combining the date-time stamped contents of each table in the report time frame in a logical OR operation to obtain a final table and means for counting the total number of positions in the final table set to the second state.

42. Apparatus as recited in claim 41 wherein the reporter command has one parameter specifying a resynchronization time in which corresponding updates should be completed between the first and second remote data storage facilities, said report generating means providing the required bandwidth of the communications path to provide that resynchronization time.

43. Apparatus as recited in claim 42 wherein the communications path includes at least one path with a characteristic bandwidth, said report generating means including means for dividing the required bandwidth by the number of paths to obtain the characteristic bandwidth.

44. Apparatus as recited in claim 42 wherein the communications path will include at least one path with a characteristic bandwidth, said report generating means including means for dividing the required bandwidth by the characteristic bandwidth to determine the number of required paths.

45. Apparatus as recited in claim 41 wherein the reporter command specifies an available bandwidth of the communications path, said report generating means providing a resynchronization time during which all updates to the first remote data storage facility will be transferred over the communications path to the second remote data storage facility.

46. Apparatus as recited in claim 45 wherein the communications path includes at least one path having a characteristic bandwidth, said report generating means including means for generating the product of the number of paths and the characteristic bandwidth.

47. Apparatus as recited in claim 39 wherein the local data storage facility includes a plurality logical volumes and the defined disk tracks include all the disk tracks in the controller, said reporter application responding to a logical volume report command identifying a set of at least one logical volume and said combining means combining all the track changes for each of the defined logical volumes.

48. Apparatus as recited in claim 47 wherein said combining means includes means for combining all the contents of the table for a logical volume in a logical OR operation to obtain a final table and means for summing the positions in the final table that are set to the second state.

49. Apparatus as recited in claim 41 wherein the local disk storage facility includes a plurality logical volumes and each logical volume includes at least one dataset, said reported application responding to a command defining a dataset report with an identification of a set of at least one dataset, said combining means combining all the track changes for each of the defined datasets.

50. Apparatus as recited in claim 49 wherein said reported application additionally includes means for defining a mask of the tracks forming the set of defined datasets, means for generating a final table that is the logical OR of all the tables in the logical volume containing the defined datasets and means for combining the mask and the final table in a logical AND operation.