System workload characterization

• US 7,203,621 B1
• Filed: 06/06/2002
• Issued: 04/10/2007
• Est. Priority Date: 06/06/2002
• Status: Active Grant

• Current Assignee: Mosaid Technologies Incorporated (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Anderson, Eric, Wilkes, John, Veitch, Alistair
• Primary Examiner(s): Meky; Moustafa M.
• Application Number:
  US10/164,534
• Time in Patent Office: 1,769 Days
• Field of Search: 702118-121, 702/182, 702/186, 702/187, 717/131, 709/223, 713/340, 714 47- 48
• US Class Current: 702/182
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

A technique of workload characterization for a computer system. Records of activity in the system are generated and identified as corresponding to each of a plurality of selected time periods. The records of activity for each selected time period are processed to form a characterization of the activity within each corresponding time period. Further, whether little or no activity occurs during a time period can be indicated by a binary representation. Periods of simultaneous activity of processes can then be determined by applying simple logical operations to the binary representations. Resources of the system may be reallocated based on the characterizations of the activity.

Claims

1. A method of workload characterization for a computer system for which a machine generates records of activity for the computer system, the method comprising collecting records of activity occurring during each of a plurality of selected time periods, each of the time periods starting at intervals and the time periods and intervals selected such that each interval is longer than the corresponding time period and records of activity occurring between successive time periods are not collected, and processing the collected records of activity for each selected time period to form a characterization of the activity within each corresponding time period.

2. The method according to claim 1, further comprising reallocating resources of the computer system based on the characterizations of the activity.

3. The method according to claim 1, wherein durations of at least one of said selected time periods and intervals are varied while said records are being collected.

4. The method according to claim 3, wherein said durations are selected randomly.

5. The method according to claim 3, wherein said durations are adaptively selected based on previously collected records.

6. The method according to claim 3, wherein said durations are selected in accordance with a predetermined sequence.

7. The method according to claim 1, wherein the computer system comprises a data storage system and wherein the activity records are records of I/O activity for the storage system.

8. The method according to claim 7, further comprising sorting said records of I/O activity according to a plurality of streams and wherein each characterization is for I/O activity of a corresponding stream.

9. The method according to claim 8, each stream being directed to a corresponding store.

10. The method according to claim 8, wherein each characterization for a stream includes one or more on times and one or more off times for each selected time period.

11. The method according to claim 10, wherein an overlap fraction is determined for each of a plurality of the time periods based on the on times and off times that occur within the same selected time period for a pair of the streams.

12. The method according to claim 8, wherein each characterization includes an indication of whether an activity level exceeding a predetermined threshold of activity occurred during the corresponding time period.

13. The method according to claim 12, wherein said indication is a binary value.

14. The method according to claim 13, further comprising performing a logic operation on selected ones of said binary values corresponding to a selected one of the time periods wherein a result of said logic operation indicates simultaneous activity occurred during the selected time period.

15. The method according to claim 14, wherein said logic operation is performed on a pair of binary values.

16. The method according to claim 14, wherein said logic operation is performed on more than two binary values.

17. The method according to claim 8, wherein each characterization includes a plurality of values, each value being for a corresponding parameter and each value including a statistical measure for the corresponding parameter.

18. The method according to claim 17, further comprising forming a data structure of said values wherein said data structure comprises a table having a first dimension for each stream or store and a second dimension for each time period and each entry at an intersection of a row and column including the corresponding values.

19. The method according to claim 17, the parameters including one or more parameters selected from the group consisting of:

a request rate parameter, a request size parameter, a run-count parameter and a queue size parameter.

20. The method according to claim 1, further comprising discarding portions of the characterization of the activity.

21. The method according to claim 20, said discarding being based on aging of the portions.

22. The method according to claim 20, said discarding being based on a level of activity reflected in the portions.

23. The method according to claim 12, said discarding being based on similarity of the portions.

24. The method according to claim 1, further comprising discarding records of the activity based on aging of the records.

25. The method according to claim 1, further comprising discarding records of the activity based on a level of activity reflected in the records.

26. The method according to claim 1, further comprising discarding records of the activity based on similarity of the records.

27. An article of manufacture for a computer having access to records of I/O activity for a data storage system, the article comprising machine-readable code for instructing the computer to collect records of I/O activity occurring during each of a plurality of selected time periods, each of the time periods starting at intervals and the time periods and intervals selected such that each interval is longer than the corresponding time period and records of activity occurring between successive time periods are not collected, and to process the records of I/O activity for each selected time period to form a characterization of the I/O activity within each corresponding time period.

28. The article according to claim 27, wherein durations of at least one of said selected time periods and intervals are varied while said records are being collected.

29. The method according to claim 28, wherein said durations are selected randomly.

30. The method according to claim 28, wherein said durations are adaptively selected based on previously collected records.

31. The method according to claim 28, wherein said durations are selected in accordance with a predetermined sequence.

32. The method according to claim 27, wherein each characterization includes a plurality of values, each value being for a corresponding parameter and wherein each value includes a statistical measure for the corresponding parameter.

33. A method of workload characterization for a computer system for which a machine generates records of activity for the computer system, the method comprising collecting records of activity occurring during each of a plurality of selected time periods, each of the time periods starting at intervals and durations of at least one of said selected time periods and intervals being varied while said records are being collected, and processing the collected records of activity for each selected time period to form a characterization of the activity within each corresponding time period.

34. The method according to claim 33, wherein said durations are selected randomly.

35. The method according to claim 33, wherein said durations are adaptively selected based on previously collected records.

36. The method according to claim 33, wherein said durations are selected in accordance with a predetermined sequence.