Network independent profiling of applications for automatic partitioning and distribution in a distributed computing environment

• US 6,983,463 B1
• Filed: 11/20/1998
• Issued: 01/03/2006
• Est. Priority Date: 10/02/1998
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hunt, Galen C.
• Primary Examiner(s): Follansbee, John
• Assistant Examiner(s): PATEL, HARESH N
• Application Number:
  US09/197,080
• Time in Patent Office: 2,601 Days
• Field of Search: 709/316, 709/315, 709/318, 709/328, 709/331, 709/332, 709/200, 709/201, 709/202, 709/203, 709/217, 719/312, 719/313, 719/314, 719/315, 719/316, 719/317, 719/318, 719/332, 719/328, 719/331, 718/107, 718/108, 718/105, 717/101, 717/148
• US Class Current: 719/316
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/30 Profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

An instrumentation system profiles an application using structural metadata description of the application. Units (such as COM objects) of the application have strongly-typed, binary-standard interfaces, and are profiled, for example, using an executable file and DLLs for the application. A structural metadata description of the application includes compiled, interface-level type information used to identify and measure interaction between units of the application. For example, the type information is produced by analyzing IDL information. Profiling results in an application profile that includes description of the static relationships and/or dynamic interactions between units of the application. The application profile is combined with a network profile that describes a distributed computing environment. Analysis of the result yields a distribution plan, which, for example, reduces costs associated with communication between the units. During execution, units of the application are distributed through the distributed computing environment according to the distribution plan.

Claims

1. A method to profile an application having plural units, wherein the plural units communicate across strongly-typed, binary-standard interfaces, and wherein a type file describes the interfaces of the plural units, the method comprising:

generating a structural metadata description of the application from the type file, wherein the structure metadata description comprises compiled interface-level type description;

profiling the application using the structural metadata description, resulting in an application profile; and

reconfiguring the application based on the application profile, wherein the step of reconfiguring comprises;

combining the application profile with a network profile;

analyzing the combination of the application and network profiles;

generating a distribution plan; and

during execution of the application, distributing the plural units of the application in a distributed computing environment according to the distribution plan.

2. The method of claim 1 wherein the step of generating a structural metadata description comprises:

receiving a source code type description of the interfaces;

statically analyzing the source code description; and

producing the structural metadata description of the application.

3. The method of claim 1 wherein the step of generating a structural metadata description comprises:

receiving a compiled type file comprising information descriptive of the interfaces; and

producing the structural metadata description of the application.

4. The method of claim 1 wherein the step of profiling comprises:

determining static relationships between the plural units of the application; and

generating the application profile, wherein the application profile models the static relationships.

5. The method of claim 1 wherein the step of profiling comprises:

determining dynamic interaction between the plural units of the application through interfaces described in the interface-level type description; and

generating the application profile, wherein the application profile models the dynamic interaction.

6. The method of claim 1 wherein the step of profiling comprises:

measuring the number and size of communications through the interfaces of the plural units using the structural metadata description of the application; and

generating the application profile, wherein the application profile is a log of the communications between the plural units.

7. The method of claim 1 wherein the step of profiling comprises:

measuring the size of communications through the interfaces of the plural units using the structural metadata description of the application; and

generating the application profile, wherein the application profile is a log of the communications between the plural units.

8. The method of claim 7 wherein for a communication, the log stores data representing a sending unit, a receiving unit, and the size of the communication.

9. The method of claim 1 wherein the plural units of the application reside on plural computers in the distributed computing environment, and wherein the step of profiling comprises:

timing communications sent between the plural units; and

generating the application profile, wherein the application profile is a log of the communications sent between the plural units.

10. The method of claim 1 wherein the step of profiling comprises:

timing the execution of the plural units; and

generating the application profile, wherein the application profile describes the behavior of the plural units.

11. The method of claim 1 wherein the application is available for profiling only as an application binary.

12. The method of claim 11 wherein the application binary comprises an executable file.

13. The method of claim 12 wherein the application binary further comprises one or more dynamic link libraries.

14. A computer-readable medium having computer-executable instructions to perform the method of claim 1.

15. The method of claim 1 wherein the network profile comprises a description of an idealized network of plural identical computers with no communication costs.

16. The method of claim 1 wherein the network profile comprises a description of the capabilities of plural computers in a physical network.

17. A method to profile an application for partitioning and distributing plural units of the application in a distributed computing environment, wherein the plural units communicate across strongly-typed, binary-standard interfaces, wherein a type file describes the interfaces of the plural units, and wherein the application is available for profiling only as an application binary, the method comprising:

generating a static interface metadata description of the application from the type file, wherein the static interface metadata description comprises type information about the interfaces of the plural units of the application;

profiling the application by using the static interface metadata description on the application binary, resulting in an application profile;

combining the application profile with a network profile;

analyzing the combination of the application and network profiles;

generating a distribution plan; and

during execution of the application, distributing the plural units in the distributed computing environment according to the distribution plan.

18. The method of claim 17 wherein the step of generating a static interface metadata description comprises:

receiving a source code type description of the interfaces;

analyzing the source code description with static analysis; and

producing the static interface metadata description of the interfaces.

19. The method of claim 17 wherein the step of generating a static interface metadata description comprises:

receiving a compiled type file comprising information descriptive of the interfaces; and

producing the static interface metadata description of the interfaces.

20. The method of claim 17 wherein the step of profiling comprises:

determining dynamic interaction between the plural units of the application through interfaces described in the static interface metadata description; and

generating the application profile, wherein the application profile models the dynamic interaction.

21. The method of claim 17 wherein the step of profiling comprises:

measuring the number and size of communications through the interfaces of the plural units using the static interface metadata description of the application; and

generating the application profile, wherein the application profile is a log of the communications between the plural units.

22. The method of claim 17 wherein the step of profiling comprises:

measuring the size of communications through the interfaces of the plural units using the static interface metadata description of the application; and

generating the application profile, wherein the application profile is a log of the communications between the plural units.

23. The method of claim 22 wherein for a communication, the log stores data representing a sending unit, a receiving unit, and the size of the communication.

24. A computer-readable medium having computer-executable instructions to perform the method of claim 17.

25. A method to partition and distribute plural units of an application in a distributed computing environment, the method comprising:

reading a first set of descriptors describing the application;

reading a second set of descriptors including measurements of the distributed computing environment;

analyzing the first and second sets of descriptors;

generating a distribution plan for the application, the distribution plan comprising information specifying a partitioning of the plural units for distribution in the distributed computing environment;

executing the application; and

during execution of the application, distributing the plural units in the distributed computing environment according to the distribution plan.

26. The method of claim 25 wherein the first set of descriptors comprises metadata describing the structure of the application.

27. The method of claim 25 wherein the first set of descriptors comprises information describing one or more location constraints on the placement of the plural units of the application in the distributed computing environment.

28. The method of claim 25 wherein the first set of descriptors comprises metadata describing the behavior of the application.

29. The method of claim 25 wherein the plural units have strongly-typed, binary-standard interfaces, and wherein the first set of descriptors describes communications through these interfaces in a set of one or more profiling scenarios.

30. The method of claim 25 wherein the second set of descriptors comprises measurements of current capabilities of plural computers of a physical network of computers.

31. The method of claim 25 wherein the second set of descriptors comprises estimates of average latency and average bandwidth for a physical network.

32. The method of claim 25 wherein the step of analyzing comprises:

grouping the plural units based on the first and second sets of descriptors according to a grouping scheme.

33. The method of claim 25 wherein the step of analyzing comprises:

representing the first and second sets of descriptors as a commodity flow model; and

finding a minimum cut/maximum flow for the commodity flow model.

34. The method of claim 25 wherein the step of generating a distribution plan comprises:

associating the plural units with one or more locations in the distributed computing environment; and

producing the distribution plan, wherein the distribution plan comprises a mapping of the plural units of the application to locations in the distributed computing environment.

35. A computer-readable medium having computer-executable instructions to perform the method of claim 25.

36. The method of claim 25 wherein the second set of descriptors describes a physical network of computers;

wherein the step of analyzing comprises finding a minimum cut/maximum flow for a commodity flow model based on the first and second sets of descriptors, and wherein the distribution plan comprises a mapping of the plural units of the application to one or more computers in the physical network.

37. A computer-readable medium having computer-executable instructions to perform the method of claim 36.

38. The method of claim 25 wherein system services of the distributed computing environment support distributing the plural units.

39. The method of claim 25 wherein a combination of system services from a dedicated system and the distributed computing environment supports distributing the plural units.

40. The method of claim 25 further comprising:

defining a threshold for execution of the application in the distributed computing environment, the threshold describing an accepted deviation from the expected performance of the application;

during execution of the application, if the threshold is exceeded;

noting the recent behavior of the application in the distributed computing environment;

generating a second distribution plan for the application, the second distribution plan assimilating the recent behavior;

executing the application; and

during execution of the application, distributing the plural units in the distributed computing environment according to the second distribution plan.

41. The method of claim 40 wherein the step of noting the recent behavior comprises:

generating a new first set of descriptors; and

re-analyzing the first and second sets of descriptors.

42. The method of claim 40 wherein the step of noting the recent behavior comprises:

generating a new second set of descriptors; and

re-analyzing the first and second sets of descriptors.