Parallel runtime execution on multiple processors

• US 8,286,196 B2
• Filed: 05/03/2007
• Issued: 10/09/2012
• Est. Priority Date: 05/03/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Munshi, Aaftab, Sandmel, Jeremy
• Primary Examiner(s): Ho, Andy
• Assistant Examiner(s): SEYE, ABDOU K
• Application Number:
  US11/800,319
• Publication Number:
  US 20080276262A1
• Time in Patent Office: 1,986 Days
• Field of Search: 719/328
• US Class Current: 719/328
• CPC Class Codes: G06F 8/41 Compilation
• CPC Class Codes: G06F 8/447 Target code generation
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/44542 Retargetable
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications

Abstract

A method and an apparatus that schedule a plurality of executables in a schedule queue for execution in one or more physical compute devices such as CPUs or GPUs concurrently are described. One or more executables are compiled online from a source having an existing executable for a type of physical compute devices different from the one or more physical compute devices. Dependency relations among elements corresponding to scheduled executables are determined to select an executable to be executed by a plurality of threads concurrently in more than one of the physical compute devices. A thread initialized for executing an executable in a GPU of the physical compute devices are initialized for execution in another CPU of the physical compute devices if the GPU is busy with graphics processing threads. Sources and existing executables for an API function are stored in an API library to execute a plurality of executables in a plurality of physical compute devices, including the existing executables and online compiled executables from the sources.

Claims

1. A computer implemented method comprising:

selecting, during runtime of an application running in a first processing unit, in response to a first API (Application Programming Interface) request having a requirement from the application, a second data processing unit according to the requirement;

loading, during the runtime, in response to a second API request from the application one or more executables for the selected second processing unit to perform a data processing task of the application; and

causing, during the runtime, in response to a third API request from the application, execution of one of the one or more executables for the second processing unit,

wherein the one or more executables include description data for the one of the one or more executables,

wherein the selecting is based on the description data,

wherein the one of the one or more executables is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the one of the one or more executables includes description data indicating the latest version for the second processing unit, and

wherein the one of the one or more executables is backward compatible with the second processing unit.

2. The computer implemented method of claim 1, wherein the first processing unit and the second processing unit are central processing units (CPUs) or graphics processing units (CPUs).

3. The computer implemented method of claim 1, wherein the selected one of the one or more executables is associated with the second API request.

4. The computer implemented method of claim 1, wherein the one or more executables include description data for at least one of the one or more executables, the description data including versions and types of supported processing units.

5. The computer implemented method of claim 4, wherein the loading comprises:

comparing the description data and information of the second processing unit; and

compiling the one of the one or more executables from the source online for the second processing unit.

6. The computer implemented method of claim 5, wherein the one of the one or more executables is associated with the first API request.

7. The computer implemented method of claim 5, wherein the compiling is based on the comparing indicating the at least one of the one or more executables is not optimized for the second processing unit.

8. The computer implemented method of claim 5, wherein the compiling is based on the comparing indicating the second processing unit is not supported by the at least one of the one or more executables.

9. The computer implemented method of claim 5, wherein the compiling comprises generating updated description data for the one of the one or more executables including a version of the second processing unit;

and

storing the one of the one or more executables including the updated description data.

10. The computer implemented method of claim 9, wherein the one of the one or more executables is stored to replace the at least one of the one or more executables.

11. The computer implemented method of claim 1, wherein the one or more executables include a source compiled to generate the one or more executables.

12. The computer implemented method of claim 11, wherein the source is loaded from the application via the second API.

13. The computer implemented method of claim 11, wherein the source is loaded from a library associated with the one or more executables.

14. The computer implemented method of claim 1, wherein the selecting the second data processing unit comprises creating a compute kernel object representing the second data processing unit and wherein the execution is based on an execution instance associated with the compute kernel object.

15. The computer implemented method of claim 14, wherein the execution is ordered according to an execution queue, the method further comprising:

updating the execution queue with the execution instance; and

selecting the execution instance from the execution queue for the execution.

16. The computer implemented method of claim 15, wherein the execution queue stores the execution instance and one or more other execution instances, and wherein the execution instance is selected without dependency conditions on the other execution instances.

17. A computer implemented method comprising:

generating, during runtime of an application running in a first processing unit, a first API (Application Programming Interface) request having requirement from the application to select a second data processing unit;

generating, during the runtime, a second API request from the application running in the first processing unit to load one or more executables for the selected second data processing unit to perform a data processing task of the application; and

generating, during the runtime, a third API request from the application to execute an executable selected from the one or more executables for execution in the second processing unit,

wherein the one or more executables include description data for selected executable,

wherein the second data processing unit is selected is based on the description data,

wherein the selected executable is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the selected executable includes description data indicating the latest version for the second processing unit, and

wherein the selected executable is backward compatible with the second processing unit.

18. The computer implemented method of claim 17, wherein the first processing unit and the second processing unit are central processing units (CPUs) or graphics processing units (GPUs).

19. The computer implemented method of claim 17, wherein the first API request is associated with a source from which the one or more executables are compiled from.

20. The computer implemented method of claim 19, wherein the selected executable is compiled offline from the source.

21. The computer implemented method of claim 17, wherein the selected executable is compiled online in response to the second API request.

22. The computer implemented method of claim 17, wherein the description data includes a version and a type compatible with the second processing unit.

23. A machine readable non-transitory storage medium containing executable instructions which cause a data processing system to perform a method comprising:

selecting, during runtime of an application running in a first processing unit, in response to a first API (Application Programming Interface) request having requirement from the application, a second data processing unit according to the requirement;

loading, during the runtime, in response to a second API request from an application running in the first processing unit, one or more executables for the selected second processing unit to perform a data processing task of the application; and

causing, during the runtime, in response to a third API request from the application, execution of one of the one or more executables for the second processing unit,

wherein the one or more executables include description data for the one of the one or more executables,

wherein the selecting is based on the description data,

wherein the one of the one or more executables is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the one of the one or more executables includes description data indicating the latest version for the second processing unit, and

wherein the one of the one or more executables is backward compatible with the second processing unit.

24. The machine readable medium of claim 23, wherein the first processing unit and the second processing unit are central processing units (CPUs) or graphics processing units (CPUs).

25. The machine readable medium of claim 23, wherein the selected one of the one or more executables is associated with the second API request.

26. The machine readable medium of claim 23, wherein the one or more executables include description data for at least one of the one or more executables, the description data including versions and types of supported processing units.

27. The machine readable medium of claim 26, wherein the loading comprises:

comparing the description data and information of the second processing unit; and

compiling the one of the one or more executables from the source online for the second processing unit.

28. The machine readable medium of claim 27, wherein the one of the one or more executables is associated with the first API request.

29. The machine readable medium of claim 27, wherein the compiling is based on the comparing indicating the at least one of the one or more executables is not optimized for the second processing unit.

30. The machine readable medium of claim 27, wherein the compiling is based on the comparing indicating the second processing unit is not supported by the at least one of the one or more executables.

31. The machine readable medium of claim 27, wherein the compiling comprises:

generating updated description data for the one of the one or more executables including a version of the second processing unit; and

storing the one of the one or more executables including the updated description data.

32. The machine readable medium of claim 31, wherein the one of the one or more executables is stored to replace the at least one of the one or more executables.

33. The machine readable medium of claim 23, wherein the one or more executables include a source compiled to generate the one or more executables.

34. The machine readable medium of claim 33, wherein the source is loaded from the application via the second API.

35. The machine readable medium of claim 33, wherein the source is loaded from a library associated with the one or more executables.

36. The machine readable medium of claim 23, wherein the selecting is based on the description data selecting the second data processing unit comprises creating a compute kernel object representing the second data processing unit and wherein the execution is based on an execution instance associated with the compute kernel object.

37. The machine readable medium of claim 36, wherein the execution is ordered according to an execution queue, the method further comprising:

updating the execution queue with the execution instance; and

selecting the execution instance from the execution queue for the execution.

38. The machine readable medium of claim 37, wherein the execution queue stores the execution instance and one or more other execution instances, and wherein the execution instance is selected without dependency conditions on the other execution instances.

39. A machine readable non-transitory storage medium containing executable instructions which cause a data processing system to perform a method comprising:

generating, during runtime of an application running in a first processing unit, a first API (Application Programming Interface) request having a requirement from the application to select a second data processing unit;

generating, during the runtime, a second API request from the application running in the first processing unit to load one or more executables for the selected second processing unit to perform a data processing task of the application; and

generating, during the runtime, a third API request from the application to execute an executable selected from the one or more executables for execution in the second processing unit,

wherein the one or more executables include description data for selected executable,

wherein the second data processing unit is selected is based on the description data,

wherein the selected executable is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the selected executable includes description data indicating the latest version for the second processing unit, and

wherein the selected executable is backward compatible with the second processing unit.

40. The machine readable medium of claim 39, wherein the first processing unit and the second processing unit are central processing units (CPUs) or graphics processing units (GPUs).

41. The machine readable medium of claim 39, wherein the first API request is associated with a source from which the one or more executables are compiled from.

42. The machine readable medium of claim 41, wherein the selected executable is compiled offline from the source.

43. The machine readable medium of claim 39, wherein the selected executable is compiled online in response to the second API request.

44. The machine readable medium of claim 39, wherein the description data includes a version and a type compatible with the second processing unit.

45. An apparatus comprising:

means for selecting, during runtime of an application running in a first processing unit, in response to a first API (Application Programming Interface) request having a requirement from the application, a second data processing unit according to the requirement;

means for loading, during the runtime, in response to a second API request from the application running in the first processing unit, one or more executables for the selected second data processing unit to perform a data processing task of the application; and

means for causing, during the runtime, in response to a third API request from the application, execution of one of the one or more executables for the second processing unit,

wherein the one or more executables include description data for the one of the one or more executables,

wherein the selecting is based on the description data,

wherein the one of the one or more executables is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the one of the one or more executables includes description data indicating the latest version for the second processing unit, and

wherein the one of the one or more executables is backward compatible with the second processing unit.

46. An apparatus comprising:

means for generating, during runtime of an application running in a first processing unit, a first API (Application Programming Interface) request having a requirement from the application to select a second data processing;

means for generating, during the runtime, a second API request from the application running in the first processing unit to load one or more executables for the selected second data processing unit to perform a data processing task of the application; and

means for generating, during the runtime, a third API request from the application to execute an executable from the one or more executables for execution in the second processing unit,

wherein the one or more executables include description data for selected executable,

wherein the second data processing unit is selected is based on the description data,

wherein the selected executable is associated with a latest version among the one or more executables for the second processing unit based on the description data,

wherein the selected executable includes description data indicating the latest version for the second processing unit, and

wherein the selected executable is backward compatible with the second processing unit.