BUFFERS FOR DISPLAY ACCELERATION

• US 20130335443A1
• Filed: 06/26/2013
• Published: 12/19/2013
• Est. Priority Date: 03/04/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Harper, John, Dyke, Kenneth C.
• Granted Patent: US 8,842,133 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence

Abstract

Embodiments enable a graphics processor to more efficiently process graphics and compositing processing commands. In certain embodiments, a client application submits client graphics commands to a graphics driver. The client in certain embodiments can notify a window server that client graphics commands have been submitted. In response, the window server can generate compositing processing commands and provide these commands to the graphics driver. Advantageously, a graphics processor can execute the client graphics commands while the window server generates compositing processing commands. As a result, processing resource can be used more efficiently.

Claims

1-37. -37. (canceled)

38. A computer-implemented method for processing graphics commands for display of a data processing system, the method comprising:

in response to a first set of graphics commands received from a first client application executed by a central-processing unit (CPU) via a graphics application programming interface (API), storing the first set of graphics commands in a first command buffer that is associated with the first client application, wherein the first set of graphics commands represents a first image layer;

in response to a second set of graphics commands received from a second client application executed by the CPU via the graphics API, storing the second set of graphics commands in a second command buffer that is associated with the second client application, wherein the second set of graphics commands represents a second image layer, wherein the first and second command buffers are configured to buffer the first and second sets of graphics commands until the first and second sets of the graphics commands are ready to be processed by a graphics processing unit (GPU) of a graphics system of the data processing system;

generating, by a compositing module executed by the CPU, one or more graphics compositing commands for compositing the first image layer and the second image layer, in response to determining that there is a need to composite the first image layer and the second image layer, wherein the compositing module is configured to monitor or to receive a notification concerning the first and second sets of graphics commands received from the first and second client applications;

in response to a ready signal indicating that the first and second sets of graphics commands and the compositing graphics commands are ready for execution, transmitting the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands as a batch of graphics commands to a graphics queue to be processed by the GPU, wherein the GPU executes the graphics commands from the graphics queue in parallel with the compositing module generating the compositing graphics commands.

39. The method of claim 38, wherein the graphics API comprises at least one of OpenGL™

, OpenGL™

ES, and DirectX™

APIs.

40. The method of claim 38, wherein the batch of graphics commands corresponds to one frame of image data.

41. The method of claim 38, wherein the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands are transmitted as a batch of graphics commands to the graphics queue in response to a notification indicating that the GPU is ready to process additional graphics commands.

42. The method of claim 38, wherein the first and second command buffers are allocated from a pool of command buffers.

43. The method of claim 38, wherein the first and second client applications directly write via the graphics API the first and second sets of graphics commands to the first and second command buffers, respectively.

44. The method of claim 43, wherein the first and second client applications write the respective graphics commands to the first and second command buffers asynchronously.

45. The method of claim 43, wherein the first client application writes new graphics commands into a first portion of the first command buffer, while graphics commands previously written into a second portion of the first command buffer are processed by the GPU in parallel.

46. The method of claim 38, wherein the first and second client applications call one or more graphics routines of the graphics API to send the first and second sets of graphics commands to the compositing module, which in turn stores the received the first and second sets of graphics commands in the first and second command buffers, respectively.

47. A non-transitory machine-readable medium having instructions stored therein, which when executed by a machine, cause the machine to perform a method for processing graphics commands for display of a data processing system, the method comprising:

in response to a first set of graphics commands received from a first client application executed by a central-processing unit (CPU) via a graphics application programming interface (API), storing the first set of graphics commands in a first command buffer that is associated with the first client application, wherein the first set of graphics commands represents a first image layer;

in response to a second set of graphics commands received from a second client application executed by the CPU via the graphics API, storing the second set of graphics commands in a second command buffer that is associated with the second client application, wherein the second set of graphics commands represents a second image layer, wherein the first and second command buffers are configured to buffer the first and second sets of graphics commands until the first and second sets of the graphics commands are ready to be processed by a graphics processing unit (GPU) of a graphics system of the data processing system;

generating, by a compositing module executed by the CPU, one or more graphics compositing commands for compositing the first image layer and the second image layer, in response to determining that there is a need to composite the first image layer and the second image layer, wherein the compositing module is configured to monitor or to receive a notification concerning the first and second sets of graphics commands received from the first and second client applications;

in response to a ready signal indicating that the first and second sets of graphics commands and the compositing graphics commands are ready for execution, transmitting the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands as a batch of graphics commands to a graphics queue to be processed by the GPU, wherein the GPU executes the graphics commands from the graphics queue in parallel with the compositing module generating the compositing graphics commands.

48. The non-transitory machine-readable medium of claim 47, wherein the graphics API comprises at least one of OpenGL™

, OpenGL™

ES, and DirectX™

APIs.

49. The non-transitory machine-readable medium of claim 47, wherein the batch of graphics commands corresponds to one frame of image data.

50. The non-transitory machine-readable medium of claim 47, wherein the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands are transmitted as a batch of graphics commands to the graphics queue in response to a notification indicating that the GPU is ready to process additional graphics commands.

51. The non-transitory machine-readable medium of claim 47, wherein the first and second command buffers are allocated from a pool of command buffers.

52. The non-transitory machine-readable medium of claim 47, wherein the first and second client applications directly write via the graphics API the first and second sets of graphics commands to the first and second command buffers, respectively.

53. The non-transitory machine-readable medium of claim 52, wherein the first and second client applications write the respective graphics commands to the first and second command buffers asynchronously.

54. The non-transitory machine-readable medium of claim 52, wherein the first client application writes new graphics commands into a first portion of the first command buffer, while graphics commands previously written into a second portion of the first command buffer are processed by the GPU in parallel.

55. The non-transitory machine-readable medium of claim 47, wherein the first and second client applications call one or more graphics routines of the graphics API to send the first and second sets of graphics commands to the compositing module, which in turn stores the received the first and second sets of graphics commands in the first and second command buffers, respectively.

56. A data processing system, comprising:

a central processing unit (CPU);

a graphics system having a graphics processing unit (GPU); and

a memory coupled to the CPU to store instructions, which when executed from the memory, cause the CPU to

in response to a first set of graphics commands received from a first client application executed by the CPU via a graphics application programming interface (API), store the first set of graphics commands in a first command buffer that is associated with the first client application, wherein the first set of graphics commands represents a first image layer,

in response to a second set of graphics commands received from a second client application executed by the CPU via the graphics API, store the second set of graphics commands in a second command buffer that is associated with the second client application, wherein the second set of graphics commands represents a second image layer, wherein the first and second command buffers are configured to buffer the first and second sets of graphics commands until the first and second sets of the graphics commands are ready to be processed by the GPU of the graphics system of the data processing system,

generate, by a compositing module executed in the memory by the CPU, one or more graphics compositing commands for compositing the first image layer and the second image layer, in response to determining that there is a need to composite the first image layer and the second image layer, wherein the compositing module is configured to monitor or to receive a notification concerning the first and second sets of graphics commands received from the first and second client applications,

in response to a ready signal indicating that the first and second sets of graphics commands and the compositing graphics commands are ready for execution, transmit the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands as a batch of graphics commands to a graphics queue to be processed by the GPU, wherein the GPU executes the graphics commands from the graphics queue in parallel with the compositing module generating the compositing graphics commands.

57. The system of claim 56, wherein the graphics API comprises at least one of OpenGL™

, OpenGL™

ES, and DirectX™

APIs.

58. The system of claim 56, wherein the batch of graphics commands corresponds to one frame of image data.

59. The system of claim 56, wherein the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands are transmitted as a batch of graphics commands to the graphics queue in response to a notification indicating that the GPU is ready to process additional graphics commands.

60. The system of claim 56, wherein the first and second command buffers are allocated from a pool of command buffers.

61. The system of claim 56, wherein the first and second client applications directly write via the graphics API the first and second sets of graphics commands to the first and second command buffers, respectively.

62. The system of claim 61, wherein the first and second client applications write the respective graphics commands to the first and second command buffers asynchronously.

63. The system of claim 61, wherein the first client application writes new graphics commands into a first portion of the first command buffer, while graphics commands previously written into a second portion of the first command buffer are processed by the GPU in parallel.

64. The system of claim 56, wherein the first and second client applications call one or more graphics routines of the graphics API to send the first and second sets of graphics commands to the compositing module, which in turn stores the received the first and second sets of graphics commands in the first and second command buffers, respectively.

65. A computer-implemented method for processing graphics commands for display of a data processing system, the method comprising:

in response to a first set of graphics commands received from a first client application executed by a central-processing unit (CPU) via a graphics application programming interface (API), storing the first set of graphics commands in a first command buffer that is associated with the first client application, wherein the first set of graphics commands represents a first image layer;

in response to a second set of graphics commands received from a second client application executed by the CPU via the graphics API, storing the second set of graphics commands in a second command buffer that is associated with the second client application, wherein the second set of graphics commands represents a second image layer, wherein the first and second command buffers are configured to buffer the first and second sets of graphics commands until the first and second sets of the graphics commands are ready to be processed by a graphics processing unit (GPU) of a graphics system of the data processing system;

generating, by a compositing module executed by the CPU, one or more graphics compositing commands for compositing the first image layer and the second image layer, in response to determining that there is a need to composite the first image layer and the second image layer, wherein the compositing module is configured to monitor or to receive a notification concerning the first and second sets of graphics commands received from the first and second client applications;

executing, by the GPU, the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands as a batch of graphics commands to display a composited image representing a combination of the first image layer, the second image layer, and an overlay layer having at least a portion of the first image layer and the second image layer, wherein the GPU executes the graphics commands in parallel with the compositing module generating the compositing graphics commands.

66. The method of claim 65, wherein the first and second command buffers are allocated from a pool of command buffers.

67. The method of claim 65, wherein the first and second client applications directly write via the graphics API the first and second sets of graphics commands to the first and second command buffers, respectively.

68. The method of claim 67, wherein the first and second client applications write the respective graphics commands to the first and second command buffers asynchronously.

69. The method of claim 67, wherein the first client application writes new graphics commands into a first portion of the first command buffer, while graphics commands previously written into a second portion of the first command buffer are processed by the GPU in parallel.

70. The method of claim 65, wherein the first and second client applications call one or more graphics routines of the graphics API to send the first and second sets of graphics commands to the compositing module, which in turn stores the received the first and second sets of graphics commands in the first and second command buffers, respectively.

71. A non-transitory computer-readable medium having instructions stored therein, which when executed by a machine, cause the machine to perform a method for processing graphics commands for display of a data processing system, the method comprising:

in response to a first set of graphics commands received from a first client application executed by a central-processing unit (CPU) via a graphics application programming interface (API), storing the first set of graphics commands in a first command buffer that is associated with the first client application, wherein the first set of graphics commands represents a first image layer;

in response to a second set of graphics commands received from a second client application executed by the CPU via the graphics API, storing the second set of graphics commands in a second command buffer that is associated with the second client application, wherein the second set of graphics commands represents a second image layer, wherein the first and second command buffers are configured to buffer the first and second sets of graphics commands until the first and second sets of the graphics commands are ready to be processed by a graphics processing unit (GPU) of a graphics system of the data processing system;

generating, by a compositing module executed by the CPU, one or more graphics compositing commands for compositing the first image layer and the second image layer, in response to determining that there is a need to composite the first image layer and the second image layer, wherein the compositing module is configured to monitor or to receive a notification concerning the first and second sets of graphics commands received from the first and second client applications;

executing, by the GPU, the first set of graphics commands, the second set of graphics commands, and the compositing graphics commands as a batch of graphics commands to display a composited image representing a combination of the first image layer, the second image layer, and an overlay layer having at least a portion of the first image layer and the second image layer, wherein the GPU executes the graphics commands in parallel with the compositing module generating the compositing graphics commands.

72. The non-transitory computer-readable medium of claim 71, wherein the first and second command buffers are allocated from a pool of command buffers.

73. The non-transitory computer-readable medium of claim 71, wherein the first and second client applications directly write via the graphics API the first and second sets of graphics commands to the first and second command buffers, respectively.

74. The non-transitory computer-readable medium of claim 73, wherein the first and second client applications write the respective graphics commands to the first and second command buffers asynchronously.