System for optimizing graphics operations

• US 9,691,118 B2
• Filed: 02/20/2014
• Issued: 06/27/2017
• Est. Priority Date: 04/16/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Zimmer, Mark, Harper, John, Brunner, Ralph, Graffagnino, Peter
• Primary Examiner(s): Ma, Tize
• Application Number:
  US14/185,656
• Publication Number:
  US 20140184634A1
• Time in Patent Office: 1,223 Days
• Field of Search: None
• CPC Class Codes: G06F 15/76 Architectures of general purpose stored program computers with program plugboard G06F15/08; multicomputers G06F15/16
• CPC Class Codes: G06F 15/80 comprising an array of processing units with common control, e.g. single instruction multiple data processors G06F15/82 takes precedence ; for correlation function computation G06F17/15
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607

Abstract

Disclosed is a system for producing images including techniques for reducing the memory and processing power required for such operations. The system provides techniques for programmatically representing a graphics problem. The system further provides techniques for reducing and optimizing graphics problems for rendering with consideration of the system resources, such as the availability of a compatible GPU.

Claims

1. A method of applying two effects to a frame of a display stream, comprising:

using a first graphics processing resource to apply a first effect to a first frame of a display stream, wherein applying the first effect comprises providing a new value for at least one pixel of the first frame based upon sampling of multiple other pixels of the first frame;

using the first graphics processing resource to apply a second effect to a second frame of the display stream, the application of the second effect occurring immediately after the application of the first effect, wherein applying the second effect comprises at least one of altering the color of one or more pixels, combining aspects of multiple frames, and providing a new value for a least one pixel based upon sampling of multiple other pixels; and

using a second graphics processing resource to apply a third effect to the first frame, wherein the application of the third effect occurs after the application of the first effect and involves combining the first frame with a third frame,

wherein applying the second effect to the second frame by the first graphics processing resource occurs approximately during the time the second graphics processing resource is applying the third effect to the first frame.

2. The method of claim 1, wherein one of the first graphics processing resource and the second graphics processing resource is a CPU, and the other is a GPU.

3. The method of claim 1, wherein at least one of the first graphics processing resource and the second graphics processing recourse is a CPU emulating a GPU.

4. The method of claim 1, wherein the first effect is applied by running a fragment program on a first GPU.

5. The method of claim 1, wherein at least one of the first effect and the second effect is selected through an application programming interface.

6. The method of claim 5, wherein the application programming interface receives a selection of at least one of the first effect and the second effect from a software program other than a user application program.

7. A computer system comprising a plurality of processors, wherein the plurality of processors are programmed to:

use a first processor to apply a first effect to a first frame of a display stream, wherein applying the first effect comprises providing a new value for at least one pixel of the first frame based upon sampling of multiple other pixels of the first frame;

use the first processor to apply a second effect to a second frame of the display stream, the application of the second effect occurring immediately after the application of the first effect, wherein applying the second effect comprises at least one of altering the color of one or more pixels, combining aspects of multiple frames, and providing a new value for at least one pixel of the second frame based upon sampling of multiple other pixels of the second frame; and

use a second processor to apply a third effect to the first frame, wherein the application of the third effect occurs after the application of the first effect and involves combining the first frame with a third frame,

wherein applying the second effect to the second frame by the first processor occurs approximately during the time the second processor is applying the third effect to the first frame.

8. The computer system of claim 7, wherein one of the first processor and the second processor is a CPU, and the other is a GPU.

9. The computer system of claim 7, wherein at least one of the first processor and the second processor is a CPU emulating a GPU.

10. The computer system of claim 7, wherein the first effect is applied by running a fragment program on a first GPU.

11. The computer system of claim 7, wherein at least one of the first effect and the second effect is selected through an application programming interface.

12. The computer system of claim 11, wherein the application programming interface receives a selection of at least one of the first effect and the second effect from a software program other than a user application program.

13. A non-transitory program storage device, readable by one or more processors and comprising instructions stored thereon to cause one or more processors to:

use a first processor to apply a first effect to a first frame of a display stream, wherein applying the first effect comprises providing a new value for at least one pixel of the first frame based upon sampling of multiple other pixels of the first frame;

use the first processor to apply a second effect to a second frame of the display stream, wherein applying the second effect comprises at least one of altering the color of one or more pixels, combining aspects of multiple frames, and providing a new value for a least one pixel of the second frame based upon sampling of multiple other pixels of the second frame; and

use a second processor to apply a third effect to the first frame, wherein the application of the third effect occurs after the application of the first effect and involves combining the first frame with a third frame,

wherein the instructions further cause the one or more processors to apply the second effect to the second frame by the first processor approximately during the time the second processor is applying the third effect to the first frame.

14. The non-transitory program storage device of claim 13, wherein one of the first processor and the second processor is a CPU, and the other is a GPU.

15. The non-transitory program storage device of claim 13, wherein at least one of the first processor and the second processor is a CPU emulating a GPU.

16. The non-transitory program storage device of claim 13, wherein the first effect is applied by running a fragment program on a first GPU.

17. The non-transitory program storage device of claim 13, wherein at least one of the first effect and the second effect is selected through an application programming interface.

18. The non-transitory program storage device of claim 17, wherein the application programming interface receives a selection of at least one of the first effect and the second effect from a software program other than a user application program.

19. The non-transitory program storage device of claim 17, wherein the application programming interface provides multiple filters for altering the color or one or more pixels of a frame.

20. The non-transitory program storage devices of claim 17, wherein the application program interface provides filters for each of:

altering the color of one or more pixels;

combining aspects of multiple frames; and

providing a new value for a least one pixel of a frame based upon sampling of multiple other pixels of the same frame.

21. The non-transitory program storage devices of claim 13, wherein the first effect is selected through an application programming interface.