Methods and apparatuses for providing a hardware accelerated web engine

• US 8,234,392 B2
• Filed: 11/17/2006
• Issued: 07/31/2012
• Est. Priority Date: 11/17/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Graffagnino, Peter, Hyatt, Dave, Blanchard, Richard, Calhoun, Kevin, Drieu, Gilles, Stachowiak, Maciej, Melton, Don, Adler, Darin
• Primary Examiner(s): Nawaz, Asad M
• Assistant Examiner(s): MURPHY, CHARLES C
• Application Number:
  US11/601,497
• Publication Number:
  US 20080120626A1
• Time in Patent Office: 2,083 Days
• Field of Search: 345/473, 345/582, 345/619, 709/201, 709/203, 709/206, 709/213, 709/223, 709/231, 709/230, 709/232, 709/227, 709/217, 709/236, 709/225, 709/224, 709/207, 715/202, 715/234, 715/700, 715/716, 715/730, 715/764, 715/772, 715/782, 715/838, 715/757, 715/753, 715/723, 715/706, 715/277, 715/246, 715/236, 715/853, 715/760, 715/747, 715/744, 715/704, 715/273, 715/255, 716/20, 716/21, 382/176, 382/164, 382/159, 382/157, 382/154, 382/130
• US Class Current: 709/230
• CPC Class Codes: G06F 40/123 Storage facilities
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Methods and apparatuses for providing hardware acceleration of a web browser are disclosed. In one embodiment, a method of operating a web browser on a computer system includes analyzing a data stream having a plurality of fragments. The method further includes determining what fragments of the data stream should be rendered for storage into separate backing stores. The method further includes rendering the fragments into raster images intended for hardware acceleration. The method further includes storing the raster images in the backing stores located in a graphics processing unit.

Claims

1. A method for hardware accelerating a web browser on a computer system, the method comprising:

analyzing, by the web browser, a data stream having a plurality of fragments, the data stream being processed by the web browser;

determining, by the web browser, what fragments of the data stream should be rendered for storage into separate backing stores for a graphics processing unit;

rendering, by the web browser, the fragments into raster images;

causing the storage of the raster images in the backing stores for the graphics processing unit, wherein the web browser provides information identifying the raster images to the graphics processing unit; and

compositing the raster images, which each represent a decomposed portion of a complete image, into a complete image displayed as a web page in order to hardware accelerate the operation of the web browser.

2. The method of claim 1, wherein the data stream comprises a web page having a plurality of elements.

3. The method of claim 1, wherein determining what fragments of the data stream should be rendered for storage into separate backing stores occurs by assigning a backing store for each fragment of the data stream.

4. The method of claim 3, further comprising:

determining what elements of the web page should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each element having a specific attribute associated with the element, the specific attribute comprises opacity, explicit position, relative position, or explicit Z ordering.

5. The method of claim 4, wherein determining what elements of the web page should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each element that provides scripting of the web page.

6. The method of claim 4, wherein determining what elements of the web page should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each element associated with an explicit indication that the element should be an independent layer.

7. The method of claim 4, wherein determining what elements of the web page should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each element having a combination of specific attributes which individually each specific attribute is not sufficient to receive an independent layer.

8. The method of claim 1, wherein determining what fragments of the data stream should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each fragment associated with a plug-in model.

9. The method of claim 1, wherein determining what fragments of the data stream should be rendered for storage into separate backing stores occurs by allocating an independent layer associated with a backing store for each fragment requiring animation.

10. A machine-readable non-transitory medium having executable instructions to cause a machine to perform a method comprising:

analyzing, by a web browser of the machine, a data stream having a plurality of fragments, the data stream being processed by the web browser;

determining, by the web browser, what fragments of the data stream should be rendered for storage into separate backing stores for a graphics processing unit;

rendering, by the web browser, the fragments into raster images;

causing the storage of the raster images in the backing stores for the graphics processing unit, wherein the web browser provides information identifying the raster images to the graphics processing unit; and

compositing the raster images, which each represent a decomposed portion of a complete image, into a complete image displayed as a web page in order to hardware accelerate the operation of the web browser.

11. A method for hardware accelerating a web browser on a computer system, the method comprising:

constructing a first data structure by grouping elements of a web page into a first plurality of nodes;

determining, by the web browser, what elements should be rendered for storage into separate backing stores;

constructing a second data structure separate from the first data structure, the second data structure comprises a second plurality of nodes each representing an independent layer associated with a raster image intended for hardware acceleration;

rendering, by the web browser, all nodes of the first data structure to create raster images, which each represent a decomposed portion of a complete image, stored in the backing stores for a graphics processing unit (GPU); and

compositing the raster images stored in the backing stores to construct a first final image displayed as a web page in order to hardware accelerate the operation of the web browser.

12. The method of claim 11, further comprising:

receiving an input to animate a portion of the web page associated with a raster image stored in a backing store;

determining if animation of the portion of the web page requires re-rendering of any node of the first data structure; and

instructing the GPU to re-composite the raster images into a second final image displayed as a web page if no re-rendering of any node of the first data structure is required.

13. The method of claim 12, wherein animations of the portion of the web page that do not require re-rendering of any nodes of the first data structure comprise changing at least one of:

position, Z order, opacity, and scale.

14. The method of claim 12, further comprising:

re-rendering a node of the first data structure associated with the portion of the web page if re-rendering is required; and

re-compositing the raster images to construct a third final image displayed as a web page.

15. The method of claim 14, wherein animations of the portion of the web page that do require re-rendering at least one node of the first data structure comprise changing shape or updating a video frame.

16. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for each element having a specific attribute associated with the element, the specific attribute comprises opacity, explicit position, relative position, or explicit Z ordering.

17. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for each element that provides scripting of the web page.

18. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for each element associated with an explicit indication that the element should be an independent layer.

19. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for each element having a combination of specific attributes which individually each specific attribute is not sufficient to receive an independent layer.

20. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for a plug-in model associated with the element.

21. The method of claim 11, wherein determining what elements should be rendered for storage into separate backing stores occurs by allocating an independent layer for each element requiring animation.

22. The method of claim 11, wherein each independent layer is associated with a raster image stored in a single retained backing store.

23. A machine-readable non-transitory medium having executable instructions to cause a machine to perform a method comprising:

constructing a first data structure by grouping elements of a web page into a first plurality of nodes;

determining, by a web browser of the machine, what elements should be rendered for storage into separate backing stores;

constructing a second data structure separate from the first data structure, the second data structure comprises a second plurality of nodes each representing an independent layer associated with a raster image intended for hardware acceleration;

rendering, by the web browser, all nodes of the first data structure to create raster images, which each represent a decomposed portion of a complete image, stored in the backing stores for a graphics processing unit (GPU); and

compositing the raster images stored in the backing stores to construct a first final image displayed as a web page in order to hardware accelerate the operation of the web browser.

24. The machine-readable non-transitory medium of claim 23, further comprising:

receiving an input to animate a portion of the web page associated with a raster image stored in a backing store;

determining if animation of the portion of the web page requires re-rendering of any node of the first data structure; and

instructing the GPU to composite the raster images into a second final image displayed as a web page if no re-rendering of any node of the first data structure is required.

25. The machine-readable non-transitory medium of claim 24, further comprising:

re-rendering a node associated with the portion of the web page if re-rendering is required; and

re-compositing the raster images to construct a third final image displayed as a web page.

26. A data processing system, comprising:

a first microprocessor coupled to a bus;

main memory coupled to the bus with the main memory configured to store a data structure representing a render tree having nodes for grouping elements of a web page; and

a graphics processing unit coupled to the bus, the graphics processing unit having a second microprocessor and a plurality of backing stores with each backing store configured to store raster images associated with nodes of the render tree, wherein the first microprocessor executes a web browser application to determine what elements should be rendered for storage into the plurality of backing stores and to render the fragments into raster images, wherein the web browser provides information identifying the raster images to the graphics processing unit, and wherein the second microprocessor to execute a graphics application to composite the raster images, which each represent a decomposed portion of a complete image, into a complete image displayed as a web page in order to hardware accelerate the operation of the web browser.

27. The data processing system of claim 26, wherein the nodes comprise at least one of the following:

an image element, a graphic element, a text element, a vector element, and a media element.

28. The data processing system of claim 27, wherein each of the nodes are defined by a property selected from the group consisting of a position, a coordinate system, a transform, and a relationship.

29. A method of operating a web browser software application on a computer system, the method comprising:

analyzing, by the web browser software application, a data stream having a plurality of fragments, the data stream being encoded in a markup language;

determining, by the web browser software application, what fragments of the data stream should be rendered for storage into separate backing stores;

rendering, by the web browser software application, the fragments into raster images, which each represent a decomposed portion of a complete image;

causing the storage of the raster images in the backing stores for a graphics processing unit, wherein the web browser software application provides information identifying the raster images to the graphics processing unit; and

compositing the raster images into a complete image in order to hardware accelerate the operation of the web browser software application.

30. The method of claim 29, wherein the information identifying the raster images comprises location and attributes of the raster images.

31. The method of claim 30, wherein the attributes of the raster images comprise opacity and position of each raster image within a complete image.

32. A machine-readable non-transitory medium having executable instructions to cause a machine to perform a method comprising:

analyzing, by a web browser software application of the machine, a data stream having a plurality of fragments, the data stream being encoded in a markup language;

determining, by the web browser software application, what fragments of the data stream should be rendered for storage into separate backing stores;

rendering, by the web browser software application, the fragments into raster images, which each represent a decomposed portion of a complete image;

causing the storage of the raster images in the backing stores for a graphics processing unit, wherein the web browser software application provides information identifying the raster images to the graphics processing unit; and

compositing the raster images into a complete image in order to hardware accelerate the operation of the web browser software application.