Vector-based encoding for content rendering

• US 10,601,894 B1
• Filed: 09/28/2015
• Issued: 03/24/2020
• Est. Priority Date: 09/28/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Derby, Justin Jon, Killian, David Andrew, Sokolowski, Dmitry
• Primary Examiner(s): Patel, Manglesh M
• Assistant Examiner(s): Hasty, Nicholas
• Application Number:
  US14/867,576
• Time in Patent Office: 1,639 Days
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/146 Coding or compression of tree-structured data
• CPC Class Codes: G06F 40/149 Adaptation of the text data for streaming purposes, e.g. Efficient XML Interchange [EXI] format
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability

Abstract

Minimizing bandwidth consumption in rendering dynamic content, such as video streaming. Traditional streaming video applications utilize lossy protocols that often result in degraded user experiences. A dynamic content encoder/decoder allows a server-based browser application to generate vector-based rendering instructions for processing by a client-based browser application. Additionally, as the dynamic content changes the content to be displayed, a server-based browser application generates updated vector-based instructions and encoded spatial and temporal differences to minimize the data transmitted to the client-based browser.

Claims

1. A computer-implemented method for remote content rendering comprising:

in response to a content browsing request from a client computing device, obtaining, by a network based browser software application instantiated at a network computing component, a network resource from a content provider, wherein the network resource is indicated by the content browsing request;

determining, by the network based browser software application, based at least in part on the content of the network resource, a first set of display objects from a library of vector-based display objects, wherein the first set of display objects, when rendered, corresponds to at least a first portion of the network resource;

generating, by the network computing component, a first set of vector-based rendering instructions that, when executed on the client computing device, cause generation of at least a portion of the first set of display objects based at least in part on reference to a local copy of the library of vector-based display objects;

encoding, by the network computing component, the first set of vector-based rendering instructions into first data;

transmitting the first data from the network computing component to the client computing device, wherein the client computing device decodes the first data to obtain the first set of vector-based rendering instructions;

determining, by the network based browser software application, based at least in part on the content of the network resource, a second set of display objects from the library of vector-based display objects, wherein the second set of display objects, when rendered, corresponds to at least a second portion of the network resource;

generating, by the network computing component, a second set of vector-based rendering instructions, which when executed on the client computing device, cause presentation of at least a portion of the second set of display objects based at least in part on reference to the local copy of the library of display objects;

determining, by the network computing component, at least one spatial or temporal difference between the first set and second set of vector-based rendering instructions;

determining that the at least one spatial or temporal difference between the first set and second set of vector-based rendering instructions exceeds a threshold;

in response to the determination that the at least one spatial or temporal difference between the first set and second set of vector based rendering instructions exceeds the threshold, encoding, by the network computing component, the at least one spatial or temporal difference between the first set and the second set of vector-based rendering instructions into second data; and

transmitting the second data from the network computing component to the client computing device, wherein the client computing device decodes the second data to obtain the second set of vector-based rendering instructions.

2. The computer-implemented method of claim 1, wherein the network based browser software application is executed on a first virtual machine instance hosted on the network computing component.

3. The computer-implemented method of claim 2, wherein the at least one spatial or temporal difference between the first set and second set of vector-based rendering instructions is determined by a second virtual machine instance hosted on the network computing component.

4. The computer-implemented method of claim 1 further comprising:

generating, by the network computing component, a third set of vector-based rendering instructions, which when executed on the client computing device, cause presentation of at least a portion of a third set of display objects that, when rendered, correspond to at least a third portion of the network resource; and

determining, by the network computing component, that the third set of vector-based rendering instructions is substantially similar to the first set of vector-based rendering instructions, wherein the third set of vector-based rendering instructions is not encoded by the network computing component.

5. The computer-implemented method of claim 1, wherein the at least one spatial or temporal difference between the first set and second set of vector-based rendering instructions includes at least one of a difference in coordinates, size, shape, color, gradient, or timing with respect to one or more vectors.

6. A system for remote content rendering comprising:

a first processor and a first memory for executing first computer executable instructions, the first computer executable instructions implementing a network computing component that is operable to at least;

obtain a network resource, wherein the network resource is requested by a client computing device in communication with the network computing component;

generate a first set of rendering instructions based at least in part on the content of the network resource and a library of vector-based display objects, wherein the first set of rendering instructions, when executed on the client computing device, cause presentation of a first set of display objects that correspond to at least a first portion of the network resource;

transmit a first representation of the first set of rendering instructions to the client computing device;

generate a second set of rendering instructions based at least in part on the content of the network resource and the library of vector-based display objects, wherein the second set of rendering instructions, when executed on the client computing device, cause presentation of a second set of display objects that correspond to at least a second portion of the network resource;

determine at least one difference between the first set and second set of rendering instructions;

determine that the at least one difference exceeds a threshold; and

in response to the determination that the at least one difference exceeds the threshold, transmit a second representation of the second set of rendering instructions to the client computing device, wherein the second representation is an encoding of the at least one difference between the first set and the second set of rendering instructions; and

the client computing device comprising a second processor and a second memory for executing second computer executable instructions, the client computing device operable to at least;

receive the first representation of the first set of rendering instructions;

derive the first set of rendering instructions from the first representation;

execute the first set of rendering instructions based at least in part on reference to a local copy of the library of vector-based display objects;

receive the second representation of the second set of rendering instructions;

derive at least one portion of the second set of rendering instruction based, at least in part, on the second representation; and

execute the at least one portion of the second set of rendering instructions based at least in part on reference to the local copy of the library of vector-based display objects.

7. The system of claim 6, wherein the client computing device is further operable to derive the at least one difference from the second representation and wherein the client computing device derives the at least one portion of the second set of rendering instructions based on the first set of rendering instructions and the at least one difference.

8. The system of claim 6, wherein the client computing device is further operable to generate a third set of rendering instructions based on the first set of rendering instructions and the second representation of the second set of rendering instructions.

9. The system of claim 8 wherein the client computing device executes the third set of rendering instructions before executing the at least one portion of the second set of rendering instructions and after executing the first set of rendering instructions.

10. The system of claim 6, wherein the first and second sets of rendering instructions are vector-based.

11. The system of claim 6, wherein the first and second sets of rendering instructions are organized in accordance with respective hierarchical structures.

12. The system of claim 6, wherein the first and second representations are generated in accordance with the library of vector-based display objects.

13. The system of claim 6, wherein the network resource includes at least one of a Web page, Web application, picture, animation, video, audio, or script code.

14. A non-transitory computer readable storage medium storing computer executable instructions that when executed by one or more processors perform operations comprising:

obtaining a network resource, wherein the network resource is requested by a client computing device;

generating a first set of rendering instructions based at least in part on the content of the network resource and a library of vector-based display objects, wherein the first set of rendering instructions, when executed on the client computing device, cause presentation of a first set of display objects that correspond to at least a first portion of the network resource;

transmitting a first representation of the first set of rendering instructions to the client computing device;

generating a second set of rendering instructions based at least in part on the content of the network resource and the library of vector-based display objects, wherein the second set of rendering instructions, when executed on the client computing device, cause presentation of a second set of display objects that correspond to at least a second portion of the network resource;

determining at least one difference between the first set and second set of rendering instructions; and

in response to a determination that the at least one difference exceeds a threshold, transmitting a second representation of the second set of rendering instructions to the client computing device, wherein the second representation is an encoding of the at least one difference between the first set and the second set of rendering instructions.

15. The non-transitory computer readable storage medium of claim 14, wherein the operations further comprise processing the network resource to derive the first set of display objects.

16. The non-transitory computer readable storage medium of claim 15, wherein each rendering instruction of the first set of rendering instructions is associated with one of the first set of display objects.

17. The non-transitory computer readable storage medium of claim 14, wherein generating a second set of rendering instructions comprises:

determining that the at least one difference is below a threshold; and

encoding the at least one difference with reference to the first set of rendering instructions.

18. The non-transitory computer readable storage medium of claim 14, wherein generating a second set of rendering instructions comprises:

determining that the at least one difference is above a threshold; and

encoding the at least one difference without reference to the first set of rendering instructions.