Method and system for low-latency transfer protocol

• US 8,171,154 B2
• Filed: 09/29/2009
• Issued: 05/01/2012
• Est. Priority Date: 09/29/2009
• Status: Active Grant

• Current Assignee: Wickr Inc. (Amazon.com, Inc.)
• Original Assignee: Net Power & Light, Inc (Amazon.com, Inc.)
• Inventors: Vonog, Stanislav, Surin, Nikolay, Shtayura, Vadim
• Primary Examiner(s): BILGRAMI, ASGHAR H
• Application Number:
  US12/569,876
• Publication Number:
  US 20110078532A1
• Time in Patent Office: 945 Days
• Field of Search: 709/232, 709/246, 382/276
• US Class Current: 709/232
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G09G 2320/103 Detection of image changes, e.g. determination of an index representative of the image change
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 1/0009 by adapting the channel coding H04L1/1812 takes precedence
• CPC Class Codes: H04L 1/0014 by adapting the source coding
• CPC Class Codes: H04L 1/0057 Block codes H04L1/0061, H04L1/0064 take precedence
• CPC Class Codes: H04L 67/01 Protocols

Abstract

A method and system for providing computer-generated output and in particular graphical output. The system includes a network configured to carry digital information. The system includes a server in communication with the network, the server configured to execute an application and a cloud engine module. The application provides a graphical output. The output capturing and encoding engine module is further configured to intercept the graphical output from the application on the server. The output capturing and encoding engine module is further configured to convert the graphical output into at least one of: graphical commands and video codec data. The output capturing and encoding engine module is further configured to transmit the converted output over the network. The system includes a client in communication with the server over the network, the client configured to execute a graphics and video decoding and rendering engine module. The graphics and video decoding and rendering engine module is configured to, responsive to receiving the transmitted converted output, rendering the graphical output. The graphics and video decoding and rendering engine module is configured to intercept graphics and video decoding and rendering inputs at the client. The graphics and video decoding and rendering engine module is configured to transmit the intercepted user inputs to the output capturing and encoding engine module.

Claims

1. A computer implemented method, comprising:

intercepting graphical output from an executing application;

converting the intercepted output into at least one of graphical commands and video codec data, the converting including;

dividing the graphical output into a plurality of regions;

converting graphical output associated with each region;

smoothing a border area between a plurality of regions;

wherein the intercepted output is further converted into at least one of picture codec data and pass-through data, and the graphical output includes full-motion video, wherein the full-motion video is enclosed within one of the plurality of regions; and

transmitting the converted output over a network to a client, wherein transmitting comprises

parsing the converted output into a reliable queue, an unreliable queue, and an unreliable with forward error correction queue, wherein each queue has a plurality of channels, each channel associated with a priority;

transmitting the reliable queue, wherein each packet in the reliable queue is acknowledged by the client and includes a reliable packet sequence number;

transmitting the unreliable queue, wherein each packet in the unreliable queue includes an unreliable packet sequence number; and

transmitting the unreliable with forward error correction queue, wherein each packet in the unreliable with forward error correction queue includes a transform code containing redundant information to recover lost packets.

2. The computer implemented method of claim 1, wherein the graphical commands are represented by a universal intermediary graphical command language.

3. The computer implemented method of claim 1, wherein the reliable queue is used to transmit the graphical commands and the unreliable queue is used to transmit the video codec data.

4. The computer implemented method of claim 1, wherein transmitting the converted output further includes throttling transmission bandwidth based on a congestion control.

5. The computer implemented method of claim 1, further comprising:

receiving intercepted user inputs, and

providing the intercepted user inputs to the executing application for processing.

6. A computer implemented method, comprising:

receiving a converted output over a network from a server, wherein the converted output includes at least one of graphical commands. video codec data, picture codec data and pass-through data and wherein the graphical output is divided into a plurality of regions. and the graphical output associated with each region is converted into at least one of;

graphical commands, video codec data, picture codec data and pass-through data wherein the graphical output includes full-motion video, wherein the full-motion video is enclosed within one of the plurality of regions, the server executing an application providing graphical output and converting the graphical output into the converted output, wherein the receiving comprises

receiving a reliable queue, wherein each packet in the reliable queue includes a reliable packet sequence number, the receiving a reliable queue further including transmitting an acknowledgement to the server in response to receiving a reliable packet queue;

receiving an unreliable queue, wherein each packet in the unreliable queue includes an unreliable packet sequence number;

receiving an unreliable with forward error correction queue, wherein each packet in the unreliable with forward error correction queue includes a transform code containing redundant information to recover lost packets; and

compiling the reliable queue, unreliable queue, and unreliable with forward error correction queue into the received converted output;

rendering a graphical output from the received converted output, and

transmitting the intercepted user inputs to the server in response to intercepting user inputs.

7. The computer implemented method of claim 6, wherein the user inputs are transmitted to the server via the reliable queue.

8. The computer implemented method of claim 6, wherein the receiving further includes processing by a jitter buffer.

9. The computer implemented method of claim 6, wherein each queue is associated with a plurality of channels, and each channel processes incoming data with a jitter buffer.