Network-based dynamic encoding

• US 8,711,929 B2
• Filed: 10/30/2007
• Issued: 04/29/2014
• Est. Priority Date: 11/01/2006
• Status: Active Grant

• Current Assignee: Otello Corp. ASA
• Original Assignee: Skyfire Labs Inc. (Otello Corp. ASA)
• Inventors: Swenson, Erik R., Bhandari, Nitin
• Primary Examiner(s): Shah, Chirag
• Assistant Examiner(s): SMITH, JOSHUA Y
• Application Number:
  US11/929,760
• Publication Number:
  US 20080101466A1
• Time in Patent Office: 2,373 Days
• Field of Search: None
• US Class Current: 375/240.07
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2404 Monitoring of server processing errors or hardware failure error or fault detection G06F11/07; monitoring in general G06F11/30
• CPC Class Codes: H04N 21/2405 Monitoring of the internal components or processes of the server, e.g. server load allocation of resources in multiprogramming arrangements G06F9/50; performance measurement of computer activity G06F11/34
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

A network-based video encoding and decoding system encodes and decodes remotely displayed user application data on a centralized desktop computer. Remotely displayed user application data are screen captures of a browsing application run by the centralized desktop computer on user'"'"'s behalf. The encoding system optimizes its encoding performance using back channel information which includes real time network capacity information and decoder feedback. The encoding system consults a back channel information manager to dynamically adjust encoding parameters. Based on the real time network capacity information received, the encoding system adjusts its capturing sampling rate. Based on encoding errors identified by the decoding system, the encoding system selectively re-send previously encoded frames/blocks, or send intra frames on demand to allow the decoding system to correct encoding errors. In response to encoding customization requests from the decoding system, the encoding system adjusts its encoding parameters to meet such requests.

Claims

1. A system for encoding data comprising:

a capturing unit having a first control input and configured to capture data at an adjustable sampling rate in response to a first control signal received at the first control input;

an encoding unit having a data input, a data output and a second control input, for dynamically encoding data for transmission, the input of the encoding unit coupled to receive the captured data, and the output of the encoding unit providing encoded data, the encoding unit adjusting encoding parameters in response to a second control signal received at the second control input;

a command process module coupled to the second control input and coupled to receive back channel information, the command process module configured to calculate at least one encoding parameter for encoding the data input at the encoding unit, and to provide the second control signal for adjusting encoding parameters based on the back channel information comprising at least one of network capacity information, encoding error information, and encoding customization requests received from a client device;

a network capacity information manager coupled to the first control input and coupled to receive the network capacity information, the network capacity information manager configured to asynchronously calculate the sampling rate for capturing the data at the capture unit, and to provide the first control signal for adjusting the sampling rate at the capture unit based on the network capacity information; and

an encoding error recovery manager configured to identify encoding errors based on the back channel information, the encoding error recovery manager coupled to the control input of the encoding unit for instructing the encoding unit to dynamically adjust an encoding parameter responsive to identifying the encoding errors.

2. The system of claim 1, wherein asynchronously calculating the sampling rate for capturing the data at the capture unit comprises calculating the sampling rate at the network capacity information manager independently from the at least one encoding parameter.

3. The system of claim 1, wherein the encoding unit comprises:

a first encoder coupled to the data input to receive video data and to encode video data; and

a second encoder coupled to the data input to receive audio data and to encode audio data, the audio data associated with the video data.

4. The system of claim 3, wherein the encoding unit further comprises:

a synchronization module to synchronize audio and video encoding, the synchronization module coupled to the first encoder and the second encoder.

5. The system of claim 1, further comprising a back channel information manager configured to receive the back channel information and generate the second control signal to specify an encoding rate at the encoding unit, the back channel information manager coupled to the command process module and the second control input of the encoding unit.

6. The system of claim 5, wherein the back channel information manager comprises:

an encoding customization information request manager configured to update the encoding unit about encoding customization requests, the encoding customization information request manager coupled to the command process module and the control input of the encoding unit.

7. The system of claim 1, wherein the encoding unit is configured to adjust encoding parameters according to a current sampling rate.

8. The system of claim 1, wherein the encoding unit is configured to adjust encoding parameters according to the encoding error information or encoding customization information.

9. The system of claim 7, wherein the encoding unit is configured to encode data using the adjusted encoding parameters.

10. The system of claim 9, wherein the adjusted encoding parameters are video encoding parameters.

11. The system of claim 9, wherein the adjusted encoding parameters are audio encoding parameters.

12. The system of claim 1 further comprising an encryption unit coupled to the output of encoding unit for encrypting the encoded data, an output of the encryption unit providing encrypted data.

24. The system of claim 1, wherein the encoding unit further comprises:

a frame buffer for reference frame management, the frame buffer coupled to the data input to receive data.

25. The system of claim 24, wherein the frame buffer for reference frame management is configured to receive instructions from outside the encoding unit.

13. A method for encoding data comprising:

capturing data based on an adjustable sampling rate for encoding;

encoding the captured data based on dynamically adjustable encoding parameters for transmission;

receiving back channel information comprising network capacity information, encoding error information, and encoding customization requests received from a client device;

dynamically setting the adjustable encoding parameters for encoding the captured data based on the back channel information, the encoding parameters updated based on changes in the back channel information;

modifying the sampling rate of data capture for encoding asynchronously from the encoding parameters based on the received back channel information;

identify encoding errors in the encoded data based on the back channel information;

adjusting the encoding parameters responsive to the encoding error information; and

encoding data captured at the modified sampling rate using the adjusted encoding parameters for transmission.

14. The method of claim 13 wherein modifying the sampling rate of data capture for encoding asynchronously from the encoding parameters based on the received back channel information comprises calculating the sampling rate for capturing the data independently from the encoding parameters.

15. The method of claim 13, wherein the back channel information is at least one of the following:

encoding error information for notifying an encoder about coding errors; and

encoding customization information for notifying the encoder about encoding customization requests.

16. The method of claim 13 further comprising further adjusting the encoding rate according to a current sampling rate.

17. The method of claim 13, wherein encoding data using the encoding parameters comprises adjusting the encoding parameter according to an encoding rate.

18. The method of claim 13, wherein encoding data using the encoding parameters further comprises one of the following:

adjusting encoding parameters according to a coding error information; and

adjusting encoding parameters according to an encoding customization information.

19. The method of claim 13, further comprising encrypting the encoded data.

20. A system for decoding a data stream, the system comprising:

a decoding unit, of a client device, having an input, a first output and second output, for decoding a data stream, identifying encoding errors and collecting back channel information comprising network capacity information and encoding customization requests of the client device, the input of the decoding unit coupled to receive an encoded data stream, the first output of the decoding unit providing a decoded data output, and the second output of the decoding unit providing the identified encoding errors and the back channel information; and

a command process module coupled to the decoding unit for receiving the identified encoding errors and the back channel information, the command process module configured to transmit the identified encoding errors and the back channel information for;

calculating dynamically adjustable encoding parameters for encoding the data stream received at the client device based on the identified encoding errors and the back channel information; and

calculating a sampling rate for capturing data encoded for the data stream received at the client device based on the back channel information, the sampling rate adjusted asynchronously from the encoding parameters based on the network capacity information.

21. The system of claim 20, wherein the decoding unit comprises:

a first input buffer for storing a video portion of the data stream;

a first decoder for decoding video data and collecting video back channel information, the first decoder coupled to the first input buffer;

a second input buffer for storing a audio portion of the data stream; and

a second decoder for decoding the audio data and collecting audio back channel information.

22. The system of claim 20, wherein the back channel information comprises:

encoding customization information for notifying the encoder about encoding customization requests.

23. The system of claim 20, wherein the back channel information further comprises:

network capacity information for calculating a compression ratio.