Communication system, method and program

• US 9,942,553 B2
• Filed: 07/23/2013
• Issued: 04/10/2018
• Est. Priority Date: 07/24/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ozawa, Kazunori
• Primary Examiner(s): Atala, Jamie
• Assistant Examiner(s): Billah, Masum
• Application Number:
  US14/416,498
• Publication Number:
  US 20150181215A1
• Time in Patent Office: 1,722 Days
• Field of Search: 345555, 375240
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 47/38 by adapting coding or compression rate
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria

Abstract

A server transmits a probe packet to a terminal, estimates a bandwidth of the network based on information returned from the terminal, calculates a rate used for performing compression coding of screen information and/or an accompanying audio signal, generated by an application on a virtual terminal on the server, so as not to exceed the estimated bandwidth, encodes the screen information and/or the accompanying audio signal at the calculated rate, stores the encoding result in a packet of a predetermined protocol to transmit the packet.

Claims

1. A server apparatus comprising:

a virtual terminal;

an application that is invoked on the virtual terminal responsive to an operation on a first terminal connected to the server apparatus via a network, information generated by the application on the virtual terminal being transmitted to the first terminal via the network;

a packet transceiver configured to transmit a probe packet at predetermined timing to the first terminal via the network, and receive a reply packet returned via the network from the first terminal receiving the probe packet;

an encoder configured to perform compression coding of at least one of screen information and an audio signal accompanying the screen information;

a bandwidth estimation and rate control processor configured to estimate a bandwidth of the network based on information included in the reply packet, calculate a rate used in the encoder when performing compression coding of at least one of the screen information and the audio signal accompanying the screen information, and give an instruction of the calculated rate to the encoder, wherein the screen information and the audio signal are generated by the application on the virtual terminal, the calculated rate is a value that does not exceed the estimated bandwidth, and the encoder is configured to set a rate for compression coding to the calculated rate instructed by the bandwidth estimation and rate control processor; and

a packet transmitter configured to store an output from the encoder in a first packet of a predetermined protocol and transmit the first packet to the first terminal via the network, wherein the bandwidth estimation and rate control processor is configured to

receive information extracted from the reply packet, wherein the information includes a data size of the probe packet, a transmission time of the probe packet at the server apparatus, and a reception time of the probe packet at the first terminal;

perform sorting in order of transmission time and calculate a bandwidth estimation value W by the following equation;

2. The server apparatus according to claim 1, wherein using the smoothed bandwidth estimation value B(n), the bandwidth estimation and rate control processor calculates a bit rate C(n) used in the encoder in accordance with the following equation, at each predetermined point in time:

C(n)=(1−

γ

)×

C(n−

1)+γ

×

B(n)

where C(n) is a bit rate at an n-th point in time, γ

is a constant in a range of 0<

γ

<

1, and B(b) is a bandwidth estimation value after smoothing, and where the bit rate C(n) is less than or equal to B(n).

3. A communication system comprising:

a server apparatus as set forth in claim 1; and

a terminal that connects to the server apparatus via a network.

6. The server apparatus according to claim 1, wherein the packet transceiver continuously transmits a plurality of the probe packets to the first terminal at timing of reception of the instruction from the bandwidth estimation and rate control processor,

the packet transceiver includes information of at least data size of the probe packet and transmission time by the server apparatus of the probe packet in the probe packet,

the packet transceiver receives a plurality of the reply packets from the first terminal in response to a plurality of the probe packets transmitted continuously;

and

the packet transceiver extracts information of data size of the probe packet, transmission time of the probe packet at the server apparatus, and reception time of the probe packet at the terminal, and outputs the extracted information to the bandwidth estimation and rate control processor.

7. The server apparatus according to claim 1, further comprising:

a screen capture processor configured to capture screen information generated by the application on the virtual terminal at a predetermined screen resolution and a predetermined frame rate;

wherein the encoder comprises;

a first image encoder configured to receive a screen signal from the screen capture processor at a predetermined screen resolution and a predetermined frame rate, receive from the bandwidth estimation and rate control processor a bit rate of an image encoder at each of predetermined points in time, calculate a screen resolution and a frame rate of the image encoder from the bit rate of the image encoder, perform compression coding of the screen signal, and output a compression coded stream, and

wherein the packet transmitter comprises

a first packet transmitter configured to receive the compression coded stream from the image encoder, packetize the compression coded stream into the first packet, and transmit the first packet to the first terminal.

8. The server apparatus according to claim 7, wherein the encoder comprises

an audio encoder configured to perform compression coding according to a predetermined compression scheme of an audio signal output from the screen capture processor, output a compression coded stream, and when an audio signal accompanies image information generated by the application on the virtual terminal, perform compression coding according to a predetermined compression scheme of the audio signal output from the screen capture processor and output a compression coded stream, wherein

the packet transmitter comprises

a second packet transmitter configured to receive the compression coded stream from the audio encoder, packetize the compression coded stream into the first packet, and transmit the first packet to the first terminal.

9. The server apparatus according to claim 7, wherein the bandwidth estimation and rate control processor is configured to receive information of data size of the probe packet, transmission time of the probe packet at the server apparatus, and reception time of the probe packet at the terminal,

perform sorting in order of the transmission time,

calculate a bandwidth estimation value at each of predetermined points in time,

perform temporal smoothing of the bandwidth estimation value, and

using the smoothed bandwidth estimation value, calculate a value so as not to exceed the bandwidth estimation value, as a bit rate of the image encoder, at each of predetermined points in time, output the calculated bit rate to the first image encoder.

4. A communication method for a server apparatus adapted to transmit information to a first terminal connected to the server apparatus via a network, the transmitted information generated by an application configured to run on a virtual terminal of the server apparatus, the transmission of the information being responsive to an operation on the first terminal the method comprising:

transmitting a probe packet at predetermined timing to the first terminal via the network;

receiving a reply packet returned via the network from the first terminal that receives the probe packet, and estimating a bandwidth of the network based on information included in the reply packet;

calculating a rate used for performing compression coding of at least one of screen information and an audio signal accompanying the screen information, generated by the application on the virtual terminal, wherein the calculated rate is a value that does not exceed the estimated bandwidth;

using the calculated rate to perform compression coding of at least one of the screen information and the audio signal accompanying the screen information; and

storing the compression coding result in a packet of a predetermined protocol to transmit the packet, wherein the method further comprises;

based on information extracted from the reply packet, the information includes data size of the probe packet, transmission time of the probe packet at the server apparatus, and reception time of the probe packet at the first terminal,

performing sorting in order of transmission time and calculating a bandwidth estimation value W by the following equation;

5. The communication method according to claim 4, comprising:

using the smoothed bandwidth estimation value B(n), calculating a bit rate C(n) for compression coding in accordance with the following equation, at each predetermined point in time;

C(n)=(1−

γ

)×

C(n−

1)+γ

×

B(n)

where C(n) is a bit rate at an n-th point in time, γ

is a constant in a range of 0<

γ

<

1, and B(b) is a bandwidth estimation value after smoothing, and where the bit rate C(n) is less than or equal to B(n).

10. The communication method according to claim 4, comprising:

transmitting continuously a plurality of the probe packets to the first terminal, each of the probe packets including information of at least data size of the probe packet and transmission time of the probe packet by the server apparatus;

receiving a plurality of reply packets from the first terminal in response to a plurality of the probe packets transmitted continuously; and

extracting information of data size of the probe packet, transmission time of the probe packet by the server apparatus, and reception time of the probe packet by the first terminal to use the extracted information for estimation of the bandwidth.

11. A non-transitory computer readable recording medium storing a program therein, the program causing a computer forming a server apparatus adapted to transmit information to a first terminal via a network to execute processing, the information generated by an application configured to run on a virtual terminal of the server apparatus, the transmission of the information being responsive to an operation on the first terminal, the processing comprising:

transmitting a probe packet at predetermined timing to the first terminal via the network, and receiving a reply packet returned via the network from the first terminal receiving the transmitted probe packet;

estimating a bandwidth of the network based on information included in the reply packet, and calculating a rate used for performing compression coding of at least one of screen information and an audio signal accompanying the screen information, generated by the application on the virtual terminal, wherein the calculated rate is a value that does not exceed the estimated bandwidth;

using the calculated rate to perform compression coding of at least one of the screen information and an audio signal accompanying the screen information; and

storing the compression coding result in a packet of a predetermined protocol to transmit the packet to the first terminal via the network, wherein the processing further comprises;

based on information extracted from the reply packet, wherein the information includes data size of the probe packet, transmission time of the probe packet at the server apparatus, and reception time of the probe packet at the terminal,

performing sorting in order of transmission time and calculating a bandwidth estimation value W by the following equation;

12. The non-transitory computer readable recording medium according to claim 11, wherein the program stored in the recording medium causes the computer to execute processing comprising:

using the smoothed bandwidth estimation value B(n), calculating a bit rate C(n) for compression coding in accordance with the following equation, at each predetermined point in time;

C(n)=(1−

γ

)×

C(n−

1)+γ

×

B(n)

where C(n) is a bit rate at an n-th point in time, γ

is a constant in a range of 0<

γ

<

1, and B(b) is a bandwidth estimation value after smoothing, and where the bit rate C(n) is less than or equal to B(n).