Realtime communications and network browsing client

• US 10,003,624 B2
• Filed: 05/27/2015
• Issued: 06/19/2018
• Est. Priority Date: 01/15/2009
• Status: Active Grant

• Current Assignee: Sococo, Inc.
• Original Assignee: Sococo, Inc.
• Inventors: Van Wie, David, Altmaier, Joseph
• Primary Examiner(s): Chan, Wing F
• Assistant Examiner(s): Ng, Billy H
• Application Number:
  US14/722,382
• Publication Number:
  US 20150256579A1
• Time in Patent Office: 1,119 Days
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 65/60 Network streaming of media packets

Abstract

A graphical user interface is displayed. The graphical user interface includes at least one communication control for managing communications with other network nodes and at least one browser control for navigating to different network resource addresses. Human perceptible realtime output is generated from realtime communication data relating to a realtime communication session with at least one other client network node. In the process of generating of at least some of the human perceptible realtime output, operation of the client network node is managed based on at least one performance target. In the graphical user interface, a graphical representation of a network resource identified in connection with the browser control is displayed.

Claims

1. A method, comprising:

executing a server application to generate instructions for a web browsing client application being executed by a client network node and comprising a realtime framework that manages processing of realtime data streams;

in accordance with the server application instructions, the web browsing client application establishes a peer-to-peer connection for streaming realtime data streams between the client network node and another client network node executing another web browsing client application using a connectionless transport protocol;

during the streaming, the realtime framework of the web browsing application performs operations comprising managing processing of the realtime data streams, and dynamically adjusting the processing of the realtime data streams based on network performance over the peer-to-peer connection, wherein the adjusting comprises, based on a determination that a bandwidth metric has deviated from a performance target, initiating a packet throttling operational mode comprising;

starting a current cycle with a reset bandwidth budget;

on condition that there is a time-critical packet currently queued to transmit and there is sufficient bandwidth budget available in the current cycle, transmitting the time-critical packet and reducing the bandwidth budget for the current cycle by an amount corresponding the transmitted time-critical packet; and

, on condition that no time-critical packet currently is queued to transmit, there is a non-time-critical packet queued to transmit, and there is sufficient bandwidth budget available in the current cycle, transmitting the non-time-critical packet and reducing the bandwidth budget for the current cycle by an amount corresponding the transmitted non-time-critical packet.

2. The method of claim 1, wherein the bandwidth metric is based on network delay over the peer-to-peer connection.

3. The method of claim 1, wherein the bandwidth metric is based on network jitter over the peer-to-peer connection.

4. The method of claim 1, wherein the bandwidth metric is based on network packet loss rate over the peer-to-peer connection.

5. The method of claim 1, wherein the bandwidth metric is based on network bandwidth over the peer-to-peer connection.

6. The method of claim 1, wherein the bandwidth metric is based on network data rate over the peer-to-peer connection.

7. The method of claim 1, wherein the adjusting comprises dynamically adjusting quality of at least one of the realtime data streams being streamed in response to changes in network performance over the peer-to-peer connection.

8. The method of claim 1, wherein the realtime data stream is an audio data stream.

9. The method of claim 1, wherein the realtime data stream is a video data stream.

10. The method of claim 1, wherein, during the streaming, the web browsing client application performs operations comprising:

over the peer-to-peer connection, managing streaming of a realtime data stream designated as a time-critical data stream and another data stream designated as a non-time-critical data stream between the web browsing client application executing on the client network node and the other web browsing client application executing on the other client network node; and

dynamically adjusting relative amounts of network resources available over the peer-to-peer connection between the time-critical data stream and the non-time-critical data stream.

11. The method of claim 10, wherein the adjusting comprises:

by the client network node, dynamically adjusting the allocation of bandwidth available over the peer-to-peer connection between the time-critical data stream and the non-time-critical data stream.

12. The method of claim 11, wherein the adjusting comprises:

by the client network node, delaying delivery of the non-time-critical stream.

13. The method of claim 11, wherein the adjusting comprises:

by the client network node, throttling the non-time-critical data stream.

14. The method of claim 1, wherein the streaming comprises:

transmitting an outgoing realtime data stream from the web browsing client application executing on the client network node to the other web browsing client application executing on the other client network node; and

by the web browsing client application executing the client network node, receiving an incoming realtime data stream from the other web browsing client application executing on the other client network node.

15. The method of claim 1, wherein the server application is a realtime communications application enabling realtime audio conferencing communications between the client network nodes over the peer-to-peer connection.

16. The method of claim 1, wherein the server application is a realtime communications application enabling realtime video conferencing communications between the client network nodes over the peer-to-peer connection.

17. The method of claim 1, wherein the server application is a realtime communications application enabling realtime audio and video conferencing communications between the client network nodes over the peer-to-peer connection in a persistent virtual area that supports establishment of communicant presence and is associated with a visualization comprising a respective graphical indication of each communicant who is present in the virtual area;

and further comprising by the client network node displaying the visualization on a display.

18. The method of claim 1, further comprising:

in accordance with the server application, displaying on a display a graphical user interface comprising at least one communication control for managing realtime communications with other network nodes and at least one browser control for navigating to different network resource addresses, wherein the communication control and the browser control are responsive to user inputs.

19. The method of claim 1, wherein the adjusting further comprises, during the current cycle, exiting the packet throttling operational mode in response to a determination that a period exceeding a threshold duration has passed since a non-time-critical packet was sent.

20. The method of claim 1, wherein the adjusting further comprises:

on condition that the current cycle is not complete and that one or more packets were dropped since a dropped packet was last detected, decreasing the bandwidth budget for the current cycle to a current bandwidth budget and repeating the current cycle with the current bandwidth budget.

21. A system comprising one or more processors and one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

executing a server application to generate instructions for a web browsing client application being executed by a client network node and comprising a realtime framework that manages processing of realtime data streams;

in accordance with the server application instructions, the web browsing client application establishes a peer-to-peer connection for streaming realtime data streams between the client network node and another client network node executing another web browsing client application using a connectionless transport protocol;

during the streaming, the realtime framework of the web browsing application performs operations comprising managing processing of the realtime data streams, and dynamically adjusting the processing of the realtime data streams based on network performance over the peer-to-peer connection, wherein the adjusting comprises, based on a determination that a bandwidth metric has deviated from a performance target, initiating a packet throttling operational mode comprising;

starting a current cycle with a reset bandwidth budget;

on condition that there is a time-critical packet currently queued to transmit and there is sufficient bandwidth budget available in the current cycle, transmitting the time-critical packet and reducing the bandwidth budget for the current cycle by an amount corresponding the transmitted time-critical packet;

otherwise, on condition that no time-critical packet currently is queued to transmit, there is a non-time-critical packet queued to transmit, and there is sufficient bandwidth budget available in the current cycle, transmitting the non-time-critical packet and reducing the bandwidth budget for the current cycle by an amount corresponding the transmitted non-time-critical packet; and

on condition that the current cycle is not complete and that one or more packets were dropped since a dropped packet was last detected, decreasing the bandwidth budget for the current cycle to a current bandwidth budget and repeating the current cycle with the current bandwidth budget.