Systems, methods, and apparatuses for accepting late joiners with screen sharing

• US 10,157,031 B2
• Filed: 05/18/2017
• Issued: 12/18/2018
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Spencer, Barry L.
• Primary Examiner(s): Meky, Moustafa M
• Assistant Examiner(s): ISLAM, SM ZAKIRUL
• Application Number:
  US15/599,371
• Publication Number:
  US 20170255440A1
• Time in Patent Office: 579 Days
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/60 Network streaming of media packets

Abstract

In accordance with disclosed embodiments, there are provided methods, systems, and apparatuses for accepting late joiners with screen sharing including, for example, means for receiving, at a server, a key frame from a publishing client sharing its screen, the key frame defining the screen of the publishing client in its entirety at the beginning of a screen sharing session with one or more viewing clients; transmitting the key frame to the one or more viewing clients; iteratively processing each of a plurality of delta frames from the publishing client specifying changes to the screen of the publishing client, wherein the iterative processing includes: (i) receiving each delta frame, (ii) updating an aggregated current key frame with the delta frame received, and (iii) sending the delta frame to the one or more viewing clients. Such means further include: accepting a late joiner viewing client for the screen sharing session; sending the aggregated current key frame to the late joiner viewing client; and sending subsequently received delta frames to the one or more viewing clients and to the late joiner viewing client. Other related embodiments are disclosed.

Claims

1. A method comprising:

transmitting, with a video distribution hub, a key frame to one or more viewing clients, the key frame defining a screen of a publishing client in its entirety at a beginning of a screen sharing session with one or more viewing clients, wherein the screen includes one or more active windows to provide dynamically changing content;

maintaining, with the video distribution hub, at least one current key frame comprising the key frame updated, with a frame aggregator, utilizing one or more delta frames that specify changes to the key frame;

iteratively processing, with the video distribution hub, each of a plurality of delta frames specifying changes to the one or more active windows, wherein the iterative processing comprises;

(i) receiving each delta frame via a receive interface of the video distribution hub,

(ii) updating an aggregated current key frame with the delta frame received with the frame aggregator, and

(iii) sending the delta frame to the one or more viewing clients via one or more transmit interfaces of the video distribution hub;

sending the current key frame to a late joiner viewing client via the one or more transmit interfaces, where in the late joiner viewing client joins the screen sharing session after the beginning of the screen sharing session; and

sending subsequently received delta frames to the one or more viewing clients and to the late joiner viewing client via the one or more transmit interfaces.

2. The method of claim 1, further comprising:

establishing a communications interface with a publishing client prior to receiving the key frame from the publishing client; and

establishing a communications interface with the one or more viewing clients to share the screen from the publishing client prior to receiving the key frame from the publishing client.

3. The method of claim 2, further comprising:

transmitting the key frame to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session; and

transmitting one or more of the plurality of delta frames from the publishing client to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session.

4. The method of claim 1, further comprising:

storing the key frame from a publishing client sharing its screen at the server as a current key frame for time₀ corresponding to the start of the screen sharing session;

receiving a first of the plurality of delta frames from the publishing client at time₁; and

aggregating the first of the plurality of delta frames into the current key frame for time₀to result in an aggregated current key frame for time₁.

5. The method of claim 4, wherein accepting a late joiner viewing client for the screen sharing session and sending the aggregated current key frame to the late joiner viewing client, comprises:

accepting the late joiner viewing client after time₁;

sending the late joiner viewing client the aggregated current key frame for time₁, wherein the aggregated current key frame for time₁ defines a current state of the screen of the publishing client in its entirety at time time₁;

receiving a second of the plurality of delta frames from the publishing client at time₂;

aggregating the second of the plurality of delta frames into the aggregated current key frame for time₁ to result in an aggregated current key frame for time₂;

storing the aggregated current key frame for time₂; and

sending the second of the plurality of delta frames from the publishing client at time₂ to the one or more viewing clients and to the late joiner viewing client.

6. The method of claim 5, further comprising:

receiving further ones of the plurality of delta frames from the publishing client at time₃ through time_n;

aggregating each of the further ones of the plurality of delta frames into the aggregated current key frame for time₃ through time_n to result in an aggregated current key frame for time_n;

storing the aggregated current key frame for time_n; and

sending the further ones of the plurality of delta frames from the publishing client at time₃ through time_n to the one or more viewing clients and to the late joiner viewing client.

7. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, are configurable to cause the one or more processors to:

transmit, with a video distribution hub, a key frame to one or more viewing clients, the key frame defining a screen of a publishing client in its entirety at a beginning of a screen sharing session with one or more viewing clients, wherein the screen includes one or more active windows to provide dynamically changing content;

maintain, with the video distribution hub, at least one current key frame comprising the key frame updated, with a frame aggregator, utilizing one or more delta frames that specify changes to the key frame;

iteratively process, with the video distribution hub, each of a plurality of delta frames specifying changes to the one or more active windows, wherein the iterative processing comprises;

(i) receiving each delta frame via a receive interface of the video distribution hub,

(ii) updating an aggregated current key frame with the delta frame received with the frame aggregator, and

(iii) sending the delta frame to the one or more viewing clients via one or more transmit interfaces of the video distribution hub;

send the current key frame to a late joiner viewing client via the one or more transmit interfaces, where in the late joiner viewing client joins the screen sharing session after the beginning of the screen sharing session; and

send subsequently received delta frames to the one or more viewing clients and to the late joiner viewing client via the one or more transmit interfaces.

8. The non-transitory computer-readable medium of claim 7, further comprising instructions that, when executed by the one or more processors, are configurable to cause the one or more processors to:

establish a communications interface with a publishing client prior to receiving the key frame from the publishing client; and

establish a communications interface with the one or more viewing clients to share the screen from the publishing client prior to receiving the key frame from the publishing client.

9. The non-transitory computer-readable medium of claim 8, further comprising instructions that, when executed by the one or more processors, are configurable to cause the one or more processors to:

transmit the key frame to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session; and

transmit one or more of the plurality of delta frames from the publishing client to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session.

10. The non-transitory computer-readable medium of claim 7, further comprising instructions that, when executed by the one or more processors, are configurable to cause the one or more processors to:

store the key frame from a publishing client sharing its screen at the server as a current key frame for time₀ corresponding to the start of the screen sharing session;

receive a first of the plurality of delta frames from the publishing client at time₁; and

aggregating the first of the plurality of delta frames into the current key frame for time₀ to result in an aggregated current key frame for time₁.

11. The non-transitory computer-readable medium of claim 10, wherein accepting a late joiner viewing client for the screen sharing session and sending the aggregated current key frame to the late joiner viewing client, comprises instructions that, when executed by the one or more processors, are configurable to cause the one or more processors to:

accept the late joiner viewing client after time₁;

send the late joiner viewing client the aggregated current key frame for time₁, wherein the aggregated current key frame for time₁ defines a current state of the screen of the publishing client in its entirety at time time₁;

receive a second of the plurality of delta frames from the publishing client at time₂;

aggregate the second of the plurality of delta frames into the aggregated current key frame for time₁ to result in an aggregated current key frame for time₂;

store the aggregated current key frame for time₂; and

send the second of the plurality of delta frames from the publishing client at time₂ to the one or more viewing clients and to the late joiner viewing client.

12. The non-transitory computer-readable medium of claim 11, further comprising instructions that, when executed by the one or more processors, are configurable to cause the one or more processors to:

receive further ones of the plurality of delta frames from the publishing client at time₃ through time_n;

aggregate each of the further ones of the plurality of delta frames into the aggregated current key frame for time₃ through time_n to result in an aggregated current key frame for time_n;

storing the aggregated current key frame for time_n; and

send the further ones of the plurality of delta frames from the publishing client at time₃ through time_nto the one or more viewing clients and to the late joiner viewing client.

13. A system comprising:

a physical memory device;

one or more processors coupled with the physical memory device, the one or more processors configurable to

transmit, with a video distribution hub, a key frame to one or more viewing clients, the key frame defining a screen of a publishing client in its entirety at a beginning of a screen sharing session with one or more viewing clients, wherein the screen includes one or more active windows to provide dynamically changing content;

maintain, with the video distribution hub, at least one current key frame comprising the key frame updated, with a frame aggregator, utilizing one or more delta frames that specify changes to the key frame;

iteratively process, with the video distribution hub, each of a plurality of delta frames specifying changes to the one or more active windows, wherein the iterative processing comprises;

(i) receiving each delta frame via a receive interface of the video distribution hub,

(ii) updating an aggregated current key frame with the delta frame received with the frame aggregator, and

(iii) sending the delta frame to the one or more viewing clients via one or more transmit interfaces of the video distribution hub;

send the current key frame to a late joiner viewing client via the one or more transmit interfaces, where in the late joiner viewing client joins the screen sharing session after the beginning of the screen sharing session; and

send subsequently received delta frames to the one or more viewing clients and to the late joiner viewing client via the one or more transmit interfaces.

14. The system of claim 13, wherein the one or more processors are further configurable to:

establish a communications interface with a publishing client prior to receiving the key frame from the publishing client; and

establish a communications interface with the one or more viewing clients to share the screen from the publishing client prior to receiving the key frame from the publishing client.

15. The system of claim 14, wherein the one or more processors are further configurable to:

transmit the key frame to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session; and

transmit one or more of the plurality of delta frames from the publishing client to the one or more viewing clients prior to accepting the late joiner viewing client for the screen sharing session.

16. The system of claim 13, wherein the one or more processors are further configurable to:

store the key frame from a publishing client sharing its screen at the server as a current key frame for time₀ corresponding to the start of the screen sharing session;

receive a first of the plurality of delta frames from the publishing client at time₁; and

aggregating the first of the plurality of delta frames into the current key frame for time₀ to result in an aggregated current key frame for time₁.

17. The system of claim 13, wherein the one or more processors are further configurable to:

accept the late joiner viewing client after time₁;

send the late joiner viewing client the aggregated current key frame for time₁, wherein the aggregated current key frame for time₁ defines a current state of the screen of the publishing client in its entirety at time time₁;

receive a second of the plurality of delta frames from the publishing client at time₂;

aggregate the second of the plurality of delta frames into the aggregated current key frame for time₁ to result in an aggregated current key frame for time₂;

store the aggregated current key frame for time₂; and

send the second of the plurality of delta frames from the publishing client at time₂ to the one or more viewing clients and to the late joiner viewing client.

18. The system of claim 17, further comprising instructions that, wherein the one or more processors are further configurable to:

receive further ones of the plurality of delta frames from the publishing client at time₃ through time_n;

aggregate each of the further ones of the plurality of delta frames into the aggregated current key frame for time₃ through time_n to result in an aggregated current key frame for time_n;

storing the aggregated current key frame for time_n; and

send the further ones of the plurality of delta frames from the publishing client at time₃ through time_n to the one or more viewing clients and to the late joiner viewing client.