Real time collaboration and document editing by multiple participants in a content management system

• US 9,667,676 B1
• Filed: 01/29/2016
• Issued: 05/30/2017
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Lo, Min Ming, Wu, Michael, Reddy, Shravan
• Primary Examiner(s): Kim, Edward
• Application Number:
  US15/011,020
• Time in Patent Office: 487 Days
• Field of Search: 709204, 709205, 709227, 709230, 709231
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/08 Cursor circuits
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

A content management system allows users to collaboratively edit, draft, or otherwise work with the same content item simultaneously by displaying a real-time data stream of a native application displaying a content item on a host device in a screen sharing window of a webpage. The screen sharing window is capable of receiving edit inputs and sending the edit inputs to a client application on the host device. The edit inputs are provided as interpretable code for the native application for the content item and the client application provides the interpretable code to the native application to have the changes corresponding to the edit inputs incorporated into the content item. Further, the cursors of other users participating in the collaborative editing session are displayed in the screen sharing window to allow each participant to view and follow the changes made by the other participants of the session.

Claims

1. A method comprising:

receiving a request to open a content item stored in association with a user account at a content management system (CMS), the request being received by a client application associated with the CMS installed on a client device, the content item being displayed via a native application associated with the content item on the client device;

establishing, by the client application, a peer-to-peer connection with at least one second client device to initiate a collaborative editing session to edit the content item with the at least one second client device;

responsive to the peer-to-peer connection being established, sending a real-time data stream of the content item to the at least one second client device, the real-time data stream being presented on a screen sharing window within a webpage associated with the CMS displayed on the at least one second client device, wherein transmitting the real-time data stream of the content item to the at least one second client device establishes the client device as a host client device for the collaborative editing session;

receiving, by the client application, instructions from the at least one second client device that when executed by the native application on the host client device cause the native application to apply one or more changes to the content item, the instructions corresponding to a user of the at least one second client device making the one or more changes to the content item via the sharing window, wherein the instructions do not include an updated content item with the one or more changes applied thereto; and

providing, by the client application, the instructions to the native application to apply the one or more changes to the content item on the host client device.

2. The method of claim 1, further comprising:

receiving, by the client application, a request to save the content item with the one or more changes; and

causing, by the client application, the content item with the one or more changes to be saved at the CMS from the host client device, the CMS further synchronizing the content item to a local copy of the content item stored in association with the at least one second user account on the at least one second client device.

3. The method of claim 1, further comprising:

receiving, from the at least one second client device, position information for at least one cursor within the screen sharing window over the peer to peer connection;

translating, by the client application, the position information to correspond to a location of the at least one cursor within the native application on the host client device; and

providing the translated position information to the at least one second client device for display in the screen sharing window with the real-time data stream of the content item.

4. The method of claim 3, further comprising:

generating a cursor overlay identifying a user of the at least one second client device, the cursor overlay being presented adjacent the cursor in the screen sharing window with the real-time data stream of the content item; and

providing the cursor overlay to the at least one second client device.

5. The method of claim 1, further comprising:

receiving, by the client application, a request to join the collaborative editing session by the at least one second client device, the request being provided through the webpage associated with the CMS,

wherein the peer-to-peer connection is established in response to the client application receiving an acceptance command provided by a user of the host client device.

6. The method of claim 1, further comprising:

sending, by the client application, an invitation to join the collaborative editing session to the at least one second client device,

wherein the peer-to-peer connection is established in response to the at least one second client device accepting the invitation.

7. The method of claim 1, wherein the instructions are a set of interpretable codes for the native application to perform the one or more changes, and wherein the one or more changes to the content item are displayed in the real-time data stream of the content item presented on the at least one second client device via the screen sharing window.

8. A method comprising:

establishing, by a client application of a first client device, a peer-to-peer connection with a second client device to initiate a collaborative editing session to edit a content item with the second client device, the first client device being associated with a first user account and the content item being provided on the first client device via a native application of the content item;

sending a real-time data stream of a content item to the at least one second client device over the peer-to-peer connection, the real-time data stream being presented on a screen sharing window within a webpage associated with a second user account displayed on the second client device;

receiving, by the client application, interpretable code for one or more changes to the content item being made via the sharing window on the second client, wherein the interpretable code does not include an updated content item with the one or more changes applied thereto; and

providing, by the client application, a command corresponding to the interpretable code to the native application on the first client device for the native application to apply the one or more changes to the content item on the first client device.

9. The method of claim 8, further comprising:

receiving, from the second client device, position information for a second cursor within the screen sharing window;

translating, by the client application, the position information to correspond to a location of the second cursor within the native application on the first client device; and

providing the translated position information to the second client device for display in the screen sharing window with the real-time data stream of the content item.

10. The method of claim 9, further comprising:

receiving second position information for a third cursor from a third client device participant to the collaborative editing session;

translating, by the client application, the second position information to correspond to a location of the third cursor within the native application; and

providing the second translated position information to the second client device and the third client device to display the third cursor in the screen sharing window,

wherein the second cursor and the third cursor are simultaneously displayed with the real-time data stream of the content item displayed in the screen sharing window.

11. The method of claim 10, further comprising:

determining a cursor overlay identifying a second user of the second client device and a third cursor overlay identifying a third user of the third client device, the second cursor overlay and the third cursor overlay being presented adjacent each cursor in the screen sharing window with the real-time data stream of the content item; and

providing the second cursor overlay and the third cursor overlay to the second client device and the third client device.

12. The method of claim 8, further comprising:

receiving, by the client application, a request to join the collaborative editing session by the second client device, the request being provided through the webpage associated with the second user account,

wherein the peer-to-peer connection is established in response to the client application receiving an acceptance command provided by a user of the host client device.

13. The method of claim 8, wherein the one or more changes to the content item are displayed in the real-time data stream of the content item presented on the second client device via the screen sharing window.

14. The method of claim 13, further comprising:

capturing, by the client application, the real-time data stream of the content item displayed by the native application on the first client device, wherein the first client device is a host client device for the content item.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

present, via a web browser of a client device, a webpage for a content management system (CMS) for display, the webpage providing access to one or more content items associated with a first user account stored with the CMS;

send a request to join a collaborative editing session to edit a content item of the one or more content items with one or more additional client devices including a host client device, the content item being displayed by the host client device via a native application associated with the content item, wherein a CMS client application associated with a second user account with the CMS is installed on the host client device;

establish, via the web browser, a peer-to-peer connection with the CMS client application of the host client device to join the collaborative editing session;

responsive to establishing the peer-to-peer connection, present a screen sharing window within the webpage, the screen sharing window displaying a real-time data stream of the native application displaying the content item received over the peer-to-peer connection from the host client device; and

responsive to a cursor of the client device being detected within the screen sharing window, send position information for the cursor within the screen sharing window over the peer-to-peer connection to the CMS client application, the CMS client application receiving the position information and translating the position information to a location of the cursor within the native application on the host client device;

receive, from the CMS client application on the host client device, the translated position information for the location of the cursor within the native application on the host client device;

present a reproduction of the cursor corresponding to the translated position information for display in the screen sharing window with the real-time data stream of the native application displaying the content item; and

responsive to detecting an editing event corresponding to a user of the client device making an edit to the content item via the screen sharing window, send native application interpretable code for the editing event to the CMS client application on the host client device, the CMS client application providing the native application interpretable code to the native application to perform the edit to the content item on the host client device, wherein the interpretable code does not include an updated content item with the one or more changes applied thereto.

16. The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

receive second translated position information for a second cursor corresponding to an additional client device participant to the collaborative editing session; and

present the second translated position information of the second cursor for display in a screen sharing window, the second translated position information corresponding a location of the second cursor within the native application of the content item,

wherein a third cursor of the host client device is viewable via the real-time data stream of the content item displayed in the screen sharing window.

17. The non-transitory computer-readable storage medium of claim 15, where the CMS client application of the host client device transmits the translated position information to at least one additional client device participant to the collaborative editing session, the location of the cursor within the content item is displayed in a screen sharing window of the at least one additional client device participant to the collaborative editing session.

18. The non-transitory computer-readable storage medium of claim 15, wherein the one or more changes to the content item are displayed in the real-time data stream of the content item presented on the at least one additional client device participant to the collaborative editing session via the screen sharing window.

19. The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

receive, from the host client device, an invitation to join the collaborative editing session.

20. The non-transitory computer-readable storage medium of claim 15, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

send a request to the host client device to join the collaborative editing session, wherein the peer-to-peer connection is established in response to the host client device accepting the request.