Real time collaboration and document editing by multiple participants in a content management system

• US 10,298,630 B2
• Filed: 05/23/2017
• Issued: 05/21/2019
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Lo, Min Ming, Wu, Michael, Reddy, Shravan
• Primary Examiner(s): Kim, Edward
• Application Number:
  US15/602,948
• Publication Number:
  US 20170257405A1
• Time in Patent Office: 728 Days
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/08 Cursor circuits
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

A content management system allows users to collaboratively edit, draft, or otherwise work with the same content item simultaneously by displaying a real-time data stream of a native application displaying a content item on a host device in a screen sharing window of a webpage. The screen sharing window is capable of receiving edit inputs and sending the edit inputs to a client application on the host device. The edit inputs are provided as interpretable code for the native application for the content item and the client application provides the interpretable code to the native application to have the changes corresponding to the edit inputs incorporated into the content item. Further, the cursors of other users participating in the collaborative editing session are displayed in the screen sharing window to allow each participant to view and follow the changes made by the other participants of the session.

Claims

1. A method comprising:

receiving a request to open a content item stored in association with a user account at a content management system (CMS), the request being received by a client application associated with the CMS installed on a client device, the content item being displayed via a native application associated with the content item on the client device, the native application associated with the content item being different from the client application and being controlled by a different operating system process than the client application;

establishing, by the client application, a peer-to-peer connection with a second client device and a third client device to initiate a collaborative editing session to edit the content item with the second client device and the third client device;

responsive to the peer-to-peer connection being established, sending a real-time data stream of the content item to the second client device and the third client device, the real-time data stream being presented on a screen sharing window within a webpage associated with the CMS displayed on the second client device and the third client device;

generating a first cursor overlay identifying a user of the second client device for presentation on the third client device within the screen sharing window with the real-time data stream to a user of the third client device, the first cursor overlay corresponding to an overlay layer separate from the real-time data stream; and

providing the first cursor overlay corresponding to the user of the second client device to the third client device, the first cursor overlay being displayed in the overlay layer separate from the real-time data stream within the screen sharing window on the third client device;

receiving, by the client application, one or more changes to the content item being made via the sharing window on the second client device; and

providing, by the client application, a command corresponding to the one or more changes to the native application on the first client device for the native application to apply the one or more changes to the content item on the first client device.

2. The method of claim 1, wherein providing the first cursor overlay corresponding to the user of the second client device to the third client device comprises:

receiving, from the second client device, position information for a cursor of the second client device within the screen sharing window for the first cursor overlay;

translating, by the client application on the first client device, the position information to correspond to a location of the second cursor within the native application on the first client device; and

providing the translated position information to the third client device for display in the screen sharing window with the real-time data stream of the content item.

3. The method of claim 1, wherein the one or more changes are received by the client application as interpretable code for the one or more changes, the native application executing the interpretable code to cause the one or more changes to be applied to the content item.

4. The method of claim 3, wherein the interpretable code does not include an updated content item with the one or more changes applied thereto.

5. The method of claim 3, wherein the content item is a document associated with a word processing application and the one or more changes to the content item correspond to one or more changes to text in the document.

6. The method of claim 5, wherein the one or more changes to the content item are displayed in the real-time data stream of the content item presented on the second client device and the third client device via the screen sharing window.

7. The method of claim 6, further comprising:

receiving, by the client application, a request to join the collaborative editing session by the second client device, the request being provided through the webpage associated with the CMS, the peer-to-peer connection being established in response to the client application receiving an acceptance command provided by a user of the client device.

8. A method comprising:

establishing, by a client application on a first client device, a peer-to-peer connection with a second client device and a third client device to initiate a collaborative editing session to edit a content item with the second client device and the third client device, the first client device being associated with a first user account, the second client device being associated with a second user account, and the content item being provided on the first client device and the second client device via a native application of the content item, the native application of the content item being different from the client application and being controlled by a different operating system process than the client application;

sending a real-time data stream of the content item to the second client device and the third client device over the peer-to-peer connection, the real-time data stream being presented on a screen sharing window within a webpage displayed on each of the second client device and the third client device;

generating a first cursor overlay identifying a second user of the second client device for presentation within the screen sharing window with the real-time data stream to a third user of the third client device, the first cursor overlay corresponding to a first overlay layer separate from the real-time data stream;

providing the first cursor overlay corresponding to the second user of the second client device to the third client device, the first cursor overlay being displayed in the first overlay layer separate from the real-time data stream within the screen sharing window of the third client device;

generating a second cursor overlay identifying a third user of the third client device for presentation within the screen sharing window with the real-time data stream to the second user of the second client device, the second cursor overlay corresponding to a second overlay layer separate from the real-time data stream;

providing the second cursor overlay corresponding to the third user of the third client device to the second client device, the second cursor overlay being displayed in the second overlay layer separate from the real-time data stream within the screen sharing window of the second client device;

receiving, by the client application, one or more changes to the content item being made via the sharing window on the second client device; and

providing, by the client application, a command corresponding to the one or more changes to the native application on the first client device for the native application to apply the one or more changes to the content item on the first client device.

9. The method of claim 8, further comprising:

receiving, by the client application, a request to save the content item with the one or more changes; and

causing, by the client application, the content item with the one or more changes to be saved at a content management system (CMS) from the first client device, the CMS further synchronizing the content item to a local copy of the content item stored in associated with a second user account of the second client device.

10. The method of claim 8, wherein providing the first cursor overlay corresponding to the second user of the second client device to the third client device comprises:

receiving, from the second client device, position information for a cursor of the second client device within the screen sharing window for the first cursor overlay;

translating, by the client application on the first client device, the position information to correspond to a location of the second cursor within the native application on the first client device; and

providing the translated position information to the third client device for display in the screen sharing window with the real-time data stream of the content item.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a first client device to:

present, via a web browser of the first client device, a webpage for display, the webpage providing access to one or more content items associated with a first user account stored with a content management system (CMS);

send a request to join a collaborative editing session to edit a content item of the one or more content items with a second client device and a host client device, the content item being displayed by the host client device via a native application of the content item, wherein a CMS client application associated with a second user account with the CMS is installed on the host client device, the native application of the content item being different from the client application and being controlled by a different operating system process than the CMS client application;

establish, via the web browser of the first client device, a peer-to-peer connection with the CMS client application of the host client device to join the collaborative editing session, wherein establishing the peer-to-peer connection includes present a screen sharing window within the webpage that displays a real-time data stream of the native application displaying the content item received over the peer-to-peer connection from the host client device; and

responsive to a cursor of the first client device being detected within the screen sharing window, send position information for the cursor within the screen sharing window over the peer-to-peer connection to the CMS client application on the host device, the CMS client application;

receiving the position information from the first client device,

translating the position information to a location of the cursor within the native application of the content item on the host client device, and

providing the translated position information for the location of the cursor of the first client device within the native application of the content item on the host client device to the second client device, wherein a reproduction of the cursor of the first client device is presented a screen sharing window within a webpage on the second client device with the real-time data stream of the native application displaying the content item, the reproduction of the cursor being presented in an overlay layer on the second client device separate from the real-time data stream;

receiving, by the CMS client application on the host device, one or more changes to the content item being made by a user of the second client device via the screen sharing window on the second client device; and

providing, by the CMS client application, a command corresponding to the one or more changes to the native application on the host client device, the command causing the native application to apply the one or more changes to the content item on the host client device.

12. The non-transitory computer-readable storage medium of claim 11, wherein the CMS client application further:

receiving, from the second client device, position information for a second cursor of the second client device within the screen sharing window on the second client device,

translating the position information to correspond to a location of the second cursor within the native application of the content item on the host client device; and

providing the translated position information to the first client device for display in the screen sharing window with the real-time data stream of the content item, wherein a reproduction of the second cursor of the second client device is presented a screen sharing window within the webpage on the first client device with the real-time data stream of the native application displaying the content item, the reproduction of the second cursor being presented in an overlay layer on the first client device separate from the real-time data stream.

13. The non-transitory computer-readable storage medium of claim 12, wherein the one or more changes are received by the CMS client application as interpretable code for the one or more changes, the native application executing the interpretable code to cause the one or more changes to be applied to the content item.

14. The non-transitory computer-readable storage medium of claim 12, wherein the interpretable code does not include an updated content item with the one or more changes applied thereto.

15. The non-transitory computer-readable storage medium of claim 12, wherein the content item is a document associated with a word processing application and the one or more changes to the content item correspond to one or more changes to text in the document.

16. The non-transitory computer-readable storage medium of claim 11, wherein the one or more changes to the content item are displayed in the real-time data stream of the content item presented on the first client device and the second client device via the screen sharing window of the first client device and the screen sharing window on the second client device.

17. The method of claim 11, wherein the CMS client application on the host client device further:

receiving, by the CMS client application, a request to join the collaborative editing session by the first client device, the request being provided by the first client device through the webpage providing access the first user account with the CMS, the peer-to-peer connection being established in response to the CMS client application receiving an acceptance command provided by a user of the host client device.