Method and system for optimized parallel queuing

• US 9,104,742 B1
• Filed: 03/13/2013
• Issued: 08/11/2015
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Wei, Shu-Shang Sam, Korshunov, Dmitry Yuryevich, Wong, Linda J.
• Primary Examiner(s): Lewis, Cheryl
• Application Number:
  US13/798,916
• Time in Patent Office: 881 Days
• Field of Search: 707/607, 707/609, 707/610, 707/703, 707/705, 707/822, 715/200, 715/203, 715/205, 715/230, 715/233, 709/213, 709/214, 709/227, 709/231
• US Class Current: 1/1
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

A method and system for sharing and synchronizing content between a content repository and file sharing service. An application server includes a connector engine and a connector agent. The connector engine with is configured with a plurality of threads and queues designed to listen for changes at either the file sharing service or the repository, to process those changes in parallel, and to coordinate synchronization of uploads to the file sharing service and downloads to the content repository.

Claims

1. A method for sharing and synchronizing content between a content repository and a file sharing service, executed by a processor, comprising:

publishing content from the content repository to the file sharing service, wherein other computing devices connected to the file sharing service can share the content;

synchronizing changes made to content at either the file sharing service or the repository;

monitoring the file sharing service for new content or changes made to existing content, and to download the new content or changes made to existing content at the file sharing service to the content repository; and

monitoring the content repository for new content or changes made to existing content, and to upload the new content or changes made to existing content at the content repository to the file sharing service.

2. The method of claim 1, further comprising:

monitoring the content repository for new content or changes made to existing content; and

uploading the new content or changes made to existing content at the content repository to the file sharing service.

3. The method of claim 1, further comprising:

monitoring the file sharing service for new content or changes made to existing content; and

downloading the new content or changes made to existing content at the file sharing service to the content repository.

4. The method of claim 1, further comprising:

enabling a plurality of threads and queues configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to process the changes made to content in parallel.

5. The method of claim 4, further comprising:

enabling a first thread, the first thread configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to sort the changes made to content;

storing the sorted changes from the first thread into a first queue;

enabling a second thread, the second thread configured to receive and filter the sorted changes;

storing the filtered sorted changes from the second thread into a second queue;

enabling a third thread, the third thread configured to receive the filtered sorted changes, and to transfer the filtered sorted changes to either the file sharing service or the repository;

storing the status of the changes made to content in a third queue; and

enabling a fourth thread, the fourth thread configured to maintain and update content status based on the status recorded in the third queue.

6. The method of claim 5, the step of enabling a second thread further comprising enabling a plurality of second threads configured to receive a plurality of the sorted changes from the first queue in parallel, and to filter the sorted changes in parallel.

7. The method of claim 5, the step of enabling a third thread further comprising enabling a plurality of third threads configured to receive a plurality of the filtered sorted changes from the second queue in parallel, to transfer the filtered sorted changes from each of the third threads to either the file sharing service or the repository, and to record the status of the changes made to content in a third queue.

8. The method of claim 5, further comprising:

storing the filtered sorted changes from the second queue into a persistent store.

9. A non-transitory computer-readable storage medium encoded with instructions executable by a processor, for performing a method for sharing and synchronizing content between a content repository and a file sharing service, the instructions comprising:

publishing content from the content repository to the file sharing service, wherein other computing devices connected to the file sharing service can share the content;

synchronizing changes made to content at either the file sharing service or the repository

monitoring the file sharing service for new content or changes made to existing content, and to download the new content or changes made to existing content at the file sharing service to the content repository; and

monitoring the content repository for new content or changes made to existing content, and to upload the new content or changes made to existing content at the content repository to the file sharing service.

10. The computer-readable storage medium of claim 9, further comprising:

monitoring the content repository for new content or changes made to existing content; and

uploading the new content or changes made to existing content at the content repository to the file sharing service.

11. The computer-readable storage medium of claim 9, further comprising:

monitoring the file sharing service for new content or changes made to existing content; and

downloading the new content or changes made to existing content at the file sharing service to the content repository.

12. The computer-readable storage medium of claim 9, further comprising:

enabling a plurality of threads and queues configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to process the changes made to content.

13. The computer-readable storage medium of claim 12, further comprising:

enabling a first thread, the first thread configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to sort the changes made to content;

storing the sorted changes from the first thread into a first queue;

enabling a second thread, the second thread configured to receive and filter the sorted changes;

storing the filtered sorted changes from the second thread into a second queue;

enabling a third thread, the third thread configured to receive the filtered sorted changes, and to transfer the filtered sorted changes to either the file sharing service or the repository;

storing the status of the changes made to content in a third queue; and

enabling a fourth thread, the fourth thread configured to maintain and update content status based on the status recorded in the third queue.

14. The computer-readable storage medium of claim 13, the step of enabling a second thread further comprising enabling a plurality of second threads configured to receive a plurality of the sorted changes from the first queue in parallel, and to filter the sorted changes in parallel.

15. The computer-readable storage medium of claim 13, the step of enabling a third thread further comprising enabling a plurality of third threads configured to receive a plurality of the filtered sorted changes from the second queue in parallel, to transfer the filtered sorted changes from each of the third threads to either the file sharing service or the repository, and to record the status of the changes made to content in a third queue.

16. The computer-readable storage medium of claim 13, further comprising:

storing the filtered sorted changes from the second queue into a persistent store.

17. An apparatus for sharing and synchronizing content between a content repository and a file sharing service comprising:

an application server including at least one processor, the processor programmed with executable instructions, which when executed by the processor cause the processor to (i) publish content from the repository to the file sharing service so that other computing devices connected to the file sharing service can share the content;

(ii) synchronize changes made to content at either the file sharing service or the repository;

(iii) monitor the file sharing service for new content or changes made to existing content, and to download the new content or changes made to existing content at the file sharing service to the content repository; and

monitor the content repository for new content or changes made to existing content, and to upload the new content or changes made to existing content at the content repository to the file sharing service.

18. The apparatus of claim 17, the connector engine further comprising:

a plurality of threads and queues configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to process the changes made to content in parallel.

19. The apparatus of claim 18, the connector engine further comprising:

a first thread configured to monitor the content repository and the file sharing service in order to detect changes made to content, and to sort the changes made to content;

a first queue storing the sorted changes from the first thread;

a second thread configured to receive and filter the sorted changes from the first queue;

a second queue storing the filtered sorted changes from the second thread;

a third thread configured to receive the filtered sorted changes from the second queue, and to transfer the filtered sorted changes to either the file sharing service or the repository;

a third queue storing the status of the changes made to content from the third thread; and

a fourth thread configured to maintain and update content status based on the status recorded in the third queue.