Synchronizing proprietary data in an external cloud with data in a private storage system

• US 10,230,792 B2
• Filed: 10/23/2015
• Issued: 03/12/2019
• Est. Priority Date: 10/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Engbersen, Ton, Schmatz, Martin
• Primary Examiner(s): Harper, Eliyah S.
• Application Number:
  US14/921,790
• Publication Number:
  US 20170118279A1
• Time in Patent Office: 1,236 Days
• Field of Search: 707620
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

The present invention is notably directed to a method for synchronizing proprietary data in an external cloud provided by a cloud service provider with data of a private storage system. The method comprises, at a synchronization system: copying outward data from a flow of outward data sent from a private cloud to the external cloud, the outward data being proprietary data of an entity that owns data of the private cloud. Next, and in parallel to copying outward data: the synchronization system compares copied outward data with data stored on the private storage system, to determine whether the compared outward data are already replicated in the private storage system. Finally, if it is determined that the compared outward data are not yet replicated in the private storage system, it instructs to store the compared outward data on the private storage system. The present invention is further directed to related systems and computer program products.

Claims

1. A computer-executed method for synchronizing proprietary data in an first cloud computing system comprising networked computer resources provided by a cloud service provider with data of a private data storage system, the method comprising:

copying, by a processor of a computer, outward data from a continuous flow of outward data to be sent from one or more data generating and consuming computer nodes of a second cloud computing system to the first cloud computing system, the outward data being proprietary data of an entity that owns data of the second cloud computing system; and

in parallel to copying outward data at the first computer;

comparing, by said processor, consecutive fixed size blocks of copied outward data of said continuous flow at a second cloud computing system level with data stored on the private data storage system, to determine whether the compared outward data are already replicated in the private data storage system; and

if it is determined that the compared outward data are not yet replicated in the private data storage system, storing, by said processor, the compared outward data on the private data storage system, and wherein said copying of outward data is controlled by said processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections;

copying, using the processor of the first computer, inward data from a continuous flow of inward data to be read, in the one or more data generating and consuming computer nodes of said second cloud computing system, back from the first cloud computing system, the inward data being proprietary data of a same entity that owns said outward data;

and in parallel to copying inward data at the first computer;

comparing, using the processor, consecutive fixed size blocks of copied inward data of said continuous flow at a second cloud computing system level with data already stored on the private storage system, to determine whether the compared inward data are already replicated in the private data storage system; and

if it is determined that the compared inward data are not yet replicated in the private data storage system, storing, by said processor, the compared inward data on the private data storage system, said copying of inward data controlled by said processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections; and

determining, by said processor, a change from use of said second cloud computing system to a use of a further cloud computing system; and

replicating, by the processor, data stored on said private data storage system in said further cloud computing system.

2. The computer-executed method of claim 1, further comprising, at the second cloud computing system and in parallel to the copying of outward data:

monitoring data to detect novel data generated at the second cloud computing system, the novel data being proprietary data of a same entity that owns said outward data; and

upon detecting novel data, notifying the processor of said computer of the detected novel data.

3. The computer-executed method of claim 2, further comprising:

sending a copy of the detected novel data to the processor of said computer.

4. The computer-executed method of claim 2, further comprising:

upon receiving by the processor notification of the novel data by the second cloud computing system, comparing, by the processor, the novel data with data already stored on the private data storage system, to determine whether the compared novel data are already replicated in the private data storage system; and

if it is determined that the compared novel data are not yet replicated in the private data storage system, storing the compared novel data on the private data storage system.

5. The computer-executed method of claim 2, wherein said monitoring of the novel data is performed by a data crawler.

6. The computer-executed method of claim 5, wherein

the data crawler used to monitor the novel data is, or is part of, a cloud-based service run at the first cloud computing system.

7. The computer-executed method of claim 1, wherein

the second cloud computing system is owned by the same entity that owns data of the second cloud computing system.

8. The computer-executed method of claim 1, wherein

the processor of the computer is part of the first cloud computing system.

9. A data synchronization system for synchronizing enterprise data in a first cloud computing system comprising networked computer resources provided by a cloud service provider with data of a private data storage system, wherein the system comprises:

a processor;

a memory coupled to the processor, the memory comprising computer-readable instructions which when executed by the processor, cause the processor to;

copy outward data from a continuous flow of outward data being sent from one or more data generating and consuming computer nodes of a second cloud computing system to the first cloud computing system, the outward data being proprietary data; and

in parallel to the copying outward data;

comparing consecutive fixed size blocks of copied outward data of said continuous flow at a second cloud computing system level with data stored on the private data storage system, to determine whether the compared outward data are already replicated in the private data storage system; and

if it is determined that the compared outward data are not yet replicated in the private storage system, storing the compared outward data on the private storage system, and, wherein said copying of outward data is controlled by said processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections; and

wherein the computer-readable instructions when executed by the processor further cause the processor to;

in parallel to the copying said outward data;

copy inward data from a continuous flow of inward data read, in the one or more data generating and consuming computing nodes of said second cloud computing system, back from the first cloud computing system, the inward data being proprietary data;

compare consecutive fixed size blocks of copied inward data of said continuous flow at a second cloud computing system level with data already stored on the private data storage system, to determine whether the compared inward data are already replicated in the private data storage system; and

if it is determined that the compared inward data are not yet replicated in the private data storage system, store the compared inward data on the private data storage system, said copying of inward data controlled by said processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections; and

the computer-readable instructions when executed by the processor further cause the processor to;

determine a change from use of said second cloud computing system to a use of a further cloud computing system; and

replicate data stored on said private data storage system in said further cloud computing system.

10. The data synchronization system of claim 9, wherein the computer-readable instructions when executed by the processor further cause the processor to:

in parallel to copying both outward data and inward data;

receive novel data from the first cloud computing system, the novel data being proprietary data generated at the first cloud computing system;

compare the novel data with data already stored on the private data storage system, to determine whether the compared novel data are already replicated in the private data storage system; and

if it is determined that the compared novel data in said copy are not yet replicated in the private data storage system, store the compared novel data on the private data storage system.

11. A private cloud computing system comprising:

several data generating and consuming computing nodes, each computing node comprising;

a first processor;

a memory coupled to the first processor, the memory comprising computer-readable instructions which when executed by the first processor, cause the first processor to;

send proprietary data as a continuous flow of outward data to an external cloud computing system provided by a cloud service provider; and

receive proprietary data as a continuous flow of inward data from the external cloud computing system;

a private data storage system for storing enterprise data; and

a data synchronization system comprising a second processor; and

a memory coupled to the second processor, the memory comprising computer-readable instructions which when executed by the second processor, cause the second processor to;

copy proprietary data sent to and received as continuous outward data and inward data flows from the external cloud computing system by the data generating and consuming computing nodes to the private data storage system, said copying of outward data and inward data controlled by said second processor independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections;

receive, from a processor running a data monitoring program at the external cloud computing system, novel proprietary data generated at the external cloud computing system;

compare the novel proprietary data with data already stored on the private storage system, to determine whether the compared novel data are already replicated in the private storage system; and

if it is determined that the compared novel data in said copy are not yet replicated in the private storage system, store the compared novel data on the private storage system, and

determine, by said second processor, a change from use of said external cloud computing system to a use of a further cloud computing system; and

replicate, using said second processor, data stored on said private data storage system in said further cloud computing system.

12. A computer program product for synchronizing proprietary data in an first cloud computing system comprising networked computer resources provided by a cloud service provider with data of a private data storage system, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:

copy outward data from a continuous flow of outward data sent from one or more data generating and consuming computer nodes of a second cloud computing system to the first cloud computing system, the outward data being proprietary data of an entity that owns data of the second cloud computing system; and

in parallel to copying outward data;

compare consecutive fixed size blocks of copied outward data of said continuous flow at a second cloud computing system level with data stored on the private data storage system, to determine whether the compared outward data are already replicated in the private data storage system; and

if it is determined that the compared outward data are not yet replicated in the private data storage system, store the compared outward data on the private data storage system, and wherein said copying of outward data is controlled by a processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections; and

the program instructions are further executable by the computer to cause the computer to, in parallel to copying outward data;

copy inward data from a continuous flow of inward data read, in the one or more data generating and consuming computer nodes of said second cloud computing system, back from the first cloud computing system, the inward data being proprietary data; and

in parallel to copying inward data;

compare consecutive fixed size blocks of copied inward data of said continuous flow at a second cloud computing system level with data already stored on the private data storage system, to determine whether the compared inward data are already replicated in the private data storage system; and

if it is determined that the compared inward data are not yet replicated in the private data storage system, store the compared inward data on the private data storage system, said copying of inward data controlled by said processor, independent of said one or more data generating and consuming computing nodes of said second cloud computing system and without requiring detection of intermittent connections; and

the computer-readable instructions when executed by the computer further cause the computer to;

determine a change from use of said second cloud computing system to a use of a further cloud computing system; and

replicate data stored on said private data storage system in said further cloud computing system.

13. The computer program product of claim 12, wherein the program instructions are further executable by the computer to cause the computer to, in parallel to copying outward data:

receive novel data from the external cloud computing system, the novel data being proprietary data generated at the external cloud computing system, compare the novel data with data already stored on the private storage system, to determine whether the compared novel data are already replicated in the private storage system; and

if it is determined that the compared novel data in said copy are not yet replicated in the private storage system, store the compared novel data on the private storage system.