Staged data migration between data sources and cloud-based storage network

• US 9,225,791 B2
• Filed: 02/28/2011
• Issued: 12/29/2015
• Est. Priority Date: 02/28/2011
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Ferris, James Michael
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): Raza, Muhammad A
• Application Number:
  US13/037,183
• Publication Number:
  US 20120221684A1
• Time in Patent Office: 1,765 Days
• Field of Search: 709/224, 709/228, 709/217, 709/218
• US Class Current: 1/1
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

Embodiments relate to systems and methods for staged data migration between data sources and a cloud-based storage network. In aspects, a user can maintain a user premise network including servers, clients, local area networks, data stores, and/or other network assets. An administrator or other user of the premise network may wish to extract data from the premise network and transfer that data payload to a set of host storage clouds to leverage cost, redundancy, consolidation, or other cloud advantages. A cloud data delivery (CDD) service can be established to stage the upload of the data payload to the set of host storage clouds using a set of managed or dedicated lines to the CDD service, and to the host storage clouds at network edge. The connections between the premise network/CDD service and CDD service/host storage clouds can be selected and/or optimized to provide high-bandwidth connections tailored to the schedule, cost, or other requirements of the data payload to be transported.

Claims

1. A method comprising:

receiving, by a processor of a data distribution system, an identification of a data payload to be transported from a data source in a user premise network to a set of host storage clouds, wherein the user premise network includes a set of premise servers;

receiving, by the data distribution system from the user premise network, a set of transport parameters to transport the data payload, wherein the set of transport parameters comprises a data transport schedule, a bandwidth capacity requirement, and a cost specification;

identifying a set of dedicated staging channels between the data source and the data distribution system;

managing the transport of the data payload from the data source to the data distribution system in view of the set of transport parameters, wherein managing the transport of the data payload in view of the set of transport parameters comprises selecting channels from the set of dedicated staging channels in view of the set of transport parameters;

initiating the transport of the data payload from the data source in the user premise network to the data distribution system over the selected channels;

in response to initiating the transport, monitoring the transport of the data payload, wherein monitoring the transport of the data payload comprises validating the transported data payload to ensure integrity of the transported data payload;

receiving the data payload by the data distribution system from the data source in the user premise network over the selected channels;

storing the data payload for a time period in a set of staged data stores of the data distribution system, wherein the set of staged data stores comprises servers located within geographic proximity of one of the user premise network or the set of host storage clouds, and wherein during the time period, the data distribution system is to communicate with the user premise network to interrogate the data source for updates to the data payload and to incorporate the updates into the data payload stored in the set of staged data stores of the data distribution system;

applying an access control to govern access to the data payload while the data payload is stored in the set of staged data stores of the data distribution system;

identifying a set of dedicated transport channels between the data distribution system and the set of host storage clouds in view of the set of transport parameters; and

initiating the transport of the data payload from the data distribution system to the set of host storage clouds via the set of dedicated transport channels to store the data payload in a set of local cloud data stores of the set of host storage clouds.

2. The method of claim 1, wherein the data source in the user premise network comprises a set of premise data stores.

3. The method of claim 2, wherein the data payload comprises at least one of a set of selected databases, directories, or files stored in the set of premise data stores.

4. The method of claim 1, wherein the set of transport parameters comprise a cost data associated with using any of the set of dedicated staging channels, and wherein managing in view of the set of transport parameters comprises selecting channels from the set of dedicated transport channels to satisfy the bandwidth capacity specified in the set of transport parameters.

5. The method of claim 4, wherein the set of dedicated transport channels comprises at least one of a wide area network (WAN), a metropolitan area network (MAN), or an optical network connection.

6. The method of claim 4, wherein the set of host storage clouds comprises the set of local cloud data stores, and the data payload is transported from the data distribution system to the set of local cloud data stores via the set of dedicated transport channels.

7. The method of claim 6, wherein the set of dedicated transport channels are located within at least one of geographic proximity or network proximity of the set of local data stores of the set of host storage clouds.

8. The method of claim 4, wherein the set of transport parameters further comprises at least one of a set of data transport initiation conditions, a set of security settings for the data payload, an identification of one or more of the set of staging channels, or an identification of one or more of the set of dedicated transport channels.

9. The method of claim 1, wherein the set of staged data stores are located within at least one of geographic proximity or network proximity of a set of premise data stores.

10. The method of claim 1, further comprising archiving or deleting data corresponding to the data payload after completion of the transport of the data payload from the data distribution system to the set of host storage clouds.

11. The method of claim 1, further comprising registering the data payload to a file storage system associated with the set of host storage clouds after completion of the transport of the data payload from the data distribution system to the set of host storage clouds.

12. The method of claim 11, further comprising providing access to the data payload to at least one of a set of applications or services via the file storage system associated with the set of host storage clouds.

13. A system comprising:

an interface to a set of staged data stores associated with the system; and

a processor, to communicate with the set of staged data stores via the interface, the processor to;

receive an identification of a data payload to be transported from a data source in a user premise network to a set of host storage clouds, wherein the user premise network includes a set of premise servers;

receive, from the user premise network, a set of transport parameters to transport the data payload, wherein the set of transport parameters comprises a data transport schedule, a bandwidth capacity requirement, and a cost specification;

identify a set of dedicated staging channels between the data source and the system;

manage the transport of the data payload from the data source to the system in view of the set of transport parameters, wherein to manage transport in view of the set of transport parameters, the processor is to select channels from the set of dedicated staging channels in view of the set of transport parameters;

initiate the transport of the data payload from the data source in the user premise network to the system over the selected channels;

in response to initiating the transport, monitor the transport of the data payload, wherein to monitor the transport of the data payload, the processor is further to validate the transported data payload to ensure integrity of the transported data payload;

receive the data payload by the system from the data source in the user premise network over the selected channels;

store the data payload for a time period in the set of staged data stores of the system, wherein the set of staged data stores comprises servers located within geographic proximity of one of the user premise network or the set of host storage clouds, and wherein during the time period, the system is to communicate with the user premise network to interrogate the data source for updates to the data payload and to incorporate the updates into the data payload stored in the set of staged data stores of the system;

apply an access control to govern access to the data payload while the data payload is stored in the set of staged data stores of the system;

identify a set of dedicated transport channels between the system and the set of host storage clouds in view of the set of transport parameters; and

initiate the transport of the data payload from the system to the set of host storage clouds via the set of dedicated transport channels to store the data payload in a set of local cloud data stores of the set of host storage clouds.

14. The system of claim 13, wherein the data source in the user premise network comprises a set of premise data stores.

15. The system of claim 14, wherein the set of staged data stores are located within at least one of geographic proximity or network proximity of the set of premise data stores.

16. The system of claim 13,

wherein the set of transport parameters comprises a cost data associated with using any of the set of dedicated staging channels,

and wherein to manage in view of the set of transport parameters, the processor is further to select channels from the set of dedicated transport channels to satisfy the bandwidth capacity specified in the set of transport parameters.

17. The system of claim 16, wherein the set of dedicated transport channels comprises at least one of a wide area network (WAN), a metropolitan area network (MAN), or an optical network connection.

18. The system of claim 16, wherein the set of host storage clouds comprises the set of local cloud data stores, and the data payload is transported from the system to the set of local cloud data stores via the set of dedicated transport channels.

19. The system of claim 18, wherein the set of dedicated transport channels are located within at least one of geographic proximity or network proximity of the set of local data stores of the set of host storage clouds.