×

DATA PROCESSING AND DATA MOVEMENT IN CLOUD COMPUTING ENVIRONMENT

  • US 20190007495A1
  • Filed: 09/07/2018
  • Published: 01/03/2019
  • Est. Priority Date: 05/18/2016
  • Status: Active Grant
First Claim
Patent Images

1. A method for moving data from a source site to a target site in a cloud computing platform, comprising:

  • receiving a plurality of data sets to be moved from the source site to the target site at a plurality of containerized data ingest components located at the source site;

    providing the received plurality of data sets from the plurality of data ingest components to a staging cluster comprising a plurality of containerized broker components located at the source site, wherein the plurality of containerized broker components queue the plurality of data sets, wherein the staging cluster replicates one or more partitions of the received data set between broker components, and wherein each broker component performs a data deduplication operation;

    providing the queued plurality of data sets from the plurality of containerized broker components to a processing cluster comprising a plurality of containerized data processing components, wherein the plurality of containerized data processing components process the plurality of data sets, wherein the processing stage performs one or more of data encryption, data reduction, and data indexing prior to a data set being transmitted to the target site;

    transmitting the plurality of data sets from the plurality of containerized data processing components to the target site;

    wherein, for each data ingest component of the plurality of data ingest components, a respective pipeline is formed through the staging cluster and the processing cluster, and wherein the staging cluster and the processing cluster are scalable such that the method further comprises;

    adding an additional pipeline comprising a given containerized broker component in the staging cluster and a given containerized data processing component in the processing cluster when a data ingest component is added; and

    removing an existing pipeline comprising a given containerized broker component in the staging cluster and a given containerized data processing component in the processing cluster when an existing data ingest component is removed;

    wherein the source site and the target site are implemented via one or more processing devices operatively coupled via a communication network.

View all claims
  • 7 Assignments
Timeline View
Assignment View
    ×
    ×