×

METHOD AND SYSTEM FOR DATA TRANSFER BETWEEN COMPUTE CLUSTERS AND FILE SYSTEM

  • US 20140351300A1
  • Filed: 10/03/2013
  • Published: 11/27/2014
  • Est. Priority Date: 05/24/2013
  • Status: Active Grant
First Claim
Patent Images

1. A method for data migration between data generating entities and a File System, comprising the steps of:

  • (a) operatively coupling a Burst Buffer (BB) tier between at least one data generating entity and a File System, and configuring said BB tier with a plurality of Burst Buffer (BB) nodes, each operating with data input speed exceeding the data input speed of said File System;

    (b) configuring said File System with at least one Burst Buffer Network Aggregator (BBNA) unit;

    (c) sending a first data “

    write”

    request from said at least one data generating entity to said BB tier;

    (d) distributing said data into data fragments, and storing each data fragment and metadata associated therewith in a respective participating BB node from said plurality of BB nodes in said BB tier;

    (e) assigning a primary BB node from said participating BB nodes for servicing said first data “

    write”

    request;

    (f) gathering, by said primary BB node, metadata for said data fragments residing in said participating BB nodes, and generating a metadata list;

    (g) sending a command along with said metadata list from said primary BB node to said at least one BBNA unit;

    (h) subsequently, allocating, by said at least one BBNA unit, a full data stripe buffer sufficient to accommodate said first data “

    write”

    request;

    (i) transferring said data fragments from said participating BB nodes into said full data stripe buffer in accordance with said metadata list, thereby coalescing said data fragments into a full data stripe in said full data stripe buffer; and

    (j) transferring said full data stripe from said full data stripe buffer of said at least one BBNA unit into said File System.

View all claims
  • 3 Assignments
Timeline View
Assignment View
    ×
    ×