×

Map-reduce ready distributed file system

  • US 9,323,775 B2
  • Filed: 06/16/2011
  • Issued: 04/26/2016
  • Est. Priority Date: 06/19/2010
  • Status: Active Grant
First Claim
Patent Images

1. A map-reduce compatible distributed file system, comprising:

  • a plurality of containers in which each container stores file and directory meta-data as well as file content data;

    wherein references to file content data are stored on a subset of nodes on which container meta-data and data are stored; and

    wherein container data and meta-data are arranged to allow a topological sort to imply update order;

    a container location database (CLDB) configured to maintain information about where each of said plurality of containers is located;

    a plurality of cluster nodes, each cluster node containing one or more storage pools, each storage pool containing zero or more containers; and

    a plurality of inodes for structuring data within said containers;

    wherein said CLDB is configured to assign nodes as replicas of data in a container to meet policy constraints in accordance with any of the following;

    said CLDB assigns each container a master node that controls all transactions for that container;

    said CLDB designates a chain of nodes to hold replicas;

    when one of the replicas goes down or is separated from the master CLDB node, it is removed from the replication chain;

    when the master goes down or is separated, a new master is designated;

    any node that comes back after having been removed from the replication chain is reinserted at the end of the replication chain when the chain still needs another replication chain when the node returns;

    when the node returns within a first predetermined interval, no new node to replicate the container in question has been designated and the chain still needs a replication chain; and

    when the node has been gone for a second, longer predetermined interval, the CLDB may designate some other node to take a place in the chain.

View all claims
  • 7 Assignments
Timeline View
Assignment View
    ×
    ×