×

Systems and methods for reliably storing data using liquid distributed storage

  • US 9,734,007 B2
  • Filed: 12/11/2014
  • Issued: 08/15/2017
  • Est. Priority Date: 07/09/2014
  • Status: Active Grant
First Claim
Patent Images

1. A method for accessing source data of a source object stored as multiple fragments distributed across multiple storage nodes of a storage system, wherein one or more fragments of the multiple fragments includes redundant data for the source object, the method comprising:

  • receiving the source data as a stream of data;

    erasure encoding the stream of data to generate a stream of encoded data as the stream of data is arriving, wherein the redundant data is generated from the source data using an (n ;

    k;

    r) erasure code;

    producing the multiple fragments as a plurality of output fragment streams from the stream of encoded data as the stream of encoded data is being generated;

    writing, by an access server having a processor and memory, the multiple fragments to the storage nodes as each of the plurality of output fragment streams as the output fragment streams are being produced, wherein a first portion of each of the output fragment streams corresponds to a first portion of the source object and are written to the storage nodes before a second portion of the source object has been received, and wherein data of at least one fragment of the multiple fragments is stored by the multiple storage nodes using a data organization that concatenates symbols of multiple source blocks from the source object for inclusion of a symbol of each source block in each of two or more fragments of the multiple fragments of the source object;

    reading, by the access server, data of a plurality of fragments of the multiple fragments from a plurality of storage nodes of the multiple storage nodes to access a requested portion of the source data, wherein the reading data of a plurality of fragments comprises reading each of at least k fragments of the plurality of output fragments written to storage nodes as an input fragment stream; and

    erasure decoding, by the access server, the portion of the source data from the data of the plurality of fragments read from the plurality of storage nodes, wherein the erasure decoding the portion of source data comprises erasure decoding the input fragment stream to generate a stream of source data for the source object as the input fragment stream is being read.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×