Method for layered storage of enterprise data

• US 10,089,009 B2
• Filed: 01/17/2017
• Issued: 10/02/2018
• Est. Priority Date: 04/30/2013
• Status: Active Grant

• Current Assignee: Inuron NV
• Original Assignee: Inuron NV
• Inventors: Glazemakers, Kurt
• Primary Examiner(s): Phillips, III, Albert M
• Application Number:
  US15/407,413
• Publication Number:
  US 20170123675A1
• Time in Patent Office: 623 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H03M 13/154 Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial
• CPC Class Codes: H03M 13/6569 Implementation on processors, e.g. DSPs, or software implementations
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A computer-implemented method for layered storage of enterprise data comprises receiving from one or more virtual machines data blocks; time-based grouping the data blocks into data containers; dividing each data container in X fixed length mega-blocks; for each data container applying erasure encoding to the X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being larger than X; and distributed storing the Y fixed length mega-blocks across one or multiple backend storage systems.

Claims

1. A computer-implemented method with layered storage of enterprise data, said method comprising:

receiving from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

storing said data blocks in a local memory of a storage accelerator that constitutes a first tier storage layer;

time-sequentially grouping said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 256 megabyte;

dividing each data container in X fixed length mega-blocks, X being a first integer number;

for each data container applying erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed storing said Y fixed length mega-blocks over one or multiple backend storage systems that constitute a second tier storage layer.

2. A computer-implemented method for layered storage of enterprise data according to claim 1, wherein said one or multiple backend storage systems are local and/or remote cloud storage systems operated by different cloud storage operators.

3. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

storing for each data block its address, a container number and an offset number as metadata.

4. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

storing each data block that is not yet part of a data container that is remotely stored, in a distributed transaction log of another storage accelerator.

5. A computer-implemented method for layered storage of enterprise data according to claim 1, wherein said one or multiple backend storage systems comprise various types of backend storage systems.

6. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

scrubbing said mega-blocks distributed over multiple backend storage systems.

7. A data processing system comprising:

one or more processing units; and

one or more computer-readable mediums having stored thereon executable instructions, which, when executed by the one or more processing units, cause the system to perform at least the following layered storage of enterprise data;

receive from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

store said data blocks in a local memory of a storage accelerator that constitutes a first tier storage layer;

time-sequentially group said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 256 megabyte;

divide each data container in X fixed length mega-blocks, X being a first integer number;

for each data container applying erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed store said Y fixed length mega-blocks over one or multiple backend storage systems that constitute a second tier storage layer.

8. One or more non-transitory computer readable storage mediums having instructions stored thereon, which, when executed on a computer comprising one or more processing units, cause the one or more processing units to perform a method with layered storage of enterprise data, the method comprising:

receiving from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

storing said data blocks in a local memory of a storage accelerator that constitutes a first tier storage layer;

time-sequentially grouping said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 256 megabyte;

dividing each data container in X fixed length mega-blocks, X being a first integer number;

for each data container applying erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed storing said Y fixed length mega-blocks over one or multiple backend storage systems that constitute a second tier storage layer.