Method for layered storage of enterprise data

• US 9,547,453 B2
• Filed: 04/08/2014
• Issued: 01/17/2017
• Est. Priority Date: 04/30/2013
• Status: Expired due to Fees

• Current Assignee: Inuron NV
• Original Assignee: Inuron NV
• Inventors: Glazemakers, Kurt
• Primary Examiner(s): PHILLIPS, III, ALBERT M
• Application Number:
  US14/247,362
• Publication Number:
  US 20140324793A1
• Time in Patent Office: 1,015 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A computer-implemented method for layered storage of enterprise data comprises receiving from one or more virtual machines data blocks; de-duplicating the data blocks per hypervisor; storing de-duplicated data blocks in a local cache memory; time-based grouping the data blocks into data containers; dividing each data container in X fixed length mega-blocks; for each data container applying erasure encoding to the X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being larger than X; and distributed storing the Y fixed length mega-blocks across multiple backend storage systems.

Claims

1. A computer-implemented method for layered storage of enterprise data, said method comprising:

receiving from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

de-duplicating said data blocks per hypervisor serving one or more of said virtual machines;

storing de-duplicated data blocks in a local cache memory of a storage accelerator;

time-based grouping said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 16 megabyte;

dividing each data container in X fixed length mega-blocks, X being a first integer number;

for each data container applying erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed storing said Y fixed length mega-blocks over multiple backend storage systems.

2. A computer-implemented method for layered storage of enterprise data according to claim 1, wherein said local cache memory is flash storage and/or Solid State Drive storage.

3. A computer-implemented method for layered storage of enterprise data according to claim 1, wherein said multiple backend storage systems are remote cloud storage systems operated by different cloud storage operators.

4. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

storing for each de-duplicated data block its address, a hash value, a container number and an offset number as metadata.

5. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

storing each de-duplicated data block that is not yet part of a data container that is remotely stored, in a fail-over cache memory of another storage accelerator.

6. A computer-implemented method for layered storage of enterprise data according to claim 1, wherein said multiple backend storage systems comprise various types of backend storage systems.

7. A computer-implemented method for layered storage of enterprise data according to claim 1, further comprising:

scrubbing said mega-blocks distributed over multiple backend storage systems.

8. A data processing system comprising:

one or more processors; and

one or more computer-readable mediums having stored thereon executable instructions, which, when executed by the one or more processing units, cause the system to perform at least the following for layered storage of enterprise data;

receive from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

de-duplicate said data blocks per hypervisor serving one or more of said virtual machines;

store de-duplicated data blocks in a local cache memory of a storage accelerator;

time-based group said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 16 megabyte;

divide each data container in X fixed length mega-blocks, X being a first integer number;

for each data container apply erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed store said Y fixed length mega-blocks over multiple backend storage systems.

9. One or more non-transitory computer readable storage mediums having instructions stored thereon, which, when executed on a computer comprising one or more processing units, cause the one or more processing units to perform a method for layered storage of enterprise data, the method comprising:

receiving from one or more virtual machines data blocks having a fixed length block size in the range from 4 kilobyte to 64 kilobyte;

de-duplicating said data blocks per hypervisor serving one or more of said virtual machines;

storing de-duplicated data blocks in a local cache memory of a storage accelerator;

time-based grouping said data blocks into data containers having a configurable maximum container size in the range from 1 megabyte to 16 megabyte;

dividing each data container in X fixed length mega-blocks, X being a first integer number;

for each data container applying erasure encoding to said X fixed length mega-blocks to thereby generate Y fixed length mega-blocks with redundant data, Y being a second integer number larger than X; and

distributed storing said Y fixed length mega-blocks over multiple backend storage systems.