DISTRIBUTED DATA STORE

• US 20140181041A1
• Filed: 03/16/2013
• Published: 06/26/2014
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Arcserve LLC
• Original Assignee: Zetta Incorporated
• Inventors: Whitehead, Jeffrey, Harrison, Jason, Eckhardt, Andrew, Montulli, Louis V, Cook, Casey, Thro, Ray
• Granted Patent: US 9,152,643 B2
• Time in Patent Office: Days
• US Class Current: 707/652
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/2315 Optimistic concurrency control
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/86 Mapping to a database
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods are disclosed for backing up a computer with a distributed meta data store by splitting meta data from file data; storing the meta data for each file in a separate file; mapping the file data and meta data to an arbitrary file system; and reconstructing files for the arbitrary file system.

Claims

1. A method for backing up a computer with a distributed meta data store, comprising:

splitting meta data from file data;

storing the meta data for each file in a separate file;

mapping the file data and meta data to an arbitrary file system; and

reconstructing files for the arbitrary file system.

2. The method of claim 1, comprising customizing a Web-based Distributed Authoring and Versioning application program interface to access the files.

3. The method of claim 1, comprising overlaying access to every file to query, set, or override the meta data.

4. The method of claim 1, comprising creating a metadata database as a shadow of a user file in a hidden folder.

5. The method of claim 4, comprising applying the meta data database for a random access item lookup.

6. The method of claim 1, storing meta data in a database with optimistic locking.

7. The method of claim 1, comprising storing meta data in a single index database with a meta data database file size below the size of a single block.

8. The method of claim 1, comprising translating between an original native file system and a split file and metadata system.

9. The method of claim 1, storing original native file system meta-data in an analogous file system meta-data field on the target with an escape value causing an indirection to the separate meta-data file when the value cannot be represented.

10. The method of claim 1, comprising using a software client as a translation agent

11. The method of claim 1, comprising reading each file data and meta data and sending both through Web-based Distributed Authoring and Versioning application program interface.

12. The method of claim 1, comprising storing file data and meta data separately on a file system, and linking both through folder proximity and file name.

13. The method of claim 1, comprising storing as metadata file modification time, file creation time, file access time, file size, access control list (ACLs), permission list, hash of file data.

14. The method of claim 1, comprising storing metadata in sidecar files for any type of attribute.

15. The method of claim 1, comprising transferring back-up data files using a client software on the computer, wherein the client software receives inputs from a web user portal to control operations of the client software and a system management platform to back up the customer computer with one or more data storage silos, each including a plurality of storage directors communicating with the client software;

and a clustered data storage array.

16. The method of claim 1, comprising restoring files through a group consisting of:

the client software, a web based file browser replicating a look and feel of a native file explorer, and a remote data drive at the data storage silo mounted by a user simulating a local drive.

18. The system of claim 15, wherein the data storage array comprises Redundant Array of Independent Nodes (RAIN) storage nodes that ensures that even if two storage nodes go down that data is still available.

19. The system of claim 15, wherein the client software communicates using Web-based Distributed Authoring and Versioning to collaborative updating and management of files on remote web servers and wherein the client software selects the number of threads, and wherein the client software comprises a configurable parallelism selection for multi-processor efficiency and high latency connections, or for backing up NAS/SAN or RAID arrays with multiple disks.

20. The system of claim 15, wherein the data storage silos comprises a pair of head nodes and a plurality of storage nodes, wherein each storage node comprises a computer with disks and an on-disk encryptor to generate clear text and sent over an Internet Small Computer System Interface (iSCSI).

21. The system of claim 18, wherein iSCSI traffic is load balanced and fails over network links and connected to a plurality of switches.

22. The system of claim 18, wherein iSCSI data comprise whole disk mappings and wherein the head nodes create RAID stripes across the storage nodes.

17. A system, comprising:

a plurality of customer computers to be backed-up, each customer computer running a client software to transfer back-up data files with a distributed meta data store by splitting meta data from file data, storing the meta data for each file in a separate file, mapping the—

file data and meta data to an arbitrary file system, and reconstructing files for the arbitrary file system;

a system management platform coupled to the client software over the Internet, the system management platform receiving inputs from a web user portal to control operations of the client software and the system management platform to back up the customer computer; and

two or more data storage silos, each including;

a plurality of storage directors communicating with the client software; and

a clustered data storage array.