System and Method for Content Addressable Storage

• US 20090228511A1
• Filed: 03/05/2008
• Published: 09/10/2009
• Est. Priority Date: 03/05/2008
• Status: Active Grant

• Current Assignee: IP Wave Pte. Ltd.
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Dubnicki, Cezary, Ungureanu, Cristian, Calkowski, Grzegorz, Atkin, Benjamin
• Granted Patent: US 7,747,663 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats

Abstract

Information, such as files received from a client, etc., is stored in a storage system, such as a content addressable storage system. A file server receives data from a client and chunks the data into blocks of data. The file server also generates metadata for use in forming a data structure. The blocks of data are stored in a block store and a copy of the data blocks and the metadata are locally cached at the file server. A commit server retrieves the metadata. In at least one embodiment, the metadata is retrieved from an update log shared between the file server and the commit server. Based on the retrieved metadata, the commit server generates a version of a data structure. The data structure is then stored at the block store.

Claims

1. A method of storing information in a storage system comprising:

chunking data into one or more blocks of data with a file server;

generating metadata operations of a file system with the file server;

storing the one or more blocks of data in a block store;

retrieving the metadata operations with a commit server;

generating a data structure based on the metadata operations and the one or more blocks of data with the commit server; and

storing the data structure in the block store.

2. The method of claim 1 further comprising:

storing the metadata operations in an update log; and

wherein retrieving the metadata operations with the commit server comprises retrieving the metadata operations from the update log.

3. The method of claim 1 wherein storing the one or more blocks of data in the block store and storing the data structure in the block store occur substantially simultaneously.

4. The method of claim 1 further comprising:

receiving data from a client at the file server; and

wherein chunking data into one or more blocks of data with the file server comprises separating the data received from the client into the one or more blocks of data.

5. The method of claim 2 wherein chunking data into one or more blocks of data and generating metadata operations with the file server further comprises associating a timestamp with the generated metadata operations and comprising:

storing the generated metadata operations at the file server; and

cleaning the metadata stored at the file server with the metadata stored as a version of the data structure based at least in part on the one or more timestamps associated with the retrieved metadata operations.

6. The method of claim 2 wherein generating the data structure comprising the metadata with the commit server comprises:

retrieving one or more timestamps associated with the retrieved metadata operations;

modifying metadata in a metadata cache based on the retrieved metadata operations;

batching the metadata operations; and

storing the metadata as a version of the data structure based at least in part on the one or more timestamps associated with the retrieved metadata.

7. A system for storing data comprising:

a file server configured to chunk data into blocks of data and generate metadata operations;

a memory configured to store the blocks of data;

a commit server configured to retrieve the metadata operations, generate a data structure indicative of the metadata operations, and store the data structure in the memory.

8. The system of claim 7 further comprising:

an update log configured to receive the metadata operations from the file server and pass the metadata operations to the commit server.

9. A method of maintaining a metadata structure at a file server comprising a first superblock having an associated timestamp and one or more metadata objects, each of the one or more metadata objects having an associated timestamp, the method comprising:

receiving a second superblock with an associated timestamp having a higher counter number than a counter number of the first superblock;

designating at least one of the one or more metadata objects as stale based on the timestamp of the second superblock and the timestamps of the one or more dependent metadata objects; and

discarding the one or more metadata objects designated as stale.