Asynchronous namespace maintenance

• US 9,792,344 B2
• Filed: 04/27/2015
• Issued: 10/17/2017
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: DataDirect Networks Incorporated
• Original Assignee: DataDirect Networks Incorporated
• Inventors: Zalpuri, Naveen, Bounour, Joe
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US14/697,224
• Publication Number:
  US 20150227603A1
• Time in Patent Office: 904 Days
• Field of Search: 707614
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Asynchronous namespace maintenance in a distributed replicated data storage system is disclosed. An access device/program serving as a front end to the distributed replicated data storage system updates a batch of updated meta data about stored data items when data items are stored in the distributed replicated data storage system. When the elapsed time since the last batch of data item meta data was stored exceeds a first threshold value or the current batch size exceeds a second threshold value, the access device/program stores the current batch of updated meta data as an object in the distributed replicated data storage system, receiving a batch object identifier for the stored batch of updated meta data, and distributes the batch object identifier to other access devices and/or access programs which retrieve the batch of updated meta data and update their namespaces.

Claims

1. An access device including an access program, the access device coupled with and serving as a front end to a distributed replicated storage system, the access program comprising instructions which when executed cause the access device to perform actions including:

updating a current batch of updated meta data about stored data items when the access device stores data items in the distributed replicated data storage system, the updating including appending meta data about stored data items to an existing batch of updated meta data about earlier stored data items to create the current batch of updated meta data about stored data items;

when an elapsed time since a last batch of data item meta data was stored exceeds a first threshold value or when a current batch size of the current batch of updated meta data exceeds a second threshold value,

storing the current batch of updated meta data as an object in the distributed replicated data storage system, including receiving a batch object identifier for the stored batch of updated meta data,

distributing the batch object identifier to other access devices and/or access programs causing the other access devices and/or access programs to retrieve the batch of updated meta data from the distributed replicated data storage system and update their namespaces.

2. The system of claim 1 wherein the updating comprises:

receiving a plurality of storage requests for data items from clients,

processing the storage requests for the data items including forwarding the storage requests to the distributed replicated data storage system and receiving corresponding stored data item identifiers,

wherein the updated meta data includes the stored data item identifiers.

3. The system of claim 1 wherein the access program includes additional instructions which when executed cause the access device to perform further actions including:

determining whether an elapsed time since a last batch of data item meta data was stored exceeds a first threshold value,

determining whether a current batch size of the current batch of data item meta data exceeds a second threshold value.

4. The system of claim 1 wherein the access devices are server computers and/or gateways.

5. The system of claim 1 wherein the distributing the batch object identifier causes the access programs on the access devices to synchronize the namespaces among the access devices.

6. The system of claim 1 wherein the distributed replicated storage system includes at least two storage zones, each storage zone includes a plurality of storage nodes, and each storage node includes a plurality of storage devices.

7. The system of claim 6 wherein each access device is associated with and acts as a front end to at least one storage zone.

8. The system of claim 6 wherein the storage devices are selected from the group including hard disk drives and solid state storage devices.

9. The system of claim 1 wherein the meta data includes at least three selected from the group including:

object identifier, data item name, data item version, data item type, data item size, data item creation date, data item storage date, data item modification date, data item owner, data item author, data item object kind, flags, parent/predecessor data item identifier, storage policy identifier.