File state subset satellites to provide block-based version control

• US 8,874,534 B2
• Filed: 08/28/2012
• Issued: 10/28/2014
• Est. Priority Date: 08/14/2009
• Status: Active Grant

• Current Assignee: IC Manage, Inc.
• Original Assignee: IC Manage, Inc.
• Inventors: March, Roger, Sikand, Shivinder Singh
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US13/596,049
• Publication Number:
  US 20120323869A1
• Time in Patent Office: 791 Days
• Field of Search: 707/681, 707/695, 709/212, 711/163
• US Class Current: 707/695
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files

Abstract

A coherent file system and method of operation for a file state authority coupled to file state sub-set satellites addresses congestion and latency effects on a plurality of peer workstation clients organized in neighborhoods. Very large files are versioned and metadata recorded in a file state view determines which file blocks make up each version of a committed file. Metadata may be requested from neighboring workstations to locate desired file blocks. File block transmission is minimized to fulfill read requests only when not accessible at a workstation'"'"'s local file block store.

Claims

1. A system comprising a first peer workstation client apparatus communicatively coupled to at least one other peer workstation client apparatus, each workstation client apparatus comprising:

a processor configured by computer executable instructions to store, retrieve, and transform data and to track versions of files which are made up of file blocks and to intercept file block read and file block write commands issued by a program product;

a file state view circuit configured to create, store, and retrieve metadata about file blocks, the versions of files that each block is a component of, and the location of each block in a local file block store whereby file blocks that are identical in multiple versions of a file are not redundantly stored;

the local file block store for all file blocks that comprise versions of files whose metadata is stored in the file state view circuit, coupled to both the processor and to the file state view circuit;

and

at least one peer file system view circuit, whereby each peer file system view circuit stores and provides metadata previously received from other peer workstation client apparatuses

about each file block previously stored in the file block store of each other peer, the version of the files each file block was previously a component of, and all the versions of all the files which were previously accessible to the other peer workstation client apparatus, and

the location of each file block in the local file block store of the peer workstation client apparatus.

2. The system of claim 1 further comprising:

a first file sub-set satellite apparatus communicatively coupled to a plurality of workstation client apparatuses,

the file sub-set satellite apparatus comprising a store for peer workstation file system view metadata about each file block, the versions of files it is part of, and the location of the file block in a local store of a peer workstation client apparatus.

3. The system of claim 2 further comprising:

at least one other sub-set satellite apparatus communicatively coupled to said first sub-set satellite apparatus;

wherein said other sub-set satellite apparatus is communicatively coupled to a second plurality of workstation client apparatuses without shared membership in said first plurality of workstation client apparatuses.

4. The coherent file system of claim 3 further comprising:

a coherent file state authority apparatus communicatively coupled to a plurality of file state sub-set satellite apparatuses through a network, the coherent file state authority apparatus further coupled to a conventional backup server, and the coherent file state authority apparatus comprising logic circuits and storage to archive a plurality of peer file state views which mirror the file state views of all peer workstation clients in communication with any file state sub-set satellite.

5. A method of operation across a plurality of peer workstation client apparatuses in a coherent file system to minimize network traffic, the method comprising:

at a first peer workstation client apparatus,

intercepting a file block read command from an application configuring the first peer workstation client apparatus;

requesting metadata from a file state view circuit configured to create, store, and retrieve metadata about file blocks, the version(s) of file(s) that each file block is a component of, and the location of each file block;

on the condition that the requested file block is stored in a local file block store,

accessing the local file block store for a file block that comprise a version of a file whose metadata is stored in the file state view circuit, and

fulfilling the file block read without network traffic;

on the condition that the requested file block is not stored in the local file block store,

requesting metadata from a peer file state view circuit about file blocks, the versions of files that each file block is a component of, and the location of each file block at a second local file block store at a second peer workstation client apparatus;

on the condition that the requested file block is stored in a second local file block store at a second peer workstation client apparatus,

transmitting a request over a network,

fulfilling the file block read with the response,

storing the response to the first local file block store and

updating the first file state view;

on the condition that the requested version of the file block is not found in either the first file state view or the first peer file system view,

transmitting a request to at least one other peer workstation client apparatus for metadata about a directory or sub-directory of files and

updating the first peer file system view with the response, whereby metadata about file blocks, their relation to file versions, and their locations is transmitted from a second peer workstation client apparatus only upon request from a first peer workstation client apparatus.

6. The method of claim 5 further comprising:

at a second peer workstation client apparatus communicatively coupled to a first peer workstation client apparatus,

receiving a request for metadata about a version of a file stored at the second peer workstation client apparatus,

retrieving from a second file state view the requested metadata about all the file blocks of all the versions of a file stored at the second peer workstation client apparatus, and

transmitting to the first peer workstation client apparatus the requested metadata about all the file blocks stored at the second peer workstation apparatus, the file version or versions to which file block relates, and each file block'"'"'s location, whereby network traffic and congestion is minimized by providing metadata about file blocks and versions only upon request.

7. The method of claim 6 further comprising:

updating both the file system view and the peer file system view of the first peer workstation client apparatus and the second peer workstation client apparatus whereby they are mutually coherent in file system state wherein file system state is the metadata describing the version(s) of each file, the file blocks that make up each version, and the location(s) of each file block.

8. A method for operating a first file state sub-set satellite apparatus communicatively coupled to a second file sub-set satellite apparatus, the method comprising:

receiving a request for metadata about a directory or sub-directory of files,

reading from a plurality of stored peer file system views the metadata which identifies the location of each file block that makes up each version of the requested directory or sub-directory of files,

transmitting the requested metadata to the second file sub-set satellite apparatus,

requesting from the second file sub-set satellite apparatus metadata about another directory or sub-directory of files, and

updating the peer file system views stored at the first file sub-set satellite apparatus.

9. The method for operating a first file sub-set satellite apparatus communicatively coupled to a workstation client apparatus of claim 8 further comprising:

receiving a request for metadata about a directory or sub-directory of files,

reading from a plurality of stored peer file system views the metadata which identifies the location of each file block that makes up each version of the requested directory or sub-directory of files which it received from another file state sub-set apparatus, if not found, requesting from another file sub-set satellite apparatus metadata about said directory or sub-directory of files, and transmitting the requested metadata and location of the requested file block to the requesting workstation client apparatus.

10. The method for operating the file state sub-set satellite apparatus of claim 8 further comprising:

establishing communication through a network with a coherent file state authority,

reading a plurality of peer file system views from storage,

transmitting metadata about files stored at a plurality of peer workstation clients communicatively coupled to the file state sub-set satellite to a coherent file state authority, and

uploading file blocks required to fulfill the metadata to a conventional backup server accessible by the coherent file state authority.

11. The method for operating a coherent file system of claim 10, the method further comprising:

receiving from a file state sub-set satellite metadata about a plurality file blocks which make up a version of a file at one of the plurality of workstation clients, transforming the metadata to integrate a coherent file state view for every version of said file at every workstation client;

storing said coherent file state view into a coherent file system view of every version of every file at any workstation client, and requesting a file block referenced by the metadata to be stored in conventional backup server storage.

12. The method for operation of a coherent file system of claim 11 further comprising:

at the coherent file state authority apparatus,

reading metadata about a directory or sub-directory of files from file system view store,

transmitting said metadata about each file block of each version of each file to a file state sub-set satellite apparatus, whereby a conventional backup server reads upon request each file block necessary to fulfill any request for a file block of the file state sub-set metadata transmitted to the file state sub-set satellite, and transmits the requested file blocks to the file state sub-set satellite.

13. A method for operation of a first file state sub-set satellite apparatus communicatively coupled to a plurality of peer workstation clients, the method comprising:

passively monitoring data communications between a first peer workstation client and a second peer workstation client;

recording peer file system view metadata as it is transmitted among peer workstation clients;

storing into non-transitory media a plurality of peer file system view metadata on file blocks, the versions of files the file blocks are part of, and the location of each file block in the local store of each peer workstation client; and

responding to a request from a second file state sub-set satellite apparatus for metadata about file blocks, file versions, and locations of file blocks in a directory or sub-directory requested by one of the peer workstation clients communicatively coupled to the second file state sub-set satellite apparatus.

14. The method of claim 13 further comprising steps for operation of a file state sub-set satellite apparatus communicatively coupled to a coherent file state authority apparatus, the method comprising:

reading at least one stored peer file system views which it has received from a peer workstation client apparatus;

transmitting to a coherent file state authority apparatus up-to-date metadata with respect to file block of certain versions of files and their locations at certain workstation clients; and

at said coherent file state authority apparatus,

integrating, transforming, and maintaining a unified file system state of all file blocks of the versions of all of the files and their location at all of the peer workstation clients.