SYSTEMS AND METHODS FOR CACHING DATA FILES
First Claim
1. A method for transferring data over a computer network, comprising:
- storing a data file of the type that can be transferred over a computer network;
processing the stored data file to generate content metadata, the processing including;
identifying data blocks within the data file;
grouping the data blocks into one or more segments, starting at an initial block within the data file,running a one-way hash function over incrementing groups of data blocks to generate respective intermediate state hash values; and
recording each respective state hash value and the associated data blocks hashed for that state hash value to create a table of state variables recording intermediate states of the hash operation performed over the data file; and
generating from the recorded state hash values content metadata representative of a unique identifier for the data file; and
transferring the content metadata in response to receiving a request to transfer the data file over the computer network.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods including storage systems that employ local file caching processes and that generate state variables to record, for subsequent use, intermediate states of a file hash process. In certain specific examples, there are systems that interrupt a hash process as it processes the data blocks of a file, and stores the current product of the interrupted hash process as a state variable that represents the hash value generated from the data blocks processed prior to the interruption. After interruption, the hash process continues processing the file data blocks. The stored state variables may be organized into a table that associates the state variables with the range of data blocks that were processed to generate the respective state variable. Such exemplary systems can be used with any type of storage system, including filers, database systems or other storage applications.
-
Citations
20 Claims
-
1. A method for transferring data over a computer network, comprising:
-
storing a data file of the type that can be transferred over a computer network; processing the stored data file to generate content metadata, the processing including; identifying data blocks within the data file; grouping the data blocks into one or more segments, starting at an initial block within the data file, running a one-way hash function over incrementing groups of data blocks to generate respective intermediate state hash values; and recording each respective state hash value and the associated data blocks hashed for that state hash value to create a table of state variables recording intermediate states of the hash operation performed over the data file; and generating from the recorded state hash values content metadata representative of a unique identifier for the data file; and transferring the content metadata in response to receiving a request to transfer the data file over the computer network. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A system for managing data stored on a computer network, comprising:
-
data storage for storing a data file; and a hash processor for selecting data blocks from within the data file; grouping the data blocks into one or more segments, starting at an initial block within the data file, and running a one-way hash function over incrementing groups of data blocks within the segment to generate intermediate state hash values, a state hash variable table having storage to record the intermediate state hash values and associated data blocks hashed for that state hash value. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19)
-
-
20. A method for storing data on a data network using local cache memories, comprising
providing a client having a local cache for storing a copy of a reference data file stored on a remote server and a cache verification processor for generating a request for content metadata to verify accuracy of the copy, and providing a server for receiving the request for the content metadata and having a table of state variables recording intermediate states of a hash operation performed over data blocks of the reference data file, and generating the requested content metadata as a function of a detected change to the stored reference data file and the table of state variables, and including identifying an initial altered data block representative of a first occurrence of an altered data block within a sequence of data blocks making up the reference data file, identifying a state variable preceding a state variable associated with the initial altered data block; - and
computing a new state variable from the preceding state variable and data blocks occurring subsequent to data blocks associated with the preceding state variable, generating the requested content metadata from the new state variable, and at the client comparing the received content metadata against stored content metadata to verify the accuracy of the cache copy.
- and
Specification