Reducing communication for reads and updates in distributed object systems

• US 20040162885A1
• Filed: 02/18/2003
• Published: 08/19/2004
• Est. Priority Date: 02/18/2003
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Garg, Sharad K., Anderson, Todd A.
• Granted Patent: US 7,266,645 B2
• Time in Patent Office: Days
• US Class Current: 709/213
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication

Abstract

In a distributed system, a server may distribute a primary object copy having one or more chunks of data associated with a primary per-chunk metadata copy to a client over a network. The client may selectively fetch and update the one or more chunks of data of the primary object copy based on the primary per-chunk metadata copy. For example, by determining at a client whether a portion of the data in a cached data object, such as a file, is changed relative to a copy of a data object for the cached data object at a server, less than all of the data of the cached data object may be replaced with a corresponding data in the copy of the data object.

Claims

1. A method comprising:

determining whether a portion of data in a cached data object is changed; and

if so, replacing less than all of data of the cached data object with a corresponding data in a copy of a data object.

2. The method of claim 1, wherein replacing the changed portion of data comprising:

reconfigurably dividing the copy of the data object into at least two data chunks; and

associating per-chunk meta-information with said at least two data chunks for selective distribution of the cached data object.

3. The method of claim 1, wherein replacing the changed portion of data comprising:

providing a replica of the copy of the data object to a replica site.

4. The method of claim 3 comprising:

determining at a server whether the copy of the data object is changed relative to the replica of the copy of the data object; and

if so, sending the changed portion of data of the copy of the data object to the replica site.

5. The method of claim 1 comprising:

determining at a client whether the portion less than all of data in the cached data object is changed relative to the copy of the data object for the cached data object at a server.

6. The method of claim 5 comprising:

dividing the copy of the data object into at least two data chunks at the server;

associating per-chunk meta-information with said at least two data chunks for distribution thereof over a network in a distributed object system to the client; and

using the per-chunk meta-information at the client to selectively replace the changed portion of data of the copy of the data object at the server.

7. The method of claim 1 comprising:

associating per-chunk meta-information with the copy of the data object at a granularity less than the data object; and

using inter-machine communication across a client and a server in a distributed object system for selective distribution of the cached data object based on the per-chunk meta-information associated with the copy of the data object.

8. The method of claim 7, comprising:

identifying at the client a change in the per-chunk meta-information of the cached data object, wherein the per-chunk meta-information including metadata.

9. The method of claim 7 comprising:

dividing the copy of the data object into at least two data chunks of a fixed size being independent of the client.

10. The method of claim 9 comprising:

dynamically changing the fixed size of said at least two data chunks of the copy of the data object based on the per-chunk meta-information associated therewith.

11. The method of claim 9 comprising:

caching said at least two data chunks into a cache at the client;

identifying at least one data chunk of said at least two data chunks for which the per-chunk meta-information is changed; and

discarding only said at least one data chunk from the cache.

12. The method of claim 11 comprising:

storing a version of the copy of the data object at the server and a copy of the version of the data object at a replica site; and

reading said at least one data chunk in the cache from the version of the copy of the data object stored at the server.

13. The method of claim 12 comprising:

copying to the replica site said at least one data chunk to replace a corresponding data chunk within the copy of the version of the data object therein.

14. The method of claim 9 comprising:

computing a hash value for each of said at least two data chunks of the data object at the server; and

storing the computed hash value for said each of said at least two data chunks in the metadata associated with the copy of the data object.

15. The method of claim 14, wherein upon a request to the server, receiving the metadata associated with the copy of the data object at the client.

16. The method of claim 14 comprising:

calculating a new hash value for said at least one data chunk data of said at least two data chunks to update the copy of the data object for the client; and

upon a request from the client, receiving the metadata associated with the copy of the data object from the server.

17. The method of claim 16 comprising:

comparing the new hash value of said at least one data chunk with the computed hash value from the corresponding metadata of the cached data object at the client; and

based on the comparison, determining whether the new hash value is different than the computed hash value.

18. The method of claim 17 comprising:

if different, sending said at least one data chunk to the server for update thereof; and

otherwise, indicating to the server that said at least one data chunk is unchanged.

19. The method of claim 18, wherein upon receiving said at least one data chunk from the client, updating the corresponding data chunk and the corresponding metadata of the copy of the data object at the server.

20. The method of claim 19, further comprising:

collaborating between the client and the server to size said at least two data chunks of the copy of the data object.

21. An apparatus comprising:

a processor;

a dynamic random access memory, coupled to said processor, storing metadata associated with a file data object at a granularity less than the file data object for selective distribution thereof, and a client engine to use the metadata to determine whether a portion of data in a cached data object is changed, and if so, to replace less than all of data of the cached data object with a corresponding data in a copy of the file data object; and

a network controller coupled to said processor to communicate the file data object over a network.

22. The apparatus of claim 21 further comprising:

a file system having a cache to selectively retain one or more chunks of data for the file data object based on a change in the metadata associated therewith.

23. The apparatus of claim 22, said client engine comprising:

a read engine that selectively reads the one or more chunks of data in said cache based on the metadata of a previous version of the file data object.

24. The apparatus of claim 23, said client engine further comprising:

an update engine that updates the file data object based on the change in the metadata thereof rather than updating the previous version of the file data object in its entirety.

25. An article comprising a medium storing instructions that, if executed, enable a processor-based system to:

determine whether of data in a cached data object is changed; and

if so, replace less than all of data of the cached data object with a corresponding data in a copy of a data object.

26. The article of claim 25, comprising a medium storing instructions that, if executed, enable a processor-based system to:

reconfigurably divide the copy of the data object into at least two data chunks; and

associate per-chunk meta-information with said at least two data chunks for selective distribution of the cached data object.

27. The article of claim 26, comprising a medium storing instructions that, if executed, enable a processor-based system to:

provide a replica of the copy of the data object to a replica site.

28. The article of claim 27, comprising a medium storing instructions that, if executed, enable a processor-based system to:

determine at a server whether the copy of the data object is changed relative to the replica of the copy of the data object; and

if so, send the changed portion of data of the copy of the data object to the replica site.

29. The article of claim 27, comprising a medium storing instructions that, if executed, enable a processor-based system to:

determine at a client whether the portion less than all of data in the cached data object is changed relative to the copy of the data object for the cached data object at a server.

30. The article of claim 29, comprising a medium storing instructions that, if executed, enable a processor-based system to:

divide the copy of the data object into at least two data chunks at the server;

associate per-chunk meta-information with said at least two data chunks for distribution thereof over a network in a distributed object system to the client; and

use the per-chunk meta-information at the client to selectively replace the changed portion of data of the copy of the data object at the server.