Accelerated data transfer using common prior data segments

• US 20080077630A1
• Filed: 09/22/2006
• Published: 03/27/2008
• Est. Priority Date: 09/22/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Keith, Robert O.
• Granted Patent: US 9,317,506 B2
• Time in Patent Office: Days
• US Class Current: 707/204
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24562 Pointer or reference processing operations

Abstract

Accelerating data transfers is described herein. When a second computing system is requested to transfer a file to a first computing system, a data segment is sent to the first computing system instead of the entire file. The data segment is then compared to data stored within a data store on the first computing system. If the data segment and data within the data store match, then the file does not need to be transferred, and a pointer points to the file already located on the first computing system. If the data segment does not match any data stored in the data store, then the file is transferred from the second computing system to the first computing system. By comparing only the data segment instead of sending an entire file, data transfer is able to be greatly expedited in situations where the data is common between systems.

Claims

1. A method of accelerating data transfer comprising:

a. storing data in a data store on a first computing system wherein the data corresponds to one or more files stored on the first computing system;

b. transferring a data segment from a source file from a second computing system to the first computing system over a network;

c. scanning the data store for the data segment;

d. generating one or more pointers to the one or more corresponding files of one or more matching data segments, if the one or more matching data segments are identified in the data store; and

e. transferring a copy of the source file, if the one or more matching data segments are not identified in the data store.

2. The method as claimed in claim 1 wherein the data store is a database.

3. The method as claimed in claim 1 wherein the first computing system is a target system and the second computing system is a source system.

4. The method as claimed in claim 1 wherein the first computing system is a server and the second computing system is a client system.

5. The method as claimed in claim 4 wherein the client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console.

6. The method as claimed in claim 1 wherein the first computing system is a first client system and the second computing system is a second client system.

7. The method as claimed in claim 1 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.

8. The method as claimed in claim 1 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.

9. The method as claimed in claim 1 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.

10. The method as claimed in claim 1 wherein the data store grows as more files are stored on the first computing system.

11. The method as claimed in claim 1 wherein the files stored on the first computing system are minimized by implementing the data store.

12. The method as claimed in claim 1 wherein one or more additional computing systems are coupled to the first computing system.

13. The method as claimed in claim 1 further comprising transferring only a first section of the source file when only a second section of the source file is found within the data store.

14. The method as claimed in claim 1 wherein a standard operating system and file system are utilized on the first computing system and the second computing system.

15. A system for accelerating data transfer comprising:

a. a first computing system for storing one or more files and a data store for storing data corresponding to the one or more files; and

b. a second computing system coupled to the first computing system,

wherein a data segment is compared to the data within the data store on the first computing system after being received from the second computing system, further wherein a pointer to the one or more files is added on the first computing system if the data segment is found within the data store, but a copy of a source file corresponding to the data segment is transferred from the second computing system to the first computing system if the data segment is not found in the data store.

16. The system as claimed in claim 15 wherein the data store is a database.

17. The system as claimed in claim 15 wherein the first computing system is a target system and the second computing system is a source system.

18. The system as claimed in claim 15 wherein the first computing system is a server and the second computing system is a client system.

19. The system as claimed in claim 18 wherein the client system is selected from the group consisting of a personal computer, a PDA, a cell phone, a laptop, a thin client, a Mac computer, an mp3 player and a gaming console.

20. The system as claimed in claim 15 wherein the first computing system is a first client system and the second computing system is a second client system.

21. The system as claimed in claim 15 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.

22. The system as claimed in claim 15 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.

23. The system as claimed in claim 15 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.

24. The system as claimed in claim 15 wherein the data store grows as more files are stored on the first computing system.

25. The system as claimed in claim 15 wherein the files stored on the first computing system are minimized by implementing the data store.

26. The system as claimed in claim 15 further comprising one or more additional computing systems coupled to the first computing system.

27. The system as claimed in claim 15 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.

28. The system as claimed in claim 15 wherein a standard operating system and file system are utilized on the first computing system and the second computing system.

29. The system as claimed in claim 15 further comprising a network coupling the first computing system and the second computing system.

30. A network of systems for accelerating data transfers comprising:

a. one or more source systems for transferring a data segment corresponding to a source file stored on the one or more source systems; and

b. one or more target systems coupled to the one or more source systems for storing data in a data store corresponding to one or more files and for comparing the data segment received from the one or more source systems with the data in the data store where if the data segment is found, a pointer is generated to point to a corresponding file in the one or more files on the target system instead of transferring the source file over a network.

31. The network of systems as claimed in claim 30 wherein the data store is a database.

32. The network of systems as claimed in claim 30 wherein the one or more target systems are one or more servers and the one or more source systems are one or more client systems.

33. The network of systems as claimed in claim 30 wherein the one or more client systems are selected from the group consisting of personal computers, PDAs, cell phones, laptops, thin clients, Mac computers, mp3 players and gaining consoles.

34. The network of systems as claimed in claim 30 wherein the data segment is one or more cyclic redundancy checks and the data in the data store includes cyclic redundancy checks and the data segment and the data are compared.

35. The network of systems as claimed in claim 30 wherein the data segment is a unique database key and the data in the data store includes database keys and the data segment and the data are compared.

36. The network of systems as claimed in claim 30 wherein the data segment is a hash and the data in the data store includes hashes and the data segment and the data are compared.

37. The network of systems as claimed in claim 30 wherein the data store grows as more files are stored on the one or more target systems.

38. The network of systems as claimed in claim 30 wherein the files stored on the one or more target systems are minimized by implementing the data store.

39. The network of systems as claimed in claim 30 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.

40. The network of systems as claimed in claim 30 wherein a standard operating system and file system are utilized on the one or more target systems and the one or more source systems.

41. A storage system configured to receive data from a plurality of computing systems comprising:

a. one or more files;

b. a set of information corresponding to the one or more files; and

c. a data store for storing the set of information,

wherein a data segment received from a source system is compared with the set of information stored within the data store and a pointer is generated to point to a corresponding file in the one or more files if the data segment is found but if the data segment is not found within the data store, a copy of a source file corresponding to the data segment is transferred.

42. The storage system as claimed in claim 41 wherein the data store is a database.

43. The storage system as claimed in claim 41 wherein the data segment is one or more cyclic redundancy checks and the set of information in the data store includes cyclic redundancy checks and the data segment and the set of information are compared.

44. The storage system as claimed in claim 41 wherein the data segment is a unique database key and the set of information in the data store includes database keys and the data segment and the set of information are compared.

45. The storage system as claimed in claim 41 wherein the data segment is a hash and the set of information in the data store includes hashes and the data segment and the set of information are compared.

46. The storage system as claimed in claim 41 wherein the data store grows as more files are stored on the storage system.

47. The storage system as claimed in claim 41 wherein the files stored on the storage system are minimized by implementing the data store.

48. The storage system as claimed in claim 41 wherein only a first section of the source file is transferred when only a second section of the source file is found within the data store.

49. The storage system as claimed in claim 41 wherein a standard operating system and file system are utilized on the storage system.