Group based complete and incremental computer file backup system, process and apparatus

• US 8,051,043 B2
• Filed: 05/04/2007
• Issued: 11/01/2011
• Est. Priority Date: 05/05/2006
• Status: Active Grant

• Current Assignee: Hybir Inc.
• Original Assignee: Hybir Inc.
• Inventors: Young, Hadley Rasch
• Primary Examiner(s): Leroux; Etienne
• Assistant Examiner(s): DAYE, CHELCIE L
• Application Number:
  US11/744,741
• Publication Number:
  US 20070266062A1
• Time in Patent Office: 1,642 Days
• Field of Search: 707/204, 707/646, 707/681, 711/162
• US Class Current: 707/646
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 9/3247 involving digital signatures

Abstract

The present invention is capable of determining the rights to a file based on providing a descriptor. The descriptor can be calculated using an algorithm, which may be cryptographic and/or non-cryptographic. The descriptor may further be based on the file contents, metadata of the file, other file data, or any combination thereof to uniquely identify the file in a shared file repository. Since the descriptor is generated based on file data it will be the same regardless of which user generates it. Accordingly, only one copy of the file needs to be maintained in the shared file repository, thereby reducing the amount of network bandwidth required to assure the file is backed up and further reducing the amount of storage required to backup the files. This results in a vastly more efficient method of backup in terms of processing time, network bandwidth, and storage requirements.

Claims

1. A method for managing an electronic data backup system, comprising:

receiving, at a backup server, a first inventory of electronic data stored on a first remote storage medium, wherein the electronic data comprises actual file data to be backed-up from the first remote storage medium, wherein the first inventory comprises at least one descriptor and wherein the at least one descriptor comprises a cryptographic signature computed based on the actual file data, the cryptographic signature uniquely identifying the electronic data to be backed-up;

comparing, at the backup server, the at least one descriptor of the first inventory to a list of descriptors associated with a second inventory of electronic data stored on a backup data storage medium, wherein the second inventory of electronic data comprises shared electronic data from among a plurality of storage mediums including storage mediums different from the first remote storage medium and wherein the at least one descriptor of the first inventory is compared with descriptors in the second inventory that correspond, at least partially, to data stored on the backup data storage medium by storage mediums different from the first remote storage medium;

determining that a first descriptor from the at least one descriptor of the first inventory substantially matches a second descriptor from the list of descriptors associated with the second inventory; and

reporting, from the backup server, to the first remote storage medium that electronic data associated with the first descriptor is already stored in the backup data storage medium.

2. The method of claim 1, wherein the at least one descriptor of the first inventory substantially uniquely identifies at least one of a file and data structure that is on the first remote storage medium and wherein the second descriptor identifies at least one of a file and data structure that was data stored on the backup data storage medium by a storage medium different from the first remote storage medium.

3. The method of claim 1, further comprising removing the electronic data associated with the first descriptor from a list of electronic data that is to be transmitted to the backup data storage medium.

4. The method of claim 3, further comprising transmitting electronic data from the first remote storage medium to the backup data storage medium, wherein the electronic data transmitted comprises electronic data from the first inventory less the electronic data associated with the first descriptor.

5. The method of claim 1, wherein electronic data stored on the backup data storage medium and associated with the second descriptor comprises data originally stored on the backup data storage medium during a backup operation of a second remote storage medium that differs from the first remote storage medium.

6. The method of claim 5, wherein a first user is associated with the first remote storage medium and wherein a second different user is associated with the second remote storage medium.

7. The method of claim 1, wherein a single copy of electronic data associated with the second descriptor is maintained on the backup data storage medium for use by a plurality of different data storage mediums and wherein access to the electronic data associated with the second descriptor is restricted to a data storage mediums that provide the first descriptor to the backup data storage medium.

8. The method of claim 1, wherein the first descriptor and the second descriptor were generated using a common descriptor generating algorithm and common inputs to the common descriptor generating algorithm, and wherein the common inputs comprise both the actual file data and metadata.

9. The method of claim 8, wherein the electronic data associated with the first descriptor comprises at least one of an electronic file or data structure having a length measured in bytes.

10. The method of claim 8, wherein the first descriptor comprises a length of at least 2 bytes and wherein the descriptor generating algorithm comprises at least one of a cryptographic and non-cryptographic algorithm.

11. The method of claim 1, further comprising:

receiving a request from a third remote storage medium to perform a restore operation;

referencing a backup inventory for the third remote storage medium, wherein the backup inventory for the third remote storage medium comprises a list of descriptors associated with electronic data that was resident on the third remote storage medium at a point in time prior to receiving the request to perform a restore operation;

retrieving selected electronic data from the backup data storage medium having a descriptor that matches descriptors in the list of descriptors associated with the third remote storage medium; and

providing the selected electronic data to the third remote storage medium.

12. The method of claim 11, further comprising:

receiving a second inventory from the third remote storage medium, wherein the second inventory for the third remote storage medium comprises a list of descriptors associated with electronic data currently resident on the third remote storage medium; and

removing the electronic data currently on the third remote storage medium from the selected electronic data prior to providing the selected electronic data to the third remote storage medium.

13. The method of claim 11, wherein at least one electronic file from the selected electronic data was originally uploaded to the backup data storage medium by a remote storage medium other than the third remote storage medium.

14. A non-transitory computer readable storage medium comprising processor executable instructions operable to, when executed, perform the method of claim 1.

15. A device for managing an electronic data backup system, comprising:

a backup server including a backup application adapted to receive a first inventory of electronic data stored on a first remote storage medium, wherein the first inventory comprises at least one descriptor generated based on computing a cryptographic signature using actual file data of the electronic data as an input to a cryptographic signature function, compare the at least one descriptor of the first inventory to a list of descriptors associated with a second inventory of electronic data stored on a backup data storage medium, wherein the second inventory of electronic data comprises descriptors identifying electronic data stored by the backup server from a plurality of storage mediums different from the first remote storage medium, determine that a first descriptor from the at least one descriptor of the first inventory substantially matches a second descriptor from the list of descriptors associated with the second inventory, and report to the first remote storage medium that electronic data associated with the first descriptor is already stored in the backup data storage medium.

16. The device of claim 15, wherein the backup application is further adapted to remove the electronic data associated with the first descriptor from a list of electronic data that is to be transmitted to the backup data storage medium.

17. The device of claim 16, wherein the backup application is further adapted to transmit electronic data from the first remote storage medium to the backup data storage medium, wherein the electronic data transmitted comprises electronic data from the first inventory less the electronic data associated with the first descriptor.

18. The device of claim 15, wherein electronic data stored on the backup data storage medium and associated with the second descriptor comprises data originally stored on the backup data storage medium during a backup operation of a second remote storage medium that differs from the first remote storage medium.

19. The device of claim 18, wherein a first user is associated with the first remote storage medium and wherein a second different user is associated with the second remote storage medium.

20. The device of claim 15, wherein a single copy of electronic data associated with the second descriptor is maintained on the backup data storage medium for use by a plurality of different data storage mediums and wherein access to the electronic data associated with the second descriptor is restricted to data storage mediums that provide the first descriptor to the backup data storage medium.

21. The device of claim 15, wherein the first descriptor and the second descriptor were generated using a common descriptor generating algorithm and common inputs to the common descriptor generating algorithm, and wherein the common inputs comprise the actual file data of the electronic data associated with the first descriptor.

22. The device of claim 15, further comprising:

a restore application adapted to receive a request from a third remote storage medium to perform a restore operation, reference an backup inventory for the third remote storage medium, wherein the backup inventory for the third remote storage medium comprises a list of descriptors associated with electronic data that was resident on the third remote storage medium at a point in time prior to receiving the request to perform a restore operation, retrieve selected electronic data from the backup data storage medium having a descriptor that matches descriptors in the list of descriptors associated with the third remote storage medium, and provide the selected electronic data to the third remote storage medium.

23. The device of claim 22, wherein the restore application is further adapted to receive a second inventory from the third remote storage medium, wherein the second inventory for the third remote storage medium comprises a list of descriptors associated with electronic data currently resident on the third remote storage medium, and remove the electronic data currently on the third remote storage medium from the selected electronic data prior to providing the selected electronic data to the third remote storage medium.

24. The device of claim 22, wherein at least one electronic file from the selected electronic data was originally uploaded to the backup data storage medium by a remote storage medium other than the third remote storage medium.

34. The method of claim 22, wherein at least one descriptor in the list of descriptors associated with electronic data currently resident on the remote storage medium is computed without the metadata.

25. An electronic data backup system, comprising:

means for storing a first set of sharable electronic data for a plurality of remote storage mediums at least two of which are associated with different users;

means for receiving a request to backup a second set of electronic data from a first remote storage medium;

means for determining that at least a first electronic file having actual file data is in the first set of electronic data and the second set of electronic data by comparing descriptors of the at least a first electronic file in both the first set of electronic data and all descriptors in the second set of electronic data, wherein the descriptors comprise a cryptographic signature computed based on the actual file data of the at least first a electronic file and wherein the descriptors uniquely identify the actual file data;

means for notifying the first remote storage medium that the at least first electronic file already resides in the first set of electronic data because it was backed up there by a second remote storage medium that is different from the first remote storage medium;

means for transferring, from the first remote storage medium to the means for storing a first set of electronic data, electronic files associated with the second set of electronic data less the at least first electronic file; and

means for adding the electronic files associated with the second set of electronic data less the at least first electronic file to the means for storing the first set of electronic data.

26. The system of claim 25, wherein the at least a first electronic file was transferred to the means for storing the first set of electronic data by a second remote storage medium that is different than the first remote storage medium.

27. The system of claim 25, further comprising a means for allowing a subset of the plurality of remote storage mediums to access electronic files associated with the second set of electronic data, wherein the subset of the plurality of remote storage mediums provide descriptors for the electronic files associated with the second set of electronic data prior to gaining access.

28. The system of claim 27, wherein the descriptors for the electronic files associated with the second set of electronic data substantially uniquely identify the electronic files.

29. The system of claim 27, wherein the descriptors comprise an alphanumeric string of data generated using a predetermined descriptor generating algorithm that utilizes, as inputs, the data as well as the metadata from the electronic files.

30. A method, comprising:

receiving a request from a remote storage medium to perform a restore operation;

referencing a backup inventory for the remote storage medium, wherein the backup inventory for the remote storage medium comprises a list of descriptors associated with electronic data that was resident on the remote storage medium at a point in time prior to receiving the request to perform a restore operation, wherein the electronic data comprises actual file data, and wherein a descriptor in the list of descriptors comprises a cryptographic signature computed with the actual file data such that the actual file data is uniquely identified by the descriptor;

retrieving selected electronic data from the backup data storage medium having a descriptor that matches descriptors in the list of descriptors associated with the remote storage medium; and

providing the selected electronic data to the remote storage medium.

31. The method of claim 30, further comprising:

receiving a second inventory from the remote storage medium, wherein the second inventory for the remote storage medium comprises a list of descriptors associated with electronic data currently resident on the remote storage medium; and

removing the electronic data corresponding to descriptors included in the second inventory from the selected electronic data prior to providing the selected electronic data to the remote storage medium.

32. The method of claim 30, wherein at least one electronic file from the selected electronic data was originally uploaded to the backup data storage medium by a second remote storage medium that is different than the remote storage medium.

33. The method of claim 32, wherein only one copy of the at least one electronic file is stored on the backup data storage medium and is made accessible to both the remote storage medium and the second remote storage medium.

35. A device for managing an electronic data backup system, comprising:

a non-transitory data storage medium configured to store instructions and a processor configured to execute the instructions stored in the data storage medium, the instructions stored in the data storage medium including;

a restore application adapted to receive a request from a remote storage medium to perform a restore operation, reference a backup inventory for the remote storage medium, wherein the backup inventory for the remote storage medium comprises a list of descriptors associated with electronic data that was resident on the remote storage medium at a point in time prior to receiving the request to perform a restore operation, wherein the electronic data comprises actual file data, and wherein a descriptor in the list of descriptors comprises a cryptographic signature computed with the actual file data thereby uniquely identifying the actual file data, the restore application further adapted to retrieve selected electronic data from the backup data storage medium having a descriptor that matches descriptors in the list of descriptors associated with the remote storage medium and provide the selected electronic data to the remote storage medium.

36. The device of claim 35, wherein the restore application is further adapted to receive a second inventory from the remote storage medium, wherein the second inventory for the remote storage medium comprises a list of descriptors associated with electronic data currently resident on the remote storage medium, and remove the electronic data currently on the remote storage medium from the selected electronic data prior to providing the selected electronic data to the remote storage medium.

37. The device of claim 35, wherein at least one electronic file from the selected electronic data was originally uploaded to the backup data storage medium by a second remote storage medium that is different from the remote storage medium which transmitted the request to perform a restore operation.

38. The device of claim 35, wherein the cryptographic signature is also computed with the metadata.

39. The device of claim 35, wherein the cryptographic signature is computed without the metadata.

40. A method, comprising:

scanning a list of files stored in memory of a client device;

identifying a first set of files in the list of files as one or more of new and changed since a last backup operation was performed;

for each file in the first set of files, generating, by the client device, a cryptographic signature for the file, wherein the cryptographic signature is computed by using actual data of the file as an input to a cryptographic signature generating algorithm which computes the cryptographic signature value;

generating, at the client device, a first inventory including the cryptographic signatures calculated for the first set of files;

transmitting the first inventory from the client device to a backup server;

receiving, at the backup server, the first inventory including the cryptographic signatures calculated for the first set of files;

extracting, by the backup server, the cryptographic signatures from the first inventory;

comparing, by the backup server, the cryptographic signatures to a list containing all cryptographic signatures identifying raw data currently stored in a backup system managed by the backup server regardless of whether the raw data currently stored in the backup system was stored there by the client device or by a second client device other than the client device which generated the first inventory;

based on the comparison step, generating a second inventory, the second inventory including the cryptographic signatures found in the first inventory and not found in the list containing all cryptographic signatures identifying actual data currently stored in the backup system;

transmitting the second inventory from the backup server to the client device;

receiving, at the client device, the second inventory;

transmitting, from the client device to the backup server, every file in the list of files having a cryptographic signature corresponding to a cryptographic signature found in the second inventory; and

causing the files transmitted from the client device to the backup server to be stored in the backup system.