GROUP BASED COMPLETE AND INCREMENTAL COMPUTER FILE BACKUP SYSTEM, PROCESS AND APPARATUS

• US 20150220747A1
• Filed: 04/17/2015
• Published: 08/06/2015
• Est. Priority Date: 05/05/2006
• Status: Active Grant

• Current Assignee: Hybir Inc.
• Original Assignee: Hybir Inc.
• Inventors: Young, Hadley Rasch
• Granted Patent: US 9,679,146 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 9/3247 involving digital signatures

Abstract

The present invention is capable of determining the rights to a file based on providing a descriptor. The descriptor can be calculated using an algorithm, which may be cryptographic and/or non-cryptographic. The descriptor may further be based on the file contents, metadata of the file, other file data, or any combination thereof to uniquely identify the file in a shared file repository. Since the descriptor is generated based on file data it will be the same regardless of which user generates it. Accordingly, only one copy of the file needs to be maintained in the shared file repository, thereby reducing the amount of network bandwidth required to assure the file is backed up and further reducing the amount of storage required to backup the files. This results in a vastly more efficient method of backup in terms of processing time, network bandwidth, and storage requirements.

Claims

1. A method of synchronizing electronic data stored on a first client device with at least a second client device, the method comprising:

receiving, at a backup server, a first inventory of electronic data stored on a first remote storage medium associated with a first client device, wherein the electronic data comprises actual file data to be backed-up from the first remote storage medium, wherein the first inventory comprises at least one descriptor and wherein the at least one descriptor comprises a cryptographic signature computed based on the actual file data, the cryptographic signature uniquely identifying the electronic data to be backed-up;

comparing, at the backup server, the at least one descriptor of the first inventory to a list of descriptors associated with a second inventory of electronic data stored on a backup data storage medium, wherein the second inventory of electronic data comprises shared electronic data from a second remote storage medium associated with the at least a second client device as well a plurality of other storage mediums different from the first and second remote storage mediums, and wherein the at least one descriptor of the first inventory is compared with descriptors in the second inventory that correspond, at least partially, to data stored on the backup data storage medium by storage mediums different from the first remote storage medium;

determining that a first descriptor from the at least one descriptor of the first inventory does not match a second descriptor from the list of descriptors associated with the second inventory;

storing, on the backup data storage medium, the electronic data that is uniquely identified by the cryptographic signature;

determining that the second remote storage medium does not have the electronic data that is uniquely identified by the cryptographic signature stored therein;

determining that the second remote storage medium is permitted to have the electronic data that is uniquely identified by the cryptographic signature; and

providing the electronic data that is uniquely identified by the cryptographic signature from the backup server to the at least a second client device so that the electronic data that is uniquely identified by the cryptographic signature can be stored on the second remote storage medium.

2. The method of claim 1, wherein the step of determining that the second remote storage medium does not have the electronic data that is uniquely identified by the cryptographic signature stored therein is initiated in response to receiving a request from the at least a second client device.

3. The method of claim 1, wherein the electronic data that is uniquely identified by the cryptographic signature is provided from the backup server to the at least a second client device via a distributed communication network.

4. The method of claim 3, wherein the distributed communication network comprises the Internet.

5. The method of claim 3, wherein the electronic data that is uniquely identified by the cryptographic signature is encrypted prior to being transmitted across the distributed communication network.

6. The method of claim 1, wherein the first client device and the at least a second client device are associated with a common user.

7. The method of claim 6, wherein a third client device comprises a third remote storage medium that is included in the plurality of other storage mediums, wherein the third client device is associated with a user that is different from the common user.

8. A method, comprising:

scanning a list of filed stored in memory of a client device;

identifying a first set of files in the list of files as one or more of new and changed since a last backup operation was performed;

categorizing a first file in the first set of files as a file that is updated frequently;

dividing the first file into chunks of a first size;

categorizing a second file in the first set of files as a file that is not updated frequently;

dividing the second file into chunks of a second size, wherein the second size is different from the first size;

for each chunk in the first file and for each chunk in the second file, generating, by the client device, a cryptographic signature value for the chunk, wherein the cryptographic signature value is computed by using actual data of the chunk as an input to a cryptographic signature-generating algorithm which computes the cryptographic signature value;

generating, at the client device, a first inventory including the cryptographic signatures calculated for the chunks in the first and second files;

transmitting the first inventory from the client device to a backup server.

9. The method of claim 8, further comprising:

receiving, at the backup server, the first inventory including the cryptographic signatures calculated for the chunks in the first and second files;

extracting, by the backup server, the cryptographic signatures from the first inventory;

comparing, by the backup server, the cryptographic signatures to a list containing all cryptographic signatures identifying raw data currently stored in a backup system managed by the backup server regardless of whether the raw data currently stored in the backup system was stored there by the client device or by a second client device other than the client device which generated the first inventory;

based on the comparison step, generating a second inventory, the second inventory including the cryptographic signatures found in the first inventory and not found in the list containing all cryptographic signatures identifying actual data currently stored in the backup system;

transmitting the second inventory from the backup server to the client device;

receiving, at the client device, the second inventory;

transmitting, from the client device to the backup server, every chunk having a cryptographic signature corresponding to a cryptographic signature found in the second inventory; and

causing the chunks transmitted from the client device to the backup server to be stored in the backup system.

10. The method of claim 8, wherein the second size is larger than the first size.

11. The method of claim 10, wherein the first size is less than 1 MB and wherein the second size is less than 10 MB.

12. The method of claim 8, wherein the first file is categorized by comparing a creation timestamp of the first file with an update timestamp of the first file.

13. The method of claim 12, wherein the second file is categorized by comparing a creation timestamp of the second file with an update timestamp of the second file.

14. The method of claim 13, wherein the first file is also categorized by comparing the update timestamp of the first file with a current time.

15. The method of claim 8, wherein the first inventory further includes one or more markers that identify sizes of chunks of the first and second file.

16. A method, comprising:

scanning a list of filed stored in memory of a client device;

identifying a first set of files in the list of files as one or more of new and changed since a last backup operation was performed;

analyzing metadata of a first file in the first set of files;

based on the analysis of the metadata of the first file, dividing the first file into chunks of a predetermined size;

for each chunk in the first file, generating, by the client device, a cryptographic signature value for the chunk, wherein the cryptographic signature value is computed by using actual data of the chunk as an input to a cryptographic signature-generating algorithm which computes the cryptographic signature value;

generating, at the client device, a first inventory including the cryptographic signatures calculated for the chunks in the first file; and

transmitting the first inventory from the client device to a backup server.

17. The method of claim 16, further comprising:

receiving, at the backup server, the first inventory including the cryptographic signatures calculated for the chunks in the first file;

extracting, by the backup server, the cryptographic signatures from the first inventory;

comparing, by the backup server, the cryptographic signatures to a list containing all cryptographic signatures identifying raw data currently stored in a backup system managed by the backup server regardless of whether the raw data currently stored in the backup system was stored there by the client device or by a second client device other than the client device which generated the first inventory;

based on the comparison step, generating a second inventory, the second inventory including the cryptographic signatures found in the first inventory and not found in the list containing all cryptographic signatures identifying actual data currently stored in the backup system;

transmitting the second inventory from the backup server to the client device;

receiving, at the client device, the second inventory;

transmitting, from the client device to the backup server, every chunk having a cryptographic signature corresponding to a cryptographic signature found in the second inventory; and

causing the chunks transmitted from the client device to the backup server to be stored in the backup system.

18. The method of claim 16, wherein the metadata includes a file type.

19. The method of claim 18, wherein the metadata further includes a creation timestamp and an update timestamp.

20. The method of claim 18, wherein the predetermined size is determined, at least in part, based on the file type.