Distributed catalog, data store, and indexing

• US 9,135,264 B2
• Filed: 03/14/2011
• Issued: 09/15/2015
• Est. Priority Date: 03/12/2010
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Copiun Incorporated (Blackberry Limited)
• Inventors: Chaudhry, Puneesh, Jain, Sanjay
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/047,273
• Publication Number:
  US 20110225141A1
• Time in Patent Office: 1,646 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files

Abstract

This disclosure relates to a system and method for distributed catalog processing, data caching, and indexing to create an efficient, scalable, secure, high availability, disaster recovery enabled backup and storage system.

Claims

1. A client computer system configured to manage distributed backup data, comprising:

a data store comprising;

computer-executable software;

a production volume comprising a plurality of data files, and

an agent backup catalog comprising local backup data corresponding to remote backup data stored in a remote backup catalog of a remote server;

a processor in data communication with the data store and configured to execute the software, the software being configured to implement an agent indexer on the client computer system and to cause the agent indexer to build an index for the production volume by crawling through file contents of the production volume using a virtual file pointer,

the software being further configured to implement a backup agent on the client computer system and to cause the backup agent to;

receive a message from a local user to start a backup process;

generate change data based on a comparison of the agent backup catalog and the production volume; and

send the change data to the remote server, the change data comprising metadata that indicates one or more differences between the current state of the production volume and the agent backup catalog, and the change data being configured to be used by the remote sever to update the remote backup data stored in the remote backup catalog.

2. The client computer system of claim 1, wherein the backup agent is further configured to:

update the agent backup catalog based on the change data.

3. The client computer system of claim 1, wherein the backup agent is further configured to:

identify, based on the change data, a file in the production volume to be backed-up to the remote server;

transfer a plurality of storage blocks corresponding to the identified file to a local cache on the client computer system; and

transfer the plurality of storage blocks corresponding to the file to the remote server.

4. The client computer system of claim 3, wherein the backup agent is further configured to:

determine that the local cache has exceeded a capacity threshold;

determine a timestamp associated with each storage block of the plurality of storage blocks in the local cache, the timestamp for each respective storage block indicating a time at which the respective block was last backed up to the remote server;

delete any respective storage block when the timestamp for the respective storage block precedes a time of the last successful backup to the remote server.

5. The client computer system of claim 3, wherein the backup agent is further configured to:

receive an instruction to recover a file;

determine that the file is not stored in a local cache on the client computer system;

request the file from the remote server;

receive a plurality of storage blocks associated with the file from the remote server; and

store the plurality of storage blocks associated with the file in the local cache.

6. The client computer system of claim 1, wherein the agent backup catalog is a subset of the remote backup catalog on the remote server.

7. The client computer system of claim 1, wherein the agent backup catalog is associated with a single user and the remote backup catalog on the remote server is associated with a plurality of users, including the single user.

8. The client computer system of claim 1, wherein the remote backup data stored in the remote backup catalog of the remote server corresponding to the local backup data is only a subset of the total backup data stored in the remote backup catalog.

9. A method of managing distributed backup data on a client computer system an agent backup catalog, the method comprising:

building, by an agent indexer, an index for a production volume comprising a plurality of data files by crawling through file contents of the production volume using a virtual file pointer;

receiving, at a backup agent, a message from a local user to start a backup process;

generating, by the backup agent, change data based on a comparison of an agent backup catalog comprising local backup data corresponding to remote backup data stored in a remote backup catalog of a remote server and the production volume; and

sending, from the backup agent, the change data to the remote server, the change data comprising metadata that indicates one or more differences between the current state of the production volume and the agent backup catalog, and the change data being configured to be used by the remote sever to update the remote backup data stored in the remote backup catalog.

10. The method of claim 9, further comprising:

updating, by the backup agent, the agent backup catalog based on the change data.

11. The method of claim 9, further comprising:

identifying, by the backup agent, a file in the production volume to be backed-up to the remote server based on the change data;

transferring, by the backup agent, a plurality of storage blocks corresponding to the identified file to a local cache on the client computer system;

transferring, by the backup agent, the plurality of storage blocks corresponding to the file to the remote server.

12. The method of claim 11, further comprising:

determining, by the backup agent, that the local cache has exceeded a capacity threshold;

determining, by the backup agent, a timestamp associated with each storage block of the plurality of storage blocks in the local cache, the timestamp for each respective storage block indicating a time at which the respective block was last backed up to the remote server;

deleting, by the backup agent, any respective storage block when the timestamp for the respective storage block precedes a time of the last successful backup to the remote server.

13. The method of claim 11, further comprising:

receiving, at the backup agent, an instruction to recover a file;

determining, by the backup agent, that the file is not stored in the local cache on the client computer system;

requesting, by the backup agent, the file from the remote server;

receiving, at the backup agent, a plurality of storage blocks associated with the file from the remote server; and

storing, by the backup agent, the plurality of storage blocks associated with the file in the local cache.

14. The method of claim 9, wherein the agent backup catalog is a subset of the remote backup catalog on the remote server.

15. The method of claim 9, wherein the agent backup catalog is associated with a single user and the remote backup catalog on the remote server is associated with a plurality of users, including the single user.

16. The method of claim 9, wherein the remote backup data stored in the remote backup catalog of the remote server corresponding to the local backup data is only a subset of the total backup data stored in the remote backup catalog.

17. A non-transitory, computer-readable medium comprising computer-executable instructions that, when executed by a processor in a client computer system, cause the client computer system to perform a method of managing distributed backup data on the system, the method comprising:

building, by an agent indexer, an index for a production volume comprising a plurality of data files by crawling through file contents of the production volume using a virtual file pointer;

receiving, at a backup agent, a message from a local user to start a backup process;

generating, by the backup agent, change data based on a comparison of an agent backup catalog comprising local backup data corresponding to remote backup data stored in a remote backup catalog of a remote server and the production volume; and

sending, from the backup agent, the change data to the remote server, the change data comprising metadata that indicates one or more differences between the current state of the production volume and the agent backup catalog, and the change data being configured to be used by the remote sever to update the remote backup data stored in the remote backup catalog.

18. The non-transitory, computer-readable medium of claim 17, wherein the method further comprises:

updating, by the backup agent, the agent backup catalog based on the change data.

19. The non-transitory, computer-readable medium of claim 17, wherein the method further comprises:

identifying, by the backup agent, a file in the production volume to be backed-up to the remote server based on the change data;

transferring, by the backup agent, a plurality of storage blocks corresponding to the identified file to a local cache on the client computer system;

transferring, by the backup agent, the plurality of storage blocks corresponding to the file to the remote server.

20. The non-transitory, computer-readable medium of claim 19, wherein the method further comprises:

determining, by the backup agent, that the local cache has exceeded a capacity threshold;

determining, by the backup agent, a timestamp associated with each storage block of the plurality of storage blocks in the local cache, the timestamp for each respective storage block indicating a time at which the respective block was last backed up to the remote server;

deleting, by the backup agent, any respective storage block when the timestamp for the respective storage block precedes a time of the last successful backup to the remote server.

21. The non-transitory, computer-readable medium of claim 19, wherein the method further comprises:

receiving, at the backup agent, an instruction to recover a file;

determining, by the backup agent, that the file is not stored in the local cache on the client computer system;

requesting, by the backup agent, the file from the remote server;

receiving, at the backup agent, a plurality of storage blocks associated with the file from the remote server; and

storing, by the backup agent, the plurality of storage blocks associated with the file in the local cache.

22. The non-transitory, computer-readable medium of claim 17, wherein the agent backup catalog is a subset of the remote backup catalog on the remote server.

23. The non-transitory, computer-readable medium of claim 17, wherein the agent backup catalog is associated with a single user and the remote backup catalog on the remote server is associated with a plurality of users, including the single user.

24. The non-transitory, computer-readable medium of claim 17, wherein remote backup data stored in the remote backup catalog of the remote server corresponding to the local backup data is only a subset of the total backup data stored in the remote backup catalog.