Archiving data objects using secondary copies

• US 8,935,492 B2
• Filed: 09/30/2011
• Issued: 01/13/2015
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Varadharajan, Prakash
• Primary Examiner(s): Shah, Sanjiv
• Assistant Examiner(s): GIARDINO JR, MARK A
• Application Number:
  US13/250,824
• Publication Number:
  US 20120084524A1
• Time in Patent Office: 1,201 Days
• Field of Search: 711/162, 711/161, 707/665, 707/673
• US Class Current: 711/161
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/24544 Join order optimisation
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0641 De-duplication techniques

Abstract

A system for archiving data objects using secondary copies is disclosed. The system creates one or more secondary copies of primary copy data that contains multiple data objects. The system maintains a first data structure that tracks the data objects for which the system has created secondary copies and the locations of the secondary copies. To archive data objects in the primary copy data, the system identifies data objects to be archived, verifies that previously-created secondary copies of the identified data objects exist, and replaces the identified data objects with stubs. The system maintains a second data structure that both tracks the stubs and refers to the first data structure, thereby creating an association between the stubs and the locations of the secondary copies.

Claims

1. A non-transitory computer-readable storage medium whose contents cause a data storage system to perform a method for archiving multiple data objects included in primary copy data, the method comprising:

receiving, from a client, both full and incremental backup copies of the client'"'"'s primary copy data;

using the received backup copies to create a secondary copy of multiple data objects included in the primary copy data;

for each of the multiple data objects for which a secondary copy was created, adding an entry for the data object to a data structure, the entry including an identifier associated with the data object;

after creating the secondary copy, identifying one or more of the multiple data objects that satisfy predetermined archival criteria; and

,

for each of the identified one or more data objects;

looking up the identified data object in the data structure using the identifier associated with the identified data object;

receiving a token for the identified data object; and

replacing the identified data object in the primary copy data with a stub referencing the secondary copy of the identified data object, wherein the stub comprises the token;

wherein the method further comprises;

determining that a first data object included in the primary copy data has been deleted and a corresponding deletion time;

only when the deletion time is less than a predetermined period of time ago, creating, from the first secondary copy of the first data object, a second secondary copy of the first data object; and

creating, from the first secondary copy of other data objects except for the first data object, a corresponding second secondary copy of the other data objects.

2. The computer-readable storage medium of claim 1 wherein the secondary copy of the multiple data objects is a first secondary copy of the multiple data objects, and wherein the determining that the first data object included in the primary copy data has been deleted includes:

receiving information regarding the first data object from a driver or a file system that detects deletions.

3. The computer-readable storage medium of claim 1 wherein the predetermined period of time is determined at least in part by an object type of the first data object.

4. The computer-readable storage medium of claim 1 wherein the predetermined period of time is determined by a storage policy.

5. The computer-readable storage medium of claim 1 wherein the predetermined archival criteria includes at least one of:

a data object type, a data object age, a data object size, a percentage of disk quota, remaining storage, and metadata.

6. A system for archiving data objects using secondary copies, the system comprising:

a processor;

a memory;

a component that creates one or more secondary copies of primary copy data that contains multiple data objects;

a first data structure that tracks the data objects for which secondary copies have been created and locations of the secondary copies;

a component that identifies data objects to be archived, verifies that previously-created secondary copies of the identified data objects exist, and replaces the identified data objects with stubs; and

a second data structure that both tracks the stubs and refers to the first data structure, thereby creating an association between the stubs and the locations of the secondary copies;

wherein the secondary copy of the multiple data objects is a first secondary copy of the multiple data objects, and wherein the system further comprises;

a component that determines that a first data object included in the primary copy data has been deleted and a corresponding deletion time;

a component that creates, only when the deletion time is less than a predetermined period of time ago, from the first secondary copy of the first data object, a second secondary copy of the first data object; and

a component that creates, from the first secondary copy of other data objects except for the first data object, a corresponding second secondary copy of the other data objects.

8. The system of claim 6, wherein the predetermined period of time is determined at least in part by an object type of the first data object.

9. The system of claim 6 wherein the secondary copy of the multiple data objects is a backup copy created at least in part by an incremental backup operation.

7. A method for archiving multiple data objects included in primary copy data, the method comprising:

creating a secondary copy of the multiple data objects included in the primary copy data;

after creating the secondary copy, identifying one or more of the multiple data objects that are to be archived; and

for each of the identified one or more data objects to be archived, replacing the identified data object in the primary copy data with a stub referencing the secondary copy of the identified data object;

wherein the secondary copy of the multiple data objects is a first secondary copy of the multiple data objects, and wherein the method further comprises;

determining that a first data object included in the primary copy data has been deleted and a corresponding deletion time;

only when the deletion time is less than a predetermined period of time ago, creating, from the first secondary copy of the first data object, a second secondary copy of the first data object; and

creating, from the first secondary copy of other data objects except for the first data object, a corresponding second secondary copy of the other data objects.

10. The method of claim 7 further comprising:

for each of the multiple data objects for which a secondary copy was created, adding an entry for the data object to a data structure, the entry including an identifier associated with the data object; and

for each of the identified one or more data objects to be archived

looking up the identified data object in the data structure using the identifier associated with the identified data object;

receiving a token for the identified data object; and

including the token in the stub.

11. The method of claim 7, wherein determining that the first data object included in the primary copy data has been deleted and the corresponding deletion time further comprises:

scanning the primary copy data to populate an image map; and

comparing the image map to a data structure that indicates a location of the first secondary copy.

12. The method of claim 7 wherein the predetermined period of time is set by a storage policy.

13. The method of claim 7 wherein the secondary copy of the multiple data objects is a first secondary copy of the multiple data objects, and wherein the method further comprises:

in response to determining that the deletion time is more than a predetermined period of time ago, deleting the secondary copy of the first data object.

14. The method of claim 13, wherein determining that the first data object included in the primary copy data has been deleted and the corresponding deletion time further comprises receiving information from a driver or file system that detects deletions.

15. The method of claim 13 wherein the predetermined period of time is determined at least in part by an object type of the first data object.

16. The method of claim 7 wherein the secondary copy of the multiple data objects is a backup copy created at least in part by an incremental backup operation.

17. The method of claim 7 wherein identifying one or more of the multiple data objects that are to be archived comprises determining that the one or more of the multiple data objects satisfy predetermined retention criteria, including at least one of:

a data object type, a data object age, a data object size, a percentage of disk quota, remaining storage, and metadata.

18. The method of claim 7 wherein identifying one or more of the multiple data objects that are to be archived comprises determining that the one or more of the multiple data objects satisfy predetermined retention criteria.

19. The method of claim 7 wherein at least one of the multiple data objects is an email or email attachment.

20. The method of claim 7 further comprising:

determining that a first data object included in the primary copy data satisfies predetermined criteria; and

in response to determining that the first data object satisfies the predetermined criteria, deleting the secondary copy of the first data object.