Detecting and managing orphan files between primary and secondary data stores

• US 7,685,177 B1
• Filed: 10/03/2006
• Issued: 03/23/2010
• Est. Priority Date: 10/03/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Tinius, Paul E., Hutchinson, Thomas Dixon, Hagerstrom, Carl F., Bharthulwar, Shridhar
• Primary Examiner(s): Vital; Pierre M
• Assistant Examiner(s): Rahman; Sabana
• Application Number:
  US11/538,351
• Time in Patent Office: 1,267 Days
• Field of Search: 707201-206
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method and system for locating and eliminating orphan files within a secondary storage device. The method includes identifying a secondary file on a secondary storage device, the secondary file being associated with file identification data, identifying a placeholder file on a primary storage device, the placeholder file being associated with an offline reference, and determining if the offline reference of the placeholder file validly references the secondary file.

Claims

1. In a distributed computing system having a primary storage device, a secondary storage device and a policy engine server, the policy engine server being configured to manage migration of files between the primary and secondary storage devices, a method for locating orphan files within the secondary storage device, the method comprising:

identifying a secondary file on a secondary storage device, the secondary file including file identification data and an online reference that references a placeholder file on a primary storage device, wherein the secondary file comprises a data file that has been migrated from a primary storage device to the secondary storage device and wherein the data file has been deleted from the primary storage and replaced with the placeholder file;

identifying the placeholder file on the primary storage device from the online reference, wherein the placeholder file includes an offline reference to the secondary file and is stored on the primary storage device to assist in locating the secondary file;

determining that the secondary file is an orphan file if the offline reference does not validly reference the secondary file.

2. The method as recited in claim 1, further comprising at least one of:

notifying a user of the policy engine server if the offline reference of the placeholder file does not validly reference the secondary file and indicating the reason for the improper reference;

or

marking the secondary file for deletion if the offline reference of the placeholder file does not validly reference the secondary file.

3. The method as recited in claim 1, wherein the offline reference of the placeholder file does not validly reference the secondary file if the placeholder file contains an offline reference having a path that does not correspond with a path in the file identification data of the secondary file.

4. The method as recited in claim 1, wherein the placeholder file and the secondary file were created upon a first migration of a data file from the primary storage to the secondary storage, wherein upon recall of the data file to the primary storage and a second migration of the data file to the secondary storage, the offline reference of the placeholder file no longer validly references the secondary file.

5. The method as recited in claim 1, wherein the offline reference of the placeholder file does not validly reference the secondary file if the offline reference of the placeholder file references a different secondary file.

6. The method as recited in claim 2, wherein marking the secondary file for deletion further includes immediately deleting the secondary file.

7. The method as recited in claim 2, wherein marking the secondary file for deletion further includes:

recording the time at which the secondary file is marked for deletion; and

deleting the marked file following a predefined period of time.

8. The method as recited in claim 7, wherein the predefined period of time is programmable by a user of a policy engine server.

9. The method as recited in claim 1, further comprising determining if any other placeholder file located on the primary storage device includes reference to a secondary file that has been marked for deletion.

10. A data migration storage system for identifying orphan files, comprising:

a primary storage device employing one of location addressable storage or content addressable storage and configured to store and detect placeholder files, each of the placeholder files including an offline reference that references a secondary file, the secondary file being a data file that has been migrated from the primary storage, wherein the data file is deleted from the primary storage device and replaced with the placeholder file after the data file is migrated;

a secondary storage device employing one of location addressable storage or content addressable storage and configured to store secondary files, each of the secondary files including at least one online reference to at least one placeholder file on the primary storage device; and

a policy engine server including a file management module for identifying secondary files on the secondary storage device, wherein the policy engine server uses the online references of the secondary files to identify placeholder files on the primary storage device and then identifies secondary files that are not identified by any offline references of placeholder files on the primary storage device, wherein secondary files that are not identified by any offline references of placeholder files are orphan files.

11. The system as recited in claim 10, wherein the file management module of the policy engine serve is further configured for performing at least one of:

notifying a user of the policy engine server when the offline references of the placeholder files do not validly reference the secondary files;

or

marking the identified secondary files that are not validly references by offline references for deletion.

12. The system as recited in claim 10, wherein identifying secondary files that are not identified by any offline references of placeholder files on the primary storage occurs because a placeholder file having an offline reference to a secondary file has been deleted.

13. The system as recited in claim 10, wherein identifying secondary files that are not identified by any offline references of placeholder files on the primary storage occurs because a data file has been recalled to the primary storage and a placeholder file having an offline reference to a secondary file has been deleted.

14. The system as recited in claim 10, wherein identifying secondary files that are not identified by any offline references of placeholder files on the primary storage occurs because a placeholder file that originally referenced a first secondary file has been changed to include a reference to a second secondary file.

15. The system as recited in claim 10, wherein the file management module is further configured to delete the identified secondary files after the completion of a predetermined period of time.

16. In a distributed computer having a primary storage device, a secondary storage device and a policy engine server, the policy engine server being configured to manage migration of files between the primary storage device, a computer program product configured to implement a method of locating and eliminating orphan files within the secondary storage device, the computer program product comprising computer executable instructions that, when executed by a processor, cause the policy engine server to perform the following:

identify a secondary file on a secondary storage device, the secondary file having file identification data and an online reference, wherein the secondary file comprises a data file that has been migrated from a primary storage device to the secondary storage device;

identify a placeholder file on a primary storage device using the online reference included in the secondary file, the placeholder file including an offline reference that references a particular secondary file stored on the secondary storage device; and

determine that the secondary file is an orphan file if the offline reference does not validly reference the secondary file.

17. The method as recited in claim 16, wherein the computer executable instructions that cause the server system to determine if the offline reference of the placeholder file validly references the secondary file further comprise instructions to cause the policy engine server to perform at least one of the following:

notify a user of the policy engine server if the offline reference of the placeholder file does not validly reference the secondary file and indicating the reason for the improper reference;

or

mark the secondary file for deletion if the offline reference of the placeholder file does not validly reference the secondary file.

18. The method as recited in claim 16, wherein the computer executable instructions that cause the server system to determine if the offline reference of the placeholder file validly references the secondary file further comprise instructions to cause the policy engine server to perform at least one of the following:

wherein the offline reference of the placeholder file does not validly reference the secondary file if the placeholder file contains an offline reference having a path that does not correspond with a path in the file identification data of the secondary file;

wherein the placeholder file and the secondary file were created upon a first migration of a data file from the primary storage to the secondary storage, wherein upon recall of the data file to the primary storage and a second migration of the data file to the secondary storage, the offline reference of the placeholder file no longer validly references the secondary file;

or

wherein the offline reference of the placeholder file does not validly reference the secondary file if the offline reference of the placeholder file references a different secondary file.

19. The method as recited in claim 16, wherein the computer executable instructions that mark the secondary file for deletion further comprise instructions, that when executed, cause the policy engine server to perform at least one of the following:

immediately delete the secondary file;

record the time at which the secondary file is marked for deletion;

or

delete the marked file following the completion of a predefined period of time.

20. The method as recited in claim 16, further comprising executable instructions that, when executed, determine if any of the other placeholder file located on the primary storage device includes reference to a secondary file that has been marked for deletion.