Detecting and managing missing parents between primary and secondary data stores for content addressed storage

• US 7,599,971 B1
• Filed: 10/03/2006
• Issued: 10/06/2009
• Est. Priority Date: 10/03/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Tinius, Paul E., Hutchinson, Thomas Dixon, Hagerstrom, Carl F., Bharthulwar, Shridhar
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): PHILLIPS, III, ALBERT M
• Application Number:
  US11/538,313
• Time in Patent Office: 1,099 Days
• Field of Search: 707/204
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method and system for locating and eliminating missing parent files within a primary storage device. The method includes identifying a placeholder file on a primary storage device, the placeholder file having an offline reference and being associated with file identification data, identifying a secondary file on a secondary storage device using a content address of the secondary file included in the offline reference of the placeholder file, identifying an online reference of the secondary file, determining if the online reference of the secondary file properly references the file identification data of the placeholder file, and updating the online reference of the secondary file so that it includes the file identification data of the placeholder file.

Claims

1. In a distributed computing system having a primary storage device, a secondary storage device and a policy engine server, the policy engine serve being configured to manage migration of files between the primary and secondary storage devices by migrating data files on the primary storage device to the secondary storage device and replacing the data files on the primary storage device with placeholder files, a method for locating placeholder files within the primary storage device that are not properly referenced by secondary files stored on the secondary storage device, the method comprising:

identifying a placeholder file on a primary storage device that has replaced a data file, the placeholder file including an offline reference to the data file stored as a secondary file on the secondary storage device, wherein the offline reference includes file identification data including a content address of the secondary file and the file identification data describes the data file;

identifying the secondary file on the secondary storage device using the content address of the secondary file included in the offline reference of the placeholder file;

identifying an online reference included in the secondary file, wherein the online reference includes an address to a particular placeholder file stored on the primary storage device;

determining that the placeholder file is a missing parent when the address to the particular placeholder file does not identify the placeholder file; and

notifying a user of the policy engine server when the online reference of the secondary file does not properly reference the placeholder file.

2. The method as recited in claim 1, further comprising at least one of:

indicating the reason for the improper reference;

or

updating the online reference of the secondary file so that it includes the file identification data of the placeholder file and properly references the placeholder file.

3. The method as recited in claim 1, wherein identifying the secondary file on the secondary storage device using the content address of the secondary file included in the offline reference of the placeholder file further comprises:

accessing a database on a policy engine server, the database storing the content address of the secondary file; and

locating the secondary file by comparing the content address in the offline reference of the placeholder file with the content address of the secondary file stored in the database.

4. The method as recited in claim 1, wherein identifying the secondary file on the secondary storage device using the content address of the secondary file included in the offline reference of the placeholder file further comprises:

performing an itemized scan of data objects on the secondary storage device;

analyzing metadata of each data object to identify the location of the data object; and

determining whether the data object contains the same content address as the content address in the offline reference of the placeholder file.

5. The method as recited in claim 1, wherein the primary storage device is further configured to communicate with a client computer system, the client computer system able to access the secondary file via the placeholder file on the primary storage device.

6. The method as recited in claim 2, wherein the file identification data of the placeholder file includes a filename that has been renamed, and wherein the online reference of the secondary file is updated to include the renamed filename of the placeholder file.

7. The method as recited in claim 2, wherein the file identification data of the placeholder file includes an additional filename, and wherein the online reference of the secondary file is updated to include the additional filename of the placeholder file.

8. The method as recited in claim 2, wherein the placeholder file has been restored from a backup location to a different location in a file system of the primary storage device than a previous placeholder file such that new file identification data of the placeholder file is created, and wherein the online reference of the secondary file is updated to include the new file identification data of the placeholder file.

9. The method as recited in claim 1, wherein the online reference of the secondary file includes a content address of the placeholder file.

10. A content addressable data migration storage system for eliminating missing parent files that are not properly identified by secondary files, the storage system comprising:

a primary storage device employing one of location addressable storage or content addressable storage and configured to store and detect a placeholder file, the detected placeholder file having replaced a data file that was migrated to a secondary storage device, the placeholder file including an offline reference with file identification data for identifying the placeholder file, the offline reference referencing the data file which is stored as a secondary file on the secondary storage device, wherein the offline address includes an address of the secondary file

the secondary storage device employing content addressable storage and configured to store the secondary file, the secondary file including file identification data for identifying the secondary file, the file identification data of the secondary file related to the offline reference of the detected placeholder file, and the secondary file including at least one online reference to a referenced placeholder file on the primary storage device, wherein the detected placeholder file is properly identified if the detected placeholder file is the referenced placeholder file; and

a policy engine server including a file management module for detecting that the detected placeholder file is not properly identified by the at least one online reference of the secondary file, wherein the detected placeholder file is a missing parent file that is eliminated from the primary storage device when not properly identified by the at least one online reference of the secondary file, wherein the file management module notifies a user of the policy engine server that the detected placeholder file is not properly identified by the at least one online reference of the secondary file.

11. The system as recited in claim 10, wherein the file management module of the policy engine server is further configured for performing at least one of:

eliminating the missing parent file by updating the at least one online reference of the secondary file such that the online reference refers to the detected placeholder file.

12. The system as recited in claim 10, wherein prior to updating the at least one online reference of the secondary file such that the online reference refers to the detected placeholder file, the file identification data of the detected placeholder file included at least one of a renamed filename, an added filename, or new file identification data from the detected placeholder file being restored from a backup location to a different location in a file system of the primary storage device than a previous placeholder file.

13. The system as recited in claim 10, wherein the file identification data of the secondary file further includes one or more filenames associated with the detected placeholder file.

14. The system as recited in claim 10, wherein the policy engine server further includes a data migration module for moving files between the primary storage device and the secondary storage device, the data migration module configured to delete a data file from the primary storage device, create the detected placeholder file to represent the deleted data file, and copy the data file to the secondary storage device in the form of the secondary file.

15. The system as recited in claim 10, further comprising a client computer system configured to access the placeholder file from the primary storage device and access the secondary file via the placeholder file.

16. The system as recited in claim 10, wherein the content address is generated from a hash function.

17. In a distributed computer having a primary storage device, a secondary storage device and a policy engine server, the policy engine server being configured to manage migration of files between the primary storage device and the secondary storage device by migrating data files on the primary storage device to the secondary storage device and replacing the data files on the primary storage device with placeholder files, a computer program product configured to implement a method of locating and eliminating placeholder files within the primary storage device that are not properly referenced by secondary files stored on the secondary storage device, the computer program product comprising one or more computer readable media having stored thereon computer executable instructions that, when executed by a processor, cause the policy engine server to perform the following:

identify a placeholder file on a primary storage device that has replaced a data file, the placeholder file including an offline reference to the data file stored as a secondary file on the secondary storage device, wherein the offline reference includes a content address of the secondary file;

identify the secondary file on the secondary storage device using the content address of the secondary file included in the offline reference of the placeholder file;

identify an online reference of the secondary file, wherein the online reference includes an address to a particular placeholder file stored on the primary storage device;

determine that the placeholder file is a missing parent when the particular placeholder file is not the placeholder file; and

notify a user of the policy engine server when the online reference of the secondary file does not properly reference the placeholder file or the file identification data of the placeholder file.

18. The method as recited in claim 17, wherein the computer executable instructions that cause the server system to identify a secondary file on a secondary storage device using a content address of the secondary file included in the offline reference of the placeholder file further comprise instructions to cause the policy engine server to perform at least one of the following:

indicating the reason for the improper reference;

or

update the online reference of the secondary file so that it includes the file identification data of the placeholder file.

19. The method as recited in claim 17, wherein the computer executable instructions that cause the server system to identify a secondary file on a secondary storage device using a content address of the secondary file included in the offline reference of the placeholder file further comprise instructions to cause the policy engine server to perform the following:

perform an itemized scan of data objects on the secondary storage device;

analyze metadata of each data object to identify the location of the data object; and

determine whether the data object contains the same content address as the content address in the offline reference of the placeholder file.

20. The method as recited in claim 17, wherein the computer executable instructions that cause the server system to identify a secondary file on a secondary storage device using a content address of the secondary file included in the offline reference of the placeholder file further comprise instructions to cause the policy engine server to perform at least one of the following:

wherein the file identification data of the placeholder file includes a filename that has been renamed, and wherein the online reference of the secondary file is updated to include the renamed filename of the placeholder file;

wherein the file identification data of the placeholder file includes an additional filename, and wherein the online reference of the secondary file is updated to include the additional filename of the placeholder file;

or

wherein the placeholder file has been restored from a backup location such that new file identification data of the placeholder file is created, and wherein the online reference of the secondary file is updated to include the new file identification data of the placeholder file.