Method and apparatus for hierarchical storage of data for efficient archiving and retrieval of data

• US 6,266,679 B1
• Filed: 10/12/1999
• Issued: 07/24/2001
• Est. Priority Date: 10/12/1999
• Status: Expired due to Term

• Current Assignee: GlobalFoundries, Inc.
• Original Assignee: AMD Company Limited (Advanced Micro Devices, Inc.)
• Inventors: Szalwinski, Bruce, Winslett, Michael E.
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): Coby, Frantz
• Application Number:
  US09/416,071
• Time in Patent Office: 651 Days
• Field of Search: 707/103, 707/200, 707/204, 707/1, 707/203
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

The present invention provides for a method and an apparatus for archiving and retrieving data. At least one top-level directory is created to store files. A file-location database is created to track stored files in the top level directory. Files from a primary database are received. Files received from the database are archived into the top level directory, in response to receiving files from a primary database. An immediate backup of the archived files is created. A long-term backup of the archived files is created.

Claims

1. A method for archiving and retrieving data from a database, comprising:

creating at least one top-level directory to store files;

creating a file-location database to track stored files in said top level directory;

receiving files from a primary database;

archiving the primary database files into said top level directory, in response to receiving files from a primary database;

creating an immediate backup of the archived primary database files; and

creating a long-term backup of said archived primary database files.

2. The method as described in claim 1, wherein said step of executing archiving process further comprises:

creating at least one subdirectory within said top level directory;

creating a file-location database to track stored files in said subdirectory;

receiving files from said primary database;

numbering said subdirectories in said top level database;

assigning said files to be stored in a highest numbered subdirectory;

determining whether said highest numbered subdirectory contains a maximum allowable number of files;

creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains a maximum allowable number of files;

creating a file location database to store name of said file and said subdirectory number; and

entering name of said file and said subdirectory number into said file location database.

3. The method as described in claim 2, wherein creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains a maximum allowable number of files further comprises creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains one thousand files.

4. The method as described in claim 1, creating an immediate backup of the archived primary database files further comprises:

creating a backup directory in a backup disk;

creating a second copy of each said archived file;

storing said second copy of said archived file into said backup directory;

copying said backup directory into a backup storage medium at a predetermined frequency of time; and

deleting said files from said backup directory in response to said backup directory being copied into said backup storage medium.

5. The method as described in claim 4, wherein said step of copying said backup directory into a backup storage medium includes copying said backup directory into a backup tape.

6. The method as described in claim 5, wherein said step of copying said backup directory into a backup storage medium at a predetermined frequency of time further comprises of copying said backup directory into a backup storage medium once every week.

7. The method as described in claim 1, wherein creating a long-term backup of said archived primary database files further comprises:

determining a period of time that said file has resided in said top level directory;

determining whether said period of time exceeds a maximum residence time;

archiving a subdirectory located in said top level directory and which contains said file, into a backup storage medium in response to said period of time exceeding said maximum residence time;

creating a text file to record identification data relating to said backup storage medium and names of said files in said subdirectory;

creating a backup database, to record identification data relating to said files and said top level directory, on said storage medium; and

deleting said files in said subdirectories in said top level directory in response to said subdirectories being archived into said backup storage medium.

8. The method as described in claim 7, wherein said step of determining whether said period of time exceeds a maximum residence time further comprises determining whether said period of time exceeds six months.

9. The method as described in claim 7, wherein said step of archiving a subdirectory located in said top level directory and which contains said file, into a backup storage medium further comprises archiving a subdirectory located in said top level directory and which contains said file, into a backup tape.

10. The method as described in claim 7, wherein said step of creating a backup database, to record identification data relating to said files and said top level directory further comprises creating a backup database, to record names and number of said files and said top level directory.

11. The method as described in claim 1, further comprising a method of retrieving said archived files comprising:

receiving a list of said archived files to be restored;

converting said list of archived files to be restored into path file names;

adding said path file names to a restoration list;

executing a restoration process at a predetermined time to restore said archived files to be restored;

determining whether all said archived files to be restored, have been restored;

placing said path file names corresponding to said archived files that are not restored, on an unrestored-file list, in response to a determination that all said archived files are not restored.

12. The method as described in claim 11, further comprising executing said restoration process to retrieve said archived files on said unrestored-file list.

13. The method as described in claim 11, wherein said step of executing a restoration process at a predetermined time further comprises executing a restoration process at a period of time of reduced data access activity.

14. The method as described in claim 13, wherein said step of executing a restoration process at a period of time of reduced data access activity, further comprises executing a restoration process during night time.

15. The method as described in claim 11, wherein said step of executing a restoration process further comprises:

locating said top level directory which contains said subdirectory that holds said archived data that is to be restored;

determining whether said subdirectory that holds said archived file that is to be restored is found in said top level directory;

copying said archived file from said subdirectory in said top level directory, in response to the determination that said subdirectory that holds said archived file is found in said top level directory;

placing said copied file in a restoration location;

uncompressing said copied file in said restoration location to allow said primary database to access said copied file;

adding the name of said archived file to a restoration-pending file list in response to the determination that said subdirectory that holds said archived file is not found in said top level directory; and

invoking a restoration procedure to restore said archived file from a backup storage medium to allow said primary database to access said archived file.

16. The method as described in claim 15, wherein said step of placing said copied file in a restoration location further comprises placing said copied file in a temporary directory.

17. The method as described in claim 1, wherein said step of invoking a restoration procedure to restore said archived file from a backup storage medium further comprises invoking a restoration procedure to restore said archived file from a backup tape.

18. The method as described in claim 1, wherein said step of invoking a restoration procedure further comprises:

reading a list of said archived files to be restored;

creating an instruction set to recover said archived files to be restored from a storage medium;

executing said instruction set to retrieve said archived files to be restored from said storage medium;

placing in memory storage, the name of said archived file that is not recovered;

restoring said archived file that is recovered from said storage medium, into a staging area;

copying said archived file from said staging area, into a location on a disk to allow said primary database to access said archived file; and

uncompressing said archived file to allow said primary database to access said archives file.

19. The method as described in claim 18, wherein said step of restoring said archived file that is recovered from said storage medium further comprises restoring said archived file that is recovered from a backup tape.

20. An apparatus for archiving data, comprising:

means for creating at least one top level directory to store files;

means for creating at least one subdirectory within said top level directory;

means for creating a file-location database to track stored files in said subdirectory and said top level directory;

means for receiving files from a primary database, to archive said files into said top level directory;

means for archiving the primary database files into said top level directory, in response to receiving files from a primary database;

means for creating an immediate backup of the archived primary database files; and

means for creating a long-term backup of said archived primary database files.

21. A system for storing, archiving, and retrieving data, comprising:

a network for transmitting and receiving data;

at least one input system, coupled to said network, for inputting data in at least a first format;

a reformatter, coupled to said network, for receiving said data and reformatting said data into a predetermined format to produce reformatted data;

a primary database, coupled to said network, for storing said reformatted data;

a loader, coupled to said primary database, for loading said reformatted data into said primary database;

at least one client workstation, coupled to said network, for receiving data search requests;

a front end server, coupled to said network, for receiving said primary database search requests and processing said search requests, querying said primary database, receiving reformatted data from said primary database, and outputting said reformatted data;

at least one data archiver, coupled to said primary database, for archiving at least a portion of the data stored in said primary database according to a predetermined archiving algorithm, and for transferring archived data to an archive storage media;

means for creating at least one top-level directory to archive said data stored in said primary database;

means for creating at least one subdirectory within said top level directory;

means for creating a file-location database to track stored files in said subdirectory and said top level directory;

means for receiving said data stored in said primary database, to archive said data said top level directory;

means for archiving the primary database files into said top level directory, in response to receiving files from a primary database;

means for creating an immediate backup of the archived primary database files; and

means for creating a long-term backup of said archived primary database files.

22. A computer readable program storage device encoded with instructions that, when executed by a computer, performs a method for archiving data, comprising:

creating at least one top-level directory to store files;

creating a file-location database to track stored files in said top level directory;

receiving files from a primary database;

archiving the primary database files into said top level directory, in response to receiving files from a primary database;

creating an immediate backup of the archived primary database files; and

creating a long-term backup of said archived primary database files.

23. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, wherein archiving said primary database files further comprises:

creating at least one subdirectory within said top level directory;

creating a file-location database to track stored files in said subdirectory;

receiving files from said primary database;

numbering said subdirectories in said top level database;

assigning said files to be stored in a highest numbered subdirectory;

determining whether said highest numbered subdirectory contains a maximum allowable number of files;

creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains a maximum allowable number of files;

creating a file location database to store name of said file and said subdirectory number; and

entering name of said file and said subdirectory number into said file location database.

24. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 23, wherein creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains a maximum allowable number of files further comprises creating a new subdirectory for file storage, in response to the determination that said highest numbered subdirectory contains one thousand files.

25. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, wherein creating an immediate backup of the archived primary database files further comprises:

creating a backup directory in a backup disk;

creating a second copy of each said archived file;

storing said second copy of said archived file into said backup directory;

copying said backup directory into a backup storage medium at a predetermined frequency of time; and

deleting said files from said backup directory in response to said backup directory being copied into said backup storage medium.

26. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 25, wherein copying said backup directory into a backup storage medium comprises copying said backup directory into a backup tape.

27. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 26, wherein copying said backup directory into a backup storage medium at a predetermined frequency of time further comprises of copying said backup directory into a backup storage medium once every week.

28. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, wherein creating a long-term backup of said archived primary database files further comprises:

determining a period of time that said file has resided in said top level directory;

determining whether said period of time exceeds a maximum residence time;

archiving a subdirectory located in said top level directory and which contains said file, into a backup storage medium in response to said period of time exceeding said maximum residence time;

creating a text file to record identification data relating to said backup storage medium and names of said files in said subdirectory;

creating a backup database, to record identification data relating to said files and said top level directory, on said storage medium; and

deleting said files in said subdirectories in said top level directory in response to said subdirectories being archived into said backup storage medium.

29. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 28, wherein determining whether said period of time exceeds a maximum residence time further comprises determining whether said period of time exceeds six months.

30. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described claim 28, wherein archiving a subdirectory located in said top level directory and which contains said file, into a backup storage medium further comprises archiving a subdirectory located in said top level directory and which contains said file, into a backup tape.

31. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 28, wherein creating a backup database, to record identification data relating to said files and said top level directory further comprises creating a backup database, to record names and number of said files and said top level directory.

32. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, further comprising a method of retrieving said archived files comprising:

receiving a list of said archived files to be restored;

converting said list of archived files to be restored into path file names;

adding said path file names to a restoration list;

executing a restoration process at a predetermined time to restore said archived files to be restored;

determining whether all said archived files to be restored, have been restored;

placing said path file names corresponding to said archived files that are not restored, on an unrestored-file list, in response to a determination that all said archived files are not restored.

33. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 32, further comprising executing said restoration process to retrieve said archived files on said unrestored-file list.

34. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 32, wherein executing a restoration process at a predetermined time further comprises executing a restoration process at a period of time of reduced data access activity.

35. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 34, wherein executing a restoration process at a period of time of reduced data access activity, further comprises executing a restoration process during night time.

36. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 32, wherein executing a restoration process further comprises:

locating said top level directory which contains said subdirectory that holds said archived data that is to be restored;

determining whether said subdirectory that holds said archived file that is to be restored is found in said top level directory;

copying said archived file from said subdirectory in said top level directory, in response to the determination that said subdirectory that holds said archived file is found in said top level directory;

placing said copied file in a restoration location;

uncompressing said copied file in said restoration location to allow said primary database to access said copied file;

adding the name of said archived file to a restoration-pending file list in response to the determination that said subdirectory that holds said archived file is not found in said top level directory; and

invoking a restoration procedure to restore said archived file from a backup storage medium to allow said primary database to access said archived file.

37. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 36, wherein placing said copied file in a restoration location further comprises placing said copied file in a temporary directory.

38. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, wherein invoking a restoration procedure to restore said archived file from a backup storage medium further comprises invoking a restoration procedure to restore said archived file from a backup tape.

39. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 22, wherein invoking a restoration procedure further comprises:

reading a list of said archived files to be restored;

creating an instruction set to recover said archived files to be restored from a storage medium;

executing said instruction set to retrieve said archived files to be restored from said storage medium;

placing in memory storage, the name of said archived file that is not recovered;

restoring said archived file that is recovered from said storage medium, into a staging area;

copying said archived file from said staging area, into a location on a disk to allow said primary database to access said archived file; and

uncompressing said archived file to allow said primary database to access said archived file.

40. The computer readable program storage device encoded with instructions that, when executed by a computer, performs the method described in claim 39, wherein restoring said archived file that is recovered from said storage medium further comprises restoring said archived file that is recovered from a backup tape.