Computer data storage physical backup and logical restore

• US 6,366,987 B1
• Filed: 08/13/1998
• Issued: 04/02/2002
• Est. Priority Date: 08/13/1998
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Tzelnic, Percy, Dunham, Scott R.
• Primary Examiner(s): Yoo, Do Hyun
• Assistant Examiner(s): NAMAZI, MEHDI
• Application Number:
  US09/133,701
• Time in Patent Office: 1,328 Days
• Field of Search: 711/161-162, 711/165, 711/112, 711/114
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence

Abstract

A data storage system is constructed to rapidly respond to a physical backup request for backup of a physical storage unit such as a disk volume or cylinder. A user, however, may request backup of a logical data structure such as a file having a rather complex mapping to a number of physical storage components such as a number of disk tracks. For backup and restore of such a logical data structure, a backup agent requests an application interface routine to backup a physical storage unit that includes the logical data structure. The application interface routine issues a corresponding physical backup request to the data storage system, and creates a catalog of information about content of the physical storage unit that would be needed for restoring the logical data structure from the backup version of the physical storage unit. Later, if and when the backup agent requests the restoration of the logical data structure, the application interface routine looks up the logical data structure in the catalog, issues a physical restore request to the data storage system to retrieve a backup version of the physical storage unit from backup data storage and load it into spare data storage, extracts the logical data structure from the physical storage unit in the spare data storage by performing a logical-to-physical translation, and restores the logical data structure into the current version of data storage.

Claims

1. In a data processing system including a primary data storage subsystem and a secondary data storage subsystem, a method of backing up and restoring a file, said method comprising:

a) in response to a back-up request specifying the file, selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem for backing up the storage unit by copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem, and

b) in response to a restore request specifying a particular version of the file contained in the backup version of the storage unit in the secondary storage in the secondary data storage subsystem, issuing a restore command to the primary data storage subsystem for fetching the backup version of the storage unit containing the particular version of the file and writing the backup version of the storage unit from the secondary storage to spare data storage, and extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the primary data storage subsystem.

2. The method as claimed in claim 1, wherein the primary data storage subsystem responds to the backup command by creating an instant snapshot copy of the storage unit, and servicing additional commands for access to the primary storage concurrently while copying the storage unit from the primary storage to the secondary data storage subsystem.

3. The method as claimed in claim 1, which includes selecting the storage unit containing the file by selecting from a storage unit hierarchy a smallest storage unit containing the file.

4. The method as claimed in claim 1, which includes responding to the request for backup of the file by creating a catalog of information about files contained in the storage unit.

5. The method as claimed in claim 4, which includes responding to the request for restoring the file by accessing the catalog of information about files contained in the storage unit, in order to obtain information about the file, and using the information about the file for restoring the file from the backup version of the storage unit.

6. The method as claimed in claim 1, which includes the secondary data storage subsystem compacting the storage unit for storage in the secondary storage by not writing, to the secondary storage, a contiguous unused portion of the storage unit received from the primary data storage subsystem.

7. The method as claimed in claim 6, wherein the contiguous unused portion of the storage unit includes unused file space at the end of a file system.

8. The method as claimed in claim 1, which includes a backup software process receiving the back-up and restore requests from a user or application program process, and which includes a backup agent process acting as an interface between the backup software process and sending the backup and restore commands to the primary data storage subsystem.

9. The method as claimed in claim 1, which includes bundling requests for backup of related files so that a set of related files is backed up in one version of a storage unit containing the set of related files.

10. The method as claimed in claim 1, which includes bundling requests for restoration of related files so that a set of related files contained in a version of a storage unit is restored by being copied out of one copy of the version of the storage unit stored in the spare data storage.

11. In a data processing system including a primary data storage subsystem and a secondary data storage subsystem, a method of backing up and restoring a file, said method comprising:

a) in response to a back-up request specifying the file, selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem for backing up the storage unit by copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem, and

b) in response to a restore request specifying a particular version of the file contained in the backup version of the storage unit in the secondary storage in the secondary data storage subsystem, issuing a restore command to the primary data storage subsystem for fetching the backup version of the storage unit containing the particular version of the file and writing the backup version of the storage unit from the secondary storage to spare data storage, and extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the primary data storage subsystem;

which includes the secondary data storage subsystem performing a background compaction process which includes deleting from the storage unit files that were not requested to be backed up.

12. In a data processing system including a primary data storage subsystem and a secondary data storage subsystem, a method of backing up a file, said method comprising:

a) in response to a back-up request specifying the file, selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem;

b) in response to receipt of the backup command, the primary data storage subsystem copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem; and

c) in response to receipt of the backup version of the storage unit from the primary data storage subsystem, the secondary data storage subsystem compacting the storage unit for storage in the secondary storage by not writing, to the secondary storage, a contiguous unused portion of the storage unit received from the primary data storage subsystem.

13. The method as claimed in claim 12, wherein the primary data storage subsystem responds to the backup command by creating an instant snapshot copy of the storage unit, and by servicing additional commands for access to the primary storage concurrently while copying the storage unit from the primary storage to the secondary data storage subsystem.

14. The method as claimed in claim 12, which includes selecting the storage unit containing the file by searching a storage unit hierarchy for a smallest storage unit containing the file.

15. The method as claimed in claim 12, wherein the contiguous unused portion of the storage unit includes unused file space at the end of a file system.

16. The method as claimed in claim 12 which includes bundling requests for backup of related files so that a set of related files is backed up in one version of a storage unit containing the set of related files.

17. In a data processing system including a primary data storage subsystem and a secondary data storage subsystem, a method of backing up a file, said method comprising:

a) in response to a back-up request specifying the file, selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem;

b) in response to receipt of the backup command, the primary data storage subsystem copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem; and

c) in response to receipt of the backup version of the storage unit from the primary data storage subsystem, the secondary data storage subsystem compacting the storage unit for storage in the secondary storage by not writing, to the secondary storage, a contiguous unused portion of the storage unit received from the primary data storage subsystem;

which includes the secondary data storage subsystem performing a background compaction process which includes deleting from the storage unit files that were not requested to be backed up.

18. In a data processing system including a primary data storage subsystem and a secondary data storage subsystem, a method of backing up and restoring a file, said method comprising:

a) in response to a back-up request for backing up the file, selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem for backing up the storage unit by copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem; and

creating a catalog of information about files contained in the storage unit; and

b) in response to a restore request for restoring the file, accessing the catalog of information about files contained in the storage unit, in order to obtain information about the file, and using the information about the file for restoring the file from the backup version of the storage unit.

19. The method as claimed in claim 18, wherein the primary data storage subsystem responds to the backup command by creating an instant snapshot copy of the storage unit, and by servicing additional commands for access to the primary storage concurrently while copying the storage unit from the primary storage to the secondary data storage subsystem.

20. The method as claimed in claim 18, which includes selecting the storage unit containing the file by searching a storage unit hierarchy for a smallest storage unit containing the file.

21. In a data processing system including a data storage system having primary storage and secondary storage for storing backup versions of files stored in the primary storage, a method of backing up a storage unit containing a file and restoring the file from the backed-up storage unit, said method comprising:

a) sending a backup command to the data storage subsystem for backing up the storage unit by copying a current version of the storage unit from the primary storage to the secondary storage to create a backup version of the storage unit in the secondary storage; and

creating a catalog of information about files contained in the backup version of the storage unit; and

b) restoring the file by accessing the catalog to find the information about the file, and using the information about the file in a restoration process including issuing a restore command to the data storage system for writing the backup version of the storage unit from secondary storage to spare data storage in the data storage system, and extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage.

22. The method as claimed in claim 21, which includes a backup software process receiving backup and restore requests from a user or application program process, and which includes a backup agent process acting as an interface between the backup software process and sending the backup and restore commands to the data storage system.

23. The method as claimed in claim 21, which includes bundling requests for backup of related files so that a set of related files is backed up in one version of a storage unit containing the set of related files.

24. The method as claimed in claim 21, which includes bundling requests for restoration of related files so that a set of related files contained in a version of a storage unit is restored by being copied out of one copy of the version of the storage unit stored in the spare data storage of the data storage system.

25. A machine readable program storage device containing a computer program that is executable by a host computer in a data processing system including a primary data storage subsystem and a secondary data storage subsystem for backup and restoration of a file,

said computer program being executable, in response to a backup request specifying the file, for selecting a storage unit containing the file, and sending a backup command to the primary data storage subsystem for backing up the storage unit by copying a current version of the storage unit from primary storage in the primary data storage subsystem to the secondary data storage subsystem to create a backup version of the storage unit in secondary storage in the secondary data storage subsystem;

and

said computer program being executable, in response to a restore request specifying the file, for issuing a restore command to the primary data storage subsystem for writing the backup version of the storage unit from the secondary storage to spare data storage, and extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the primary data storage subsystem.

26. The machine readable program storage device as claimed in claim 25, wherein said computer program is executable in response to the back-up request for selecting a storage unit containing the file by selecting from a storage unit hierarchy a smallest storage unit containing the file.

27. The machine readable program storage device as claimed in claim 25, wherein said computer program is further executable in response to the back-up request for creating a catalog of information about files contained in the storage unit.

28. The machine readable program storage device as claimed in claim 25, wherein said computer program is executable in response to the back-up request for accessing the catalog of information about files contained in the storage unit, in order to obtain information about the file, and using the information about the file for restoring the file from the backup version of the storage unit.

29. The machine readable program storage device as claimed in claim 25, wherein said computer program is executable for bundling requests for backup of related files so that a set of related files is backed up in one version of a storage unit containing the set of related files.

30. The machine readable program storage device as claimed in claim 25, wherein said computer program is executable for bundling requests for restoration of related files so that a set of related files contained in a version of a storage unit is restored by being copied out of one copy of the version of the storage unit stored in the spare data storage.

31. The machine readable program storage device as claimed in claim 25, wherein said computer program includes a backup agent program which is executable for acting as an interface between a backup software program and the primary data storage subsystem, wherein the backup agent program is called by the backup software program for performing backup and restore services by sending the backup and restore commands to the primary data storage subsystem.

32. A machine readable program storage device containing a computer program that is executable by a host computer in a data processing system having a data storage system for backing up a storage unit containing a file and restoring the file from the backed-up storage unit,

a) said computer program being executable for sending a backup command to the data storage system for backing up the storage unit by copying a current version of the storage unit from primary storage to secondary storage to create a backup version of the storage unit in the secondary storage, and creating a catalog of information about files contained in the storage unit;

and

b) said computer program being executable for restoring the file by accessing the catalog to find the information about the file, and using the information in the catalog in a restoration process including issuing a restore command to the data storage system for writing the backup version of the storage unit from the secondary storage to spare data storage in the data storage system, and extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the data storage system.

33. The machine readable program storage device as claimed in claim 32, wherein said computer program is executable for bundling requests for backup of related files so that a set of related files is backed up in one version of a storage unit containing the set of related files.

34. The machine readable program storage device as claimed in claim 32, wherein said computer program is executable for bundling requests for restoration of related files so that a set of related files contained in a version of a storage unit is restored by being copied out of one copy of the version of the storage unit stored in spare data storage of the data storage system.

35. The machine readable program storage device as claimed in claim 32, wherein said computer program includes a backup agent program which is executable for acting as an interface between a backup software program and the data storage system, and the backup agent program is called by the backup software program for performing backup and restore services by sending the backup and restore commands to the data storage system.

36. A data processing system comprising, in combination:

a host computer;

a primary data storage subsystem linked to the host computer to provide primary storage for programs executed on said host computer; and

a secondary data storage subsystem linked to the primary data storage subsystem for receiving and storing backup data from the primary data storage subsystem;

wherein the primary data storage subsystem is programmed to respond to a backup command from the host computer for transferring a copy of a specified storage unit from the primary storage to the secondary data storage subsystem for storing a backup copy of the storage unit in the secondary storage;

wherein the primary data storage subsystem is programmed to respond to a restore command from the host computer for transferring a specified backup copy of a storage unit from the secondary storage to spare data storage; and

wherein the host computer is programmed to respond to a request from a user or application program for backup of a specified file by issuing a backup command to the primary data storage subsystem for backup of a storage unit containing the file, and the host computer is programmed to respond to a request from the user or application program for restoration of the file by issuing a restore command to the primary data storage subsystem for transfer of a specified backup copy of the storage unit containing the file from the secondary storage to the spare data storage, and for extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the primary data storage subsystem.

37. The data processing system as claimed in claim 36, wherein the host is linked to the primary data storage subsystem by a data network including a multiplicity of hosts having access to the primary data storage subsystem.

38. The data processing system as claimed in claim 36, which includes more than one primary data storage subsystem linked to the data network and having a dedicated data link to the secondary data storage subsystem for transfer of backup data to the secondary data storage subsystem.

39. The data processing system as claimed in claim 36, wherein the host computer is programmed to also respond to the request from the user or application program for backup of the specified file by creating a catalog of information about files contained in the storage unit that contains the file, and wherein the host computer is also programmed to respond to the request from the user or application program for restoration of the file by using the information about the file for the restoring of the file.

40. The data processing system as claimed in claim 36, wherein the programming for the host computer includes backup software for receiving backup and restore requests from the user or application program, and a backup agent called by the backup software for issuing backup and restore commands to the primary data storage subsystem.

43. The data processing system as claimed in claim 36, wherein the host computer is programmed to bundle user or application program requests for backup of related files in order to satisfy requests for backup of related files by issuing one backup command to the primary data storage subsystem for backup of one storage unit containing the related files.

41. A data processing system comprising, in combination:

a host computer;

a primary data storage subsystem linked to the host computer to provide primary storage for programs executed on said host computer; and

a secondary data storage subsystem linked to the primary data storage subsystem for receiving and storing backup data from the primary data storage subsystem;

wherein the primary data storage subsystem is programmed to respond to a backup command from the host computer for transferring a copy of a specified storage unit from the primary storage to the secondary data storage subsystem for storing a backup copy of the storage unit in the secondary storage;

wherein the primary data storage subsystem is programmed to respond to a restore command from the host computer for transferring a specified backup copy of a storage unit from the secondary storage to spare data storage; and

wherein the host computer is programmed to respond to a request from a user or application program for backup of a specified file by issuing a backup command to the primary data storage subsystem for backup of a storage unit containing the file, and the host computer is programmed to respond to a request from the user or application program for restoration of the file by issuing a restore command to the primary data storage subsystem for transfer of a specified backup copy of the storage unit containing the file from the secondary storage to the spare data storage, and for extracting the file from the spare data storage by performing a logical-to-physical translation to identify storage components of the file in the spare data storage, and by copying the storage components of the file from the spare data storage to create a current version of the file in the primary storage of the primary data storage subsystem;

wherein the primary data storage subsystem is programmed for performing an instant remote copy function permitting backup data to be copied from the primary storage to the secondary data storage subsystem as a background process while concurrently servicing host requests for access to the primary storage, and wherein the secondary data storage subsystem is programmed for compacting the backup data received from the primary data storage subsystem and storing compacted backup data in the secondary storage.

42. The data processing system as claimed in claim 41, wherein the secondary data storage subsystem is programmed to perform a compaction process of eliminating unused contiguous data storage in end portions of storage units received from the primary data storage subsystem, and also a background compaction process of deleting from secondary storage files that are contained in physical data units received from the primary data storage subsystem and that a user or application program did not request to be backed up.