System and method for restoring data to a storage device based on a backup image

• US 8,504,529 B1
• Filed: 06/19/2009
• Issued: 08/06/2013
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Zheng, Ling, Yuan, Hsing
• Primary Examiner(s): Mofiz, Apu
• Assistant Examiner(s): WALKER, BRYAN L
• Application Number:
  US12/488,468
• Time in Patent Office: 1,509 Days
• Field of Search: 707/679, 707/999.202, 707/686
• US Class Current: 707/679
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06

Abstract

A system for restoring data to a storage server generates a lookup data structure based on a backup image, where the lookup data structure defines a correspondence between individual block identifiers and individual data blocks of a plurality of data blocks in the backup image. The system then generates a directory structure on the storage server based on the backup image. After generating the directory structure, the system generates an empty restored file in the directory structure based on the backup image. The system then populates file contents of the restored file based on the lookup data structure.

Claims

1. A method for restoring data to a storage controller, the method comprising:

receiving a backup image including file system metadata and a plurality of data blocks, wherein the file system metadata includes information defining a plurality of directories and information defining a plurality of data containers, the information defining a plurality of data containers including a listing of data blocks associated with individual data containers of the plurality of data containers;

copying the plurality of data blocks to the storage controller;

generating a data warehouse structure based on the file system metadata, wherein the data warehouse structure stores references to individual data blocks of the plurality of data blocks and wherein the references are indexed by an identifier determined based on the file system metadata;

generating a directory structure in a file system associated with the storage controller based on the file system metadata;

generating a plurality of empty data containers in the generated directory structure based on the file system metadata, wherein generating an empty data container includes generating a data structure and populating the data structure with metadata based on the file system metadata; and

populating data container contents of individual data containers of the plurality of empty data containers by associating individual data blocks of the plurality of data blocks with the individual data containers based on the data warehouse structure, wherein populating data container contents of an individual data container includes determining a volume block identifier associated with a data block in the individual data container from the file system metadata and copying a reference associated with the volume block identifier in the data warehouse structure into the data structure.

2. The method of claim 1, wherein generating the directory structure comprises:

reading first directory definition information associated with a first directory from the file system metadata, the first directory definition information including information specifying a second directory contained in the first directory;

directing an operating system associated with the storage controller to generate the first directory based on the first directory definition information;

reading second directory definition information associated with the second directory from the file system metadata; and

directing the operating system to generate the second directory as a subdirectory of the first directory based on the second directory definition information.

3. The method of claim 1, wherein the file system metadata defines a first data container and a second data container and wherein the listing of data blocks associated with the first data container and the listing of data blocks associated with the second data container include a shared data block.

4. The method of claim 1, wherein the file system metadata defines a first data container and a second data container and wherein the listing of data blocks associated with the first data container and the listing of data blocks associated with the second data container include a shared data block, and wherein populating data container contents of individual data containers comprises:

determining a shared volume block identifier associated with the shared data block;

copying a shared reference associated with the shared volume block identifier into a first data container structure associated with the first data container; and

copying the shared reference into a second data container structure associated with the second data container.

5. A storage server comprising:

a network interface through which to communicate with a backup client over a network;

a storage interface through which to access a backup storage facility;

a storage device;

a memory; and

a processor coupled to the network interface, the storage interface, and the memory, wherein the memory stores code, execution of which by the processor causes the storage server to perform a process which includes;

receiving a backup image through the storage interface, the backup image having a metadata portion and a data warehouse portion, wherein the metadata section includes information defining a plurality of directories and information defining a plurality of data containers, the information defining a plurality of data containers including a listing of data blocks associated with individual data containers of the plurality of data containers;

restoring a plurality of data blocks from the data warehouse portion of the backup image to the storage device;

generating a lookup structure based on the backup image, wherein the lookup structure maps identifiers to individual data blocks of the plurality of data blocks;

generating a directory structure in a file system associated with the storage server based on the metadata portion of the backup image;

generating a data container associated with a file system object defined by the metadata portion, wherein generating a data container includes generating a data structure and populating the data structure with metadata associated with the file system object; and

inserting a data block into the data container by associating the data block with the data container based on the lookup structure, wherein inserting a data block into the data container includes determining a volume block identifier associated with the data block from the metadata portion of the backup image and copying a reference associated with the volume block identifier in the lookup structure into the data structure.

6. The storage server of claim 5, wherein the storage interface comprises a data mover configured to receive data from a tape drive.

7. The storage server of claim 5, wherein generating the data container comprises generating a data structure containing metadata associated with the data container and not including references to data blocks associated with the data container.

8. The storage server of claim 5, wherein the lookup structure is a data warehouse structure created on the storage server, wherein the data warehouse structure stores references to individual data blocks received from the data portion of the backup image, and wherein the references are indexed by a volume block number determined based on the metadata portion of the backup image.

9. The storage server of claim 5, wherein generating the directory structure comprises:

reading multiple directory entries from the metadata portion of the backup image; and

generating a new directory for each directory entry of the multiple directory entries, wherein generating the new directory includes associating the new directory with a parent directory in the directory structure.

10. The storage server of claim 5, wherein the process further comprises:

receiving an updated backup image including an updated metadata portion and an updated data portion, wherein the updated backup image defines an incremental backup based on the backup image;

modifying the lookup structure based on the updated backup image; and

modifying the data container by adding or removing a data block from the data container based on the modified lookup structure.

11. The storage server of claim 5, wherein generating the lookup structure comprises:

generating a warehouse data container on the storage server;

reading multiple data blocks from the backup image;

for individual data blocks of the multiple data blocks, determining a block identifier; and

adding individual data blocks of the multiple data blocks to the warehouse data container, wherein the individual data blocks are added to the warehouse data container such that the associated block identifier is equal to a file block number associated with a position of the individual data block in the warehouse data container.

12. A system for restoring backup data to a storage device, the system comprising:

a storage interface configured to communicate with a storage facility to receive a backup image, wherein the backup image includes a metadata section and a storage data section having a plurality of storage units and wherein the metadata section includes information defining a plurality of directories and information defining a plurality of data containers, the information defining a plurality of data containers including a listing of data blocks associated with individual data containers of the plurality of data containers;

a storage subsystem;

a memory;

a processor;

a block management component configured to copy individual storage units of the plurality of storage units from the storage data section of the backup image to the storage subsystem;

a data warehouse generator component configured to generate a data warehouse lookup structure based on the backup image, wherein the data warehouse lookup structure provides an index mapping a storage unit identifier to an individual storage unit of the plurality of storage units;

a directory processor component configured to generate a restored directory and a restored data container in a file system associated with the storage subsystem based on the backup image, wherein the restored data container is a file system object containing only metadata information and wherein the restored data container is created in the restored directory; and

a data container populate component configured to associate a storage unit from the backup image with the restored data container by associating individual storage units of the plurality of storage units with the restored data container based on the data warehouse lookup structure, wherein associating individual storage units of the plurality of storage units with the restored data container includes determining a volume block identifier associated with a data block in the restored data container from the metadata section of the backup image and copying a reference associated with the volume block identifier in the data warehouse lookup structure into the restored data container.

13. The system of claim 12, wherein generating the restored data container comprises generating a data structure and populating the data structure with data container metadata based on the metadata section.

14. The system of claim 12, wherein the data warehouse lookup structure is a warehouse data container created in the storage subsystem and wherein the warehouse data container includes references to the plurality of data blocks.

15. The system of claim 12, wherein the directory processor component is further configured to:

read multiple directory entries from the metadata section of the backup image; and

generate a new directory for each directory entry of the multiple directory entries, wherein generating the new directory includes associating the new directory with a parent directory in the directory structure.

16. The system of claim 12, wherein the metadata section includes a first listing of data blocks associated with a first data container and a second listing of data blocks associated with a second data container, wherein the first listing and the second listing include a shared data block.

17. The system of claim 12, wherein the storage facility is a data mover receiving the backup image from a tape.

18. A system for restoring backup data to a storage device, the system comprising:

a memory;

a processor;

storage interface means for communicating with a storage facility to receive a backup image, wherein the backup image includes file system metadata and a data portion having a plurality of data blocks, wherein the file system metadata includes information defining a plurality of directories and information defining a plurality of data containers, the information defining a plurality of data containers including a listing of data blocks associated with individual data containers of the plurality of data containers;

restoring means for restoring individual data blocks of the plurality of data blocks from the data portion of the backup image to the storage device;

lookup generation means for generating a lookup data structure based on the backup image, wherein the lookup data structure defines a mapping from individual identifiers in the file system metadata to the individual data blocks of the plurality of data blocks;

directory means for generating a directory structure in a file system associated with the storage facility based on the file system metadata;

empty data container means for generating an empty data container on the storage facility based on the file system metadata, wherein the empty data container means includes data structure means for generating a data structure and populating the data structure with metadata based on the file system metadata; and

data container populating means for populating the empty data container by associating individual data blocks of the plurality of data blocks with the empty data container based on the lookup data structure, wherein the data container populating means includes identifier determining means for determining a volume block identifier associated with a data block in the empty data container from the file system metadata and copying a reference associated with the volume block identifier in the lookup data structure into the data structure.

19. One or more computer-readable memories storing computer-executable instructions for implementing a method for restoring data to a storage server, comprising:

instructions for receiving a backup image through a storage interface, the backup image having a metadata portion and a data warehouse portion, wherein the metadata portion includes information defining a plurality of directories and information defining a plurality of data containers, the information defining a plurality of data containers including a listing of data blocks associated with individual data containers of the plurality of data containers;

instructions for restoring a plurality of data blocks from the data warehouse portion of the backup image to a storage subsystem associated with the storage server;

instructions for generating a lookup structure based on the metadata portion of the backup image, wherein the lookup structure maps identifiers to individual data blocks of the plurality of data blocks and wherein the identifiers are determined based on the metadata portion of the backup image;

instructions for generating a directory structure in a file system associated with the storage server based on the metadata portion of the backup image;

instructions for generating a plurality of empty data containers in the directory structure based on the metadata portion of the backup image, wherein generating the plurality of empty data containers includes generating a data structure and populating the data structure with metadata based on the metadata portion of the backup image; and

instructions for populating data container contents of individual data containers of the plurality of empty data containers by associating individual data blocks of the plurality of data blocks with the individual data containers based on the lookup structure, wherein populating the data container contents of an individual data container includes determining a volume block identifier associated with a data block in the individual data container from the metadata portion of the backup image and copying a reference associated with the volume block identifier in the lookup structure into the data structure.

20. The one or more computer-readable memories of claim 19, wherein the lookup data structure is a data warehouse data container created in the file system associated with the storage server, wherein the warehouse data container stores references to individual data blocks of the plurality of data blocks, and wherein the references are indexed by a volume block number determined based on the metadata portion of the backup image.

21. The one or more computer-readable memories of claim 19, wherein instructions for generating a directory structure further comprises:

instructions for reading multiple directory entries from the metadata portion of the backup image; and

instructions for generating a new directory for each directory entry of the multiple directory entries, wherein generating the new directory includes associating the new directory with a parent directory in the directory structure.

22. The one or more computer-readable memories of claim 19, wherein the backup image is stored on a sequential medium, the non-transitory computer-readable storage medium further comprising instructions to generate a metadata map structure based on the metadata portion of the backup image, the metadata map structure being accessible for random access.

23. The one or more computer-readable memories of claim 19, wherein the metadata portion of the backup image includes first data container information defining a first data container and second data container information defining a second data container, the first data container information including first data container metadata and a first listing of data blocks and the second data container information including second data container metadata and a second listing of data blocks, wherein the first listing and the second listing include a shared data block, wherein generating the plurality of empty data containers comprises generating a first empty data container based on the first data container information and generating a second empty data container based on the second data container information, and wherein populating data container contents comprises:

determining a shared reference associated with the shared data block;

copying the shared reference into the first empty data container; and

copying the shared reference into the second empty data container.

24. The one or more computer-readable memories of claim 19, further comprising:

instructions for receiving an updated backup image including updated system metadata and a plurality of updated data blocks, wherein the updated backup image defines an incremental backup based on the backup image;

instructions for modifying the lookup data structure based on the updated metadata; and

instructions for modifying the data container contents based on the modified lookup data structure.

25. The one or more computer-readable memories of claim 19, wherein instructions for generating a lookup data structure further comprises:

instructions for generating a warehouse data container on the storage server;

instructions for reading multiple data blocks from the backup image;

for individual data blocks of the multiple data blocks, instructions for determining a block identifier; and

instructions for adding individual data blocks of the multiple data blocks to the warehouse data container, wherein the individual data blocks are added to the warehouse data container such that the associated block identifier is equal to a file block number associated with a position of the individual data block in the warehouse data container.

26. The one or more computer-readable memories of claim 19, wherein the backup image is received from a tape.