Storage architecture for backup application

• US 8,868,882 B2
• Filed: 06/08/2011
• Issued: 10/21/2014
• Est. Priority Date: 06/08/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ali, Abid, Singla, Amit, Khanzode, Pankaj, Sen, Vijay, Prabhu, Vanita, Durge, Sachin
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): DOAN, HAN V
• Application Number:
  US13/155,401
• Publication Number:
  US 20120317379A1
• Time in Patent Office: 1,231 Days
• US Class Current: 711/206
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems

Abstract

Aspects of the subject matter described herein relate to a storage architecture. In aspects, an address provided by a data source is translated into a logical storage address of virtual storage. This logical storage address is translated into an identifier that may be used to store data on or retrieve data from a storage system. The address space of the virtual storage is divided into chunks that may be streamed to the storage system.

Claims

1. A method implemented at least in part by a computer, the method comprising:

receiving an indication of a virtual storage address space;

dividing the virtual storage address space into fixed-sized chunks;

receiving data from a data source;

receiving address information associated with the data;

determining at least one of the chunks based on the address information and a block address translation table that maps from blocks of a file system volume to physical addresses of a storage device;

providing the data corresponding to the at least one of the chunks to a storage system;

updating a data structure to map from the address information to an identifier that identifies the at least one of the chunks to the storage system;

merging a first backup set and a second backup set stored on the storage system by sending a message to delete one or more blocks of the first backup set to the storage system and sending metadata to the storage system that indicates that other of the blocks of the first backup set are to be included in the second backup set; and

generating metadata regarding a backup dataset that includes the at least one of the chunks and providing the metadata regarding the backup dataset to the storage system prior to sending the at least one of the chunks, the metadata regarding the backup dataset indicating a location of the at least one of the chunks, the metadata regarding the backup dataset is used for obtaining the at least one chunk from the backup dataset even if other data of the backup dataset is not stored on the storage system.

2. The method of claim 1, wherein receiving an indication of a virtual storage address space comprises receiving a range of addresses corresponding to the virtual storage address space.

3. The method of claim 2, wherein updating a data structure to map from the address to an identifier comprises updating a first field and a second field of a row of a table, the updating the first field comprising storing the range of addresses in the first field, the updating the second field comprising placing a second range of addresses corresponding to storage of the storage system in which the data is or will be stored in the second field, the second range of addresses corresponding to the identifier.

4. The method of claim 1, wherein receiving address information associated with the data comprises receiving a virtual storage offset and a length of the data.

5. The method of claim 1, further comprising dividing the at least one of the chunks into smaller chunks and performing a de-duplication using the smaller chunks prior to providing the data to the storage system.

6. The method of claim 1, further comprising compressing the at least one of the chunks prior to providing the data to the storage system.

7. The method of claim 1, further comprising encrypting the at least one of the chunks prior to providing the data to the storage system.

8. A computer storage medium having computer-executable instructions, which when executed perform actions, comprising:

receiving, from a requestor, a request for data that is stored on a storage system;

receiving a first address of the data, the first address in a first address space, the first address indicating a storage location the requestor associates with the data;

determining a second address based on the first address and a data structure, the second address indicating a logical storage offset of a virtual storage address space that is divided into chunks that are mapped to storage locations of the storage system, the data structure provides mapping information between the first address space and the virtual storage address space;

mapping the second address to a third address, the third address identifying a storage location of at least one of the chunks corresponding to the data on the storage system;

retrieving the at least one of the chunks from the storage system using the third address;

providing the data to the requestor, wherein a first backup set and a second backup set stored on the storage system are merged by sending a message to delete one or more blocks of the first backup set to the storage system and sending metadata to the storage system that indicates that other of the blocks of the first backup set are to be included in the second backup set; and

generating metadata regarding a backup dataset that includes the at least one of the chunks and providing the metadata regarding the backup dataset to the storage system prior to sending the at least one of the chunks, the metadata regarding the backup dataset indicating a location of the at least one of the chunks, the metadata regarding the backup dataset is used for obtaining the at least one chunk from the backup dataset even if other data of the backup dataset is not stored on the storage system.

9. The computer storage medium of claim 8, further comprising performing a transformation on the at least one of the chunks data prior to providing the data to the requestor.

10. The computer storage medium of claim 9, wherein performing a transformation on the at least one of the chunks data prior to providing the data to the requestor comprises decrypting the at least one of the chunks.

11. The computer storage medium of claim 9, wherein performing a transformation on the at least one of the chunks data prior to providing the data to the requestor comprises decompressing the at least one of the chunks.

12. The computer storage medium of claim 9, wherein performing a transformation on the at least one of the chunks data prior to providing the data to the requestor comprises reversing a de-duplication previously applied to the at least one of the chunks.

13. The computer storage medium of claim 9, wherein performing a transformation on the at least one of the chunks data prior to providing the at least one of the chunks to the requestor comprises performing the transformation on a device that hosts the requestor.

14. The computer storage medium of claim 9, wherein performing a transformation on the at least one of the chunks prior to providing the data to the requestor comprises performing the transformation on a device that is part of the storage system.

15. In a computing environment, a system, comprising:

a data source operable to provide data to a source reader, the data source further operable to provide to the source reader a first address that the data source associates with the data; and

a computer comprising the source reader operable to create a virtual storage address space and to divide the virtual storage address space into fixed-sized chunks, the source reader further operable to use a block address translation table that maps from block of a file system volume to physical addresses of a storage device, the source reader further operable to determine at least one of the chunks based on the first address and a length of the data, the source reader further operable to stream the at least one of the chunks corresponding to the data to a storage system, the source reader further operable to maintain a data structure that maps the at least one of the chunks to a storage identifier of the storage system, wherein a first backup set and a second backup set stored on the storage system are merged by sending a message to delete one or more blocks of the first backup set to the storage system and sending metadata to the storage system that indicates that other of the blocks of the first backup set are to be included in the second backup set, the source reader further operable to generate metadata regarding a backup dataset that includes the at least one of the chunks and providing the metadata regarding the backup dataset to the storage system prior to sending the at least one of the chunks, the metadata regarding the backup dataset indicating a location of the at least one of the chunks, the metadata regarding the backup dataset is used for obtaining the at least one chunk from the backup dataset even if other data of the backup dataset is not stored on the storage system.

16. The system of claim 15, wherein the source reader is further operable to divide a fixed-sized chunk into smaller chunks in preparation for performing a de-duplication transformation using the smaller chunks.

17. The system of claim 15, wherein the source reader is further operable to compress the at least one of the chunks prior to streaming the at least one of the chunks to the storage system.

18. The system of claim 15, wherein the source reader is further operable to encrypt the at least one of the chunks prior to streaming the at least one of the chunks to the storage system.

19. The system of claim 15, the data source further operable to provide an incremental backup by provide data and metadata that describes a change to a backup set, the source reader further operable to stream the data and metadata that describes the change in one or more chucks to the storage system.