System and method for storing data off site

• US 9,372,809 B2
• Filed: 01/06/2011
• Issued: 06/21/2016
• Est. Priority Date: 01/06/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: StorSimple, Inc. (Microsoft Corporation)
• Inventors: Testardi, Richard, Cometto, Maurilio, Kulangare, Kuriakose George
• Primary Examiner(s): PATEL, KAUSHIKKUMAR M
• Application Number:
  US12/930,502
• Publication Number:
  US 20120173822A1
• Time in Patent Office: 1,993 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 12/0808 with cache invalidating means G06F12/0815 takes precedence
• CPC Class Codes: G06F 12/122 of the least frequently used [LFU] type, e.g. with individual count value
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A system and method for efficiently storing data both on-site and off-site in a cloud storage system. Data read and write requests are received by a cloud data storage system. The cloud storage system has at least three data storage layers. A first high-speed layer, a second efficient storage layer, and a third off-site storage layer. The first high-speed layer stores data in raw data blocks. The second efficient storage layer divides data blocks from the first layer into data slices and eliminates duplicate data slices. The third layer stores data slices at an off-site location.

Claims

1. A data storage system, comprising:

a processor;

a first memory configured as a first storage layer for storing blocks of data;

a second memory configured as a second storage layer for storing segments of data;

a third memory coupled to one or more processors, the memory having program instructions stored thereon that, upon execution by the processors, cause the data storage system to;

receive a storage request from an external system, the storage request including new data to be stored;

select an existing data block from the first storage layer for removal to the second storage layer;

slice the selected existing data block into segments, wherein each segment is uniquely identified by a fingerprint;

identify any segments that are duplicates of segments already stored in the second storage layer, wherein duplicates are identified based on the segment fingerprints;

transfer any non-duplicate segments from the first storage layer to the second storage layer, wherein each transfer of a segment to the second storage layer is mapped in a first storage map by the fingerprint of the segment;

store the new data in the first storage layer, wherein the storage of the new data in the first storage layer is mapped in the first storage map;

select one or more segments from the second storage layer for removal to a third storage layer, wherein the third storage layer is hosted by a bulk storage system; and

transfer the selected segments from the second storage layer to the third storage layer,

wherein each transfer of a selected segment to the third storage layer is mapped in the first storage map by the fingerprint of the selected segment.

2. The data storage system of claim 1, wherein the external system is a Storage Area Network.

3. The data storage system of claim 1, further comprising:

a fourth memory configured storing data snapshots; and

wherein the second memory having further program instructions that, upon execution by the processor, cause the data storage system to

store a snapshot of the new data in the snapshot data storage.

4. The data storage system of claim 1, wherein the first storage map is configured to track each segment transferred to the second storage layer according to its fingerprint.

5. The data storage system of claim 4, wherein the first storage map is further configured to track each segment transferred to the third storage layer according to its fingerprint.

6. The data storage system of claim 5, wherein the mapping of segment transfers to the second storage layer is further maintained in a second storage map, the second storage map configured to track the location each segment is stored in the second storage layer.

7. The data storage system of claim 6, wherein the tracking of the storage location of the segments stored in the second storage layer is according the fingerprint of the segments.

8. The data storage system of claim 1, wherein segment fingerprints are statistically unique values calculated for each slice of the data.

9. The data storage system of claim 4, wherein the second memory having further program instructions that, upon execution by the processor, cause the data storage system to:

track the storage location of the new data stored in the first storage layer in the first data map.

10. The data storage system of claim 6, wherein the bulk storage system used by the third storage layer is cloud storage and wherein the transfer of segments to the third storage layer is maintained in the second storage map, the second storage map configured to track the location each transferred segment is stored in the cloud storage.

11. A method for storing data in a three-tiered data storage system including a first storage layer configured for storing blocks of data, a second storage layer configured for storing segments of data and a third storage layer configured for bulk storage of segments of data, the method comprising:

receiving a storage request from an external system, the storage request including new data to be stored;

selecting an existing data block from the first storage layer for removal to the second storage layer;

slicing the selected existing data block into segments, wherein each segment is uniquely identified by a fingerprint;

identifying any segments that are duplicates of segments already stored in the second storage layer, wherein duplicates are identified based on the segment fingerprints;

transferring any non-duplicate segments from the first storage layer to the second storage layer, wherein each transfer of a segment to the second storage layer is mapped in a first storage map by the fingerprint of the segment;

storing the new data in the first storage layer, wherein the storage of the new data in the first storage layer is mapped in the first storage map;

selecting one or more segments from the second storage layer for removal to a third storage layer, wherein the third storage layer is hosted by a bulk storage system; and

transferring the selected segments from the second storage layer to the third storage layer,

wherein each transfer of a selected segment to the third storage layer is mapped in the first storage map by the fingerprint of the selected segment.

12. The data storage method of claim 11, wherein the external system is a Storage Area Network.

13. The data storage method of claim 11, further comprising:

storing a snapshot of the new data in a snapshot data storage.

14. The data storage method of claim 11, wherein the first storage map is configured to track each segment transferred to the second storage layer according to its fingerprint.

15. The data storage method of claim 14, wherein the first storage map is further configured to track each segment transferred to the third storage layer according to its fingerprint.

16. The data storage method of claim 15, wherein the mapping of segment transfers to the second storage layer is further maintained in a second storage map, the second storage map configured to track the location each segment is stored in the second storage layer.

17. The data storage method of claim 16, wherein the tracking of the storage location of the segments stored in the second storage layer is according the fingerprint of the segments.

18. The data storage method of claim 11, wherein segment fingerprints are statistically unique values calculated for each slice of the data.

19. The data storage method of claim 14, the method further comprising:

tracking the storage location of the new data stored in the first storage layer in the first data map.

20. The data storage method of claim 16, wherein the bulk storage system used by the third storage layer is cloud storage and wherein the transfer of segments to the third storage layer is maintained in the second storage map, the second storage map configured to track the location each transferred segment is stored in the cloud storage.