Preserving temporal locality while multiplexing streams in a stream-informed segment layout (SISL) system

• US 10,387,425 B1
• Filed: 06/30/2016
• Issued: 08/20/2019
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Jenkins, Fani, Goswami, Sudhanshu, Kamat, Mahesh
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): Morris, John J
• Application Number:
  US15/198,140
• Time in Patent Office: 1,146 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 2201/81 Threshold

Abstract

Embodiments are directed to a method for preserving temporal locality in disk-based backup systems by receiving a plurality of save streams each comprising a data stream and a metadata stream, directing the data streams to a plurality of respective data containers, and the metadata streams to a single shared metadata container, and assigning metadata streams using the shared metadata container to their own respective compression region to preserve locality at the compression region level.

Claims

1. A computer-implemented method for preserving temporal locality in a disk-based backup system, comprising:

receiving a plurality of save streams each comprising a data stream and a metadata stream;

directing the data streams to a plurality of respective data containers, and the metadata streams to a single shared metadata container; and

assigning metadata streams of the plurality of data containers to their own respective compression regions in the single shared metadata container to preserve locality at the compression region level, wherein the shared metadata container comprises a metadata page list, a compression region list, and separate page list and compression regions for the plurality of data containers as differentiated by respective save stream regions.

2. The method of claim 1 wherein metadata in each metadata stream defines references to the stored unique data segments contained in associated data stream, the method further comprising:

keeping the single shared metadata container in a separate double-ended queue with other metadata containers;

maintaining, for each metadata container, a count of streams that share a metadata container;

keeping track of a number of write streams that are open; and

using the count of streams and the number of write streams to determine an average number of streams per metadata container in order to maintain uniform sharing of metadata containers across the system using the double-ended queue.

3. The method of claim 1 wherein the containers comprise data structures stored in non-volatile RAM (NVRAM) space.

4. The method of claim 3 wherein the save streams are managed in a Stream-Informed Segment Layout (SISL) system.

5. The method of claim 1 further comprising creating a new shared metadata container when a first shared metadata container is full.

6. The method of claim 5 further comprising:

separating the data streams from the metadata streams in each save stream; and

combining the metadata streams in a multiplexer to direct the metadata streams to the shared metadata container.

7. The method of claim 5 further comprising:

directing the data and metadata streams of the plurality of save streams to a first single container;

defining a maximum number of streams contained in the single container;

merging the metadata streams when the maximum number of streams is exceeded; and

saving the merged metadata streams to the single shared metadata container.

8. The method of claim 1 wherein the disk-based backup system comprises a Data Domain Operating System deduplication backup system.

9. A system for preserving temporal locality in a disk-based backup system, comprising:

an interface receiving a plurality of save streams each comprising a data stream and a metadata stream;

a first component separating the data stream and metadata stream in each save stream into separate streams;

a transmission component directing the data streams to a plurality of respective data containers;

a multiplexer combining and transmitting the metadata streams to a single shared metadata container; and

a second component assigning metadata streams of the plurality of data containers to their own respective compression regions in the single shared metadata container to preserve locality at the compression region level, wherein the shared metadata container comprises a metadata page list, a compression region list, and separate page list and compression regions for the plurality of data containers as differentiated by respective save stream regions.

10. The system of claim 9 wherein metadata in each metadata stream defines references to the stored unique data segments contained in associated data stream, the second component further keeping the single shared metadata container in a separate double-ended queue with other metadata containers;

maintaining, for each metadata container, a count of streams that share a metadata container;

keeping track of a number of write streams that are open; and

using the count of streams and the number of write streams to determine an average number of streams per metadata container in order to maintain uniform sharing of metadata containers across the system using the double-ended queue.

11. The system of claim 10 wherein the containers comprise data structures stored in non-volatile RAM (NVRAM) space.

12. The system of claim 11 wherein the save streams are managed in a Stream-Informed Segment Layout (SISL) system.

13. The system of claim 12 further comprising a component directing the data and metadata streams of the plurality of save streams to a first single container, defining a maximum number of streams contained in the single container, merging the metadata streams when the maximum number of streams is exceeded, and saving the merged metadata streams to the single shared metadata container.

14. The method of claim 9 wherein the disk-based backup system comprises a Data Domain Operating System deduplication backup system.

15. The system of claim 12 wherein the compression region is implemented using a page list of data pages.

16. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to perform data backups by executing instructions implementing a method comprising:

receiving a plurality of save streams each comprising a data stream and a metadata stream;

directing the data streams to a plurality of respective data containers, and the metadata streams to a single shared metadata container;

maintaining a unique page list for each metadata stream; and

assigning metadata streams of the plurality of data containers to their own respective compression regions in the single shared metadata container to preserve locality at the compression region level, wherein the shared metadata container comprises a metadata page list, a compression region list, and separate page list and compression regions for the plurality of data containers as differentiated by respective save stream regions.