System and method for record level multiplexing of backup data to storage media

• US 9,244,927 B1
• Filed: 09/30/2008
• Issued: 01/26/2016
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ravan, Daniel
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): Cheung, Hubert
• Application Number:
  US12/242,418
• Time in Patent Office: 2,674 Days
• Field of Search: 707/640, 707/642
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446

Abstract

A system and method for efficiently creating a backup volume that is also efficient to recover is described. The backup volume may be created with one save set per media record by utilizing record level multiplexing rather than chunk level multiplexing. Through the use of multiple circular buffers, with each buffer associated with a respective save set, save sets written to storage media are contiguous and individual save set chunks are not separated by chunk headers within the respective media record. Recovery is therefore more efficient since the step of reading chunk headers and locating all the chunks for a save set is no longer required.

Claims

1. A method for storing data comprising:

receiving, by a server, data chunks over a network connection, the data chunks being subdivisions of a save set of a plurality of save sets, each save set originating from a different device;

creating, on the server, a plurality of circular buffers for each of the plurality of save sets, each circular buffer only including data for a corresponding save set;

storing, on the server, received data chunks of a first save set in a first circular buffer of the plurality of circular buffers, wherein the received data chunks of the first save set are not separated by chunk headers within the first circular buffer;

creating, by the server, a record header associated with all the data chunks of the first save set, the record header including metadata about the device from which the first save set originated;

storing, on the server, received data chunks of a second save set in a second circular buffer of the plurality of circular buffers, wherein the received data chunks of the second save set are not separated by chunk headers within the second circular buffer; and

in response to all of the data chunks of the first save set being stored in the first circular buffer, writing the received data chunks of the first save set and the record header to a first media record on a storage media before writing any portion of the second save set to a second media record on the storage media, the storage media being coupled to the server and comprising backup data for a plurality of client devices, the data chunks of the first save set and the record header being stored together in the first media record.

2. The method of claim 1, wherein the storage media is tape.

3. The method of claim 1, wherein the storage media is disk.

4. The method of claim 1 wherein all the data chunks of the first save set can be recovered from the first media record, and all the data chunks of the second save set can be recovered from the second media record.

5. The method of claim 1 wherein the first media record includes data from a first client and excludes data from a second client.

6. The method of claim 1 comprising:

upon receiving a request to recover the first save set, locating on the first media record each data chunk of the plurality of data chunks from the first save set.

7. The method of claim 1 comprising:

after the writing the contents of the first circular buffer to a first media record, upon receiving a request to recover the first save set, reading the record header associated with the first save set;

streaming a first data chunk written to the first media record in response to the request to recover the first save set; and

streaming a second data chunk written to the first media record after the streaming the first data chunk, wherein the first data chunk and the second data chunk are from the first save set, the first data chunk immediately follows the first record header within the first media record on the storage media, and the second data chunk immediately follows the first data chunk on the storage media.

8. A method for storing data comprising:

receiving, by a server, data chunks over a network connection, the data chunks being subdivisions of a save set of a plurality of save sets, each save set originating from a different device;

creating, on the server, a plurality of circular buffers for each of the plurality of save sets, each circular buffer only including data for a corresponding save set;

storing, on the server, received data chunks of a first save set and associated record metadata for the first save set in a first circular buffer of the plurality of circular buffers, wherein the received data chunks of the first save set are not separated by chunk headers within the first circular buffer;

creating, by the server, a record header associated with all the data chunks of the first save set, the record header including metadata about the device from which the first save set originated;

storing, on the server, received data chunks of a second save set and associated record metadata for the second save set in a second circular buffer of the plurality of circular buffers, wherein the received data chunks of the second save set are not separated by chunk headers within the second circular buffer; and

in response to all of the data chunks of the first save set being stored in the first circular buffer, writing the received data chunks of the first save set and the record header to storage media before writing any portion of the second save set, the storage media being coupled to the server and comprising backup data for a plurality of client devices, the data chunks of the first save set and the record header being stored together in the first media record.

9. The method of claim 8, wherein the storage media is tape.

10. The method of claim 8, wherein the storage media is disk.

11. A non-transitory computer readable medium containing computer readable instructions for storing data, the computer readable instructions comprising:

computer program code for receiving, by a server, data chunks over a network connection, the data chunks being subdivisions of a save set of a plurality of save sets, each save set originating from a different device;

computer program code for creating, on the server, a plurality of circular buffers for each of the plurality of save sets, each circular buffer only including data for a corresponding save set;

computer program code for storing, on the server, received data chunks of a first save set in a first circular buffer of the plurality of circular buffers, wherein the received data chunks of the first save set are not separated by chunk headers within the first circular buffer;

computer program code for creating, by the server, a record header associated with all the data chunks of the first save set, the record header including metadata about the device from which the first save set originated;

computer program code for storing, on the server, received data chunks of a second save set in a second circular buffer of the plurality of circular buffers, wherein received data chunks of the second save set are not separated by chunk headers within the first circular buffer; and

computer program code for, in response to all of the data chunks of the first save set being stored in the first circular buffer, writing the received data chunks of the first save set and the record header to storage media before writing any portion of the second save set, the storage media being coupled to the server and comprising backup data for a plurality of client devices, the data chunks of the first save set and the record header being stored together in the first media record.

12. A non-transitory computer readable medium containing computer readable instructions for storing data, the computer readable instructions comprising:

computer program code for receiving, by a server, data chunks over a network connection, the data chunks being subdivisions of a save set of a plurality of save sets, each save set originating from a different device;

computer program code for creating, on the server, a plurality of circular buffers for each of the plurality of save sets, each circular buffer only including data for a corresponding save set;

computer program code for storing, on the server, received data chunks of a first save set and metadata for the first save set in a first circular buffer of the plurality of circular buffers, wherein the received data chunks of the first save set are not separated by chunk headers within the first circular buffer;

computer program code for creating, by the server, a record header associated with all the data chunks of the first save set, the record header including metadata about the device from which the first save set originated;

computer program code for storing, on the server, received data chunks of a second save set and metadata for the second save set in a second circular buffer of the plurality of circular buffers, wherein the received data chunks of the second save set are not separated by chunk headers within the second circular buffer; and

computer program code for, in response to all of the data chunks of the first save set being stored in the first circular buffer, writing the received data chunks of the first save set and the record header to storage media before writing any portion of the second save set, the storage media being coupled to the server and comprising backup data for a plurality of client devices, the data chunks of the first save set and the record header being stored together in the first media record.

13. A system for storing data comprising:

a server with one or more processors for receiving data chunks over a network connection, the data chunks being subdivisions of a corresponding save set, each save set originating from a different device, for creating a circular buffer for each of the save sets, each circular buffer only including data for a corresponding save set, for storing received data chunks of a first save set in a first circular buffer of the created circular buffers, for creating a record header associated with all the data chunks of the first save set, the record header including metadata about the device from which the first save set originated, for storing received data chunks of a second save set in a second circular buffer of the created circular buffers, and for writing, in response to all of the data chunks of the first save set being stored in the first circular buffer, the received data chunks of the first save set and the record header to storage media before writing any portion of the second save set, wherein the first save set and the second save set are each comprised of a plurality of data chunks not separated by chunk headers, the storage media being coupled to the server and comprising backup data for a plurality of client devices, the data chunks of the first save set and the record header being stored together in a first media record.