System and method for combining data streams in pipelined storage operations in a storage network

• US 20050246510A1
• Filed: 11/15/2004
• Published: 11/03/2005
• Est. Priority Date: 11/13/2003
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kottomtharayil, Rajiv, Amarendran, Arun, Retnamma, Manoj Vijayan
• Granted Patent: US 7,315,923 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques

Abstract

Described herein are systems and methods for multiplexing pipelined data for backup operations. Various data streams are combined such as by multiplexing by a multiplexing module. The multiplexing module combines the data from the various data streams received by receiver module(s) into a single stream of chunks. The multiplexing module may combine data from multiple archive files into a single chunk. Additional modules perform other operations on the chunks of data to be transported such as encryption, compression, etc. The data chunks are transmitted via a transport channel to a receive pipeline that includes a second receiver module and other modules. The data chunks are then stored in a backup medium. The chunks are later retrieved and separated such as by demultiplexing for restoring to a client or for further storage as auxiliary copies of the separated data streams or archive files.

Claims

1. A method for performing a backup operation on a plurality of data streams containing data to be backed up, the method comprising:

combining the data streams into a single stream of one or more data chunks, including writing data from more than one of the data streams into at least one data chunk;

transmitting the one or more data chunks over a transport channel to a backup medium; and

storing the one or more data chunks on the backup medium.

2. The method of claim 1, wherein combining the data streams comprises multiplexing the data streams.

3. The method of claim 1, wherein the data streams contain data from a plurality of archive files, and wherein combining the data streams comprises writing data from more than one archive files into at least one data chunk.

4. The method of claim 3, wherein combining the data streams comprises writing data from a single archive file into more than one data chunk.

5. The method of claim 3, comprising inserting a plurality of tag headers into the at least one data chunk, each tag header describing data written in the data chunk from a corresponding archive file.

6. The method of claim 3, wherein combining the data streams comprises writing data into a data chunk until the end of an archive file has been reached.

7. The method of claim 3, comprising retrieving the data chunk from the backup medium and separating data from at least one of the archive files from the data chunk.

8. The method of claim 7, comprising separating the data from the archive files written into the data chunk from each other into separate archive file portions.

9. The method of claim 8, comprising storing the separate archive file portions on an auxiliary storage device.

10. The method of claim 9, comprising accessing the auxiliary storage device during a restore operation of a given archive file.

11. The method of claim 1, wherein combining the data streams comprises writing data into a data chunk until the data chunk reaches a predetermined size.

12. The method of claim 1, wherein combining the data streams comprises writing data into a data chunk until a configurable time interval has lapsed.

13. The method of claim 1, wherein combining the data streams comprises writing data into a data chunk in accordance with a storage policy.

14. The method of claim 1, comprising retrieving the data chunk from the backup medium.

15. The method of claim 14, comprising separating the data from at least one of the data streams from the data chunk.

16. The method of claim 15, comprising separating the data from all data streams written into the data chunk from each other into separate data stream portions.

17. The method of claim 15, wherein combining the data streams comprises multiplexing the data streams and wherein separating comprises demultiplexing the data streams written into the data chunk.

18. A system for performing a backup operation on a plurality of data streams containing data to be backed up, the system comprising:

one or more receivers for receiving the data streams;

a multiplexer for combining the data streams into a combined data stream;

a data writer for writing data from the combined data stream portion of the combined data streams into one or more data chunks; and

one or more backup media for storing the one or more data chunks.

19. The system of claim 18, comprising a transport channel for transporting the data chunks from the data writer to the backup media.

20. A data structure for a data chunk stored on a memory device, the data structure being used by a computer system to backup data, the data structure comprising:

a plurality of portions of data from different archive files written into the data chunk from multiplexed data streams containing the archive files; and

a plurality of tag headers each describing one of the archive file portions written into the data chunk.