Systems and methods for combining data streams in a storage operation

• US 8,417,908 B2
• Filed: 03/02/2012
• Issued: 04/09/2013
• Est. Priority Date: 11/13/2003
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Retnamma, Manoj Vijayan, Amarendran, Arun, Kottomtharayil, Rajiv
• Primary Examiner(s): SONG, HUA JASMINE
• Application Number:
  US13/410,770
• Publication Number:
  US 20120166745A1
• Time in Patent Office: 403 Days
• Field of Search: 711161-162, 711/165, 711/E12.103, 707/609, 707639-642, 707/647, 709/231
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques

Abstract

Described herein are systems and methods for multiplexing pipelined data for backup operations. Various data streams are combined such as by multiplexing by a multiplexing module. The multiplexing module combines the data from the various data streams received by receiver module(s) into a single stream of chunks. The multiplexing module may combine data from multiple archive files into a single chunk. Additional modules perform other operations on the chunks of data to be transported such as encryption, compression, etc. The data chunks are transmitted via a transport channel to a receive pipeline that includes a second receiver module and other modules. The data chunks are then stored in a backup medium. The chunks are later retrieved and separated such as by demultiplexing for restoring to a client or for further storage as auxiliary copies of the separated data streams or archive files.

Claims

1. A data storage system comprising:

a data multiplexing storage policy that defines the backup and multiplexing of data associated with multiple clients, wherein at least a first data stream associated with a first client and a second data stream associated with a second client are multiplexed onto the same storage media;

a storage manager that stores the data multiplexing storage policy in association therewith, the storage manager executing in one or more computer processors, wherein the storage manager controls backup storage operations associated with backup of data from one or more clients to at least a first storage device;

the storage manager configured to, based on the data multiplexing storage policy, backup data on the first client by directing the first data stream of data from a first client to a first storage media;

the storage manager further configured to, based on the data multiplexing storage policy, backup data on the second client by directing the second data stream of data from the second client to the first storage media; and

a receiver executing in one or more computer processors that receives the first and second data streams, wherein the receiver is configured to, based on the data multiplexing storage policy, multiplex at least portions of the first and second data streams to create a data chunk, wherein the multiplexer is further configured to add one or more identifiers to the data chunk that identifies in the data chunk, the portions of the first and second data streams in the data chunk and

storing the data chunk on the first storage media.

2. The system of claim 1, wherein at least one of the one or more identifiers is a tag header that comprises the size of data written in the first data chunk from a corresponding data stream.

3. The system of claim 1, wherein the first data stream is obtained from a first archive file and the second data stream is obtained from a second archive file.

4. The system of claim 1, wherein the receiver multiplexes the first and second data streams by writing data from the first and second data streams into more than one data chunk.

5. The system of claim 1 wherein the storage manager directs the reservation of a volume on the first storage device based on the data multiplexing storage policy.

6. The system of claim 1, wherein the first data stream and the second data stream are directed simultaneously to the receiver.

7. The system of claim 1, wherein the number of data streams are associated with the number of drives in a drive pool.

9. The method of claim 1, wherein at least one of the one or more identifiers is a tag header that comprises the size of data written in the first data chunk from a corresponding data stream.

10. The method of claim 1, wherein the first data stream is obtained from a first archive file and the second data stream is obtained from a second archive file.

11. The method of claim 1, wherein the multiplexing further comprises writing data from the first and second data streams into more than one data chunk.

12. The method of claim 1 further comprising reserving a volume on the first storage device based on the data multiplexing storage policy.

13. The system of claim 1, wherein the first data stream and the second data stream are directed simultaneously to the receiver.

14. The system of claim 1, wherein the number of data streams are associated with the number of drives in a drive pool.

8. A data storage system comprising:

storing a data multiplexing storage policy in association with a storage manager, wherein the data multiplexing storage policy defines the backup and multiplexing of data associated with multiple clients, wherein at least a first data stream associated with a first client and a second data stream associated with a second client are multiplexed onto the same storage media;

backing up data on the first client based on the data multiplexing storage policy, by directing the first data stream of data from the first client to a first storage media;

backing up data on the second client based on the data multiplexing storage policy, by directing the second data stream of data from the second client to the first storage media;

multiplexing with one or more computer processors, based on the data multiplexing policy, at least portions of the first and second data streams to create a data chunk; and

adding one or more identifiers to the data chunk that identifies in the data chunk, the portions of the first and second data streams in the data chunk; and

storing the data chunk on the first storage media.