Systems and methods of data storage management, such as dynamic data stream allocation

• US 9,256,606 B2
• Filed: 06/18/2013
• Issued: 02/09/2016
• Est. Priority Date: 11/15/2004
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Attarde, Deepak Raghunath, Gokhale, Parag, Klose, Michael F.
• Primary Examiner(s): Lee, Adam
• Application Number:
  US13/920,633
• Publication Number:
  US 20130282774A1
• Time in Patent Office: 966 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A system and method for choosing a stream to transfer data is described. In some cases, the system reviews running data storage operations and chooses a data stream based on the review. Additionally, the system pre-allocates disk space for data to be transferred.

Claims

1. A method of transferring data from a source storage device to a magnetic disk storage medium, the method comprising:

determining an estimated amount of disk storage space required to store the data on the magnetic disk storage medium by estimating at least a total size of the data, wherein the data includes multiple jobs each comprising one or more files, and wherein a disk file system manages individual files on the magnetic disk storage medium;

instructing the disk file system that a single file is to be stored in the estimated amount of disk storage space to reserve the estimated amount of disk storage space as a contiguous physical portion of disk storage space on the magnetic disk storage medium before transferring the data to the magnetic disk storage medium;

allocating the reserved contiguous physical portion of disk storage space on the magnetic disk storage medium; and

transferring the data from the source storage device to the allocated contiguous physical portion of disk storage space on the magnetic disk storage medium.

2. The method of claim 1, further comprising:

determining that the allocated amount of disk storage space is greater than an amount of data transferred to the allocated space; and

freeing space not containing transferred data from the allocated space.

3. The method of claim 1, further comprising:

determining that the allocated amount of storage space is not sufficient to contain an amount of data transferred to the allocated space; and

expanding the allocated space to store the transferred amount of data within the expanded allocated space, wherein the expanded space includes additional space contiguous with the allocated space.

4. The method of claim 1, further comprising:

creating an auxiliary file allocation table, auxiliary to a file allocation table of the disk file system, that lists the individual files stored in the allocated estimated amount of space.

5. The method of claim 1, wherein determining the estimated amount of storage space includes determining an amount of available space contained on the storage medium.

6. The method of claim 1, wherein allocating the estimated amount of storage space includes reserving, by the disk file system, a contiguous space of data blocks on the storage medium.

7. The method of claim 1, further comprising:

creating a data structure that contains information referencing the transferred data within the allocated space.

8. A system for transferring two or more data items to a storage device by a file system, comprising:

memory having instructions; and

a processor coupled to the memory and configured to execute the instructions, wherein the instructions include;

a data estimation component, wherein the data estimation component estimates space required to store two or more data items on the storage device by estimating at least a total size of the two or more data items;

a data allocation component, wherein the data allocation component instructs the file system that a single data item is to be stored in the estimated space to reserve the estimated space as a contiguous space on the storage device in preparation for transferring the two or more data items to the storage device and provides defragmentation services by causing the file system to allocate the contiguous space on the storage device for the two or more data items where the file system otherwise allocates two or more non-contiguous spaces of the storage device; and

a data transfer component in communication with the data allocation component, wherein the data transfer component transfers the two or more data items to the allocated contiguous space on the storage device.

9. The system of claim 8, further comprising:

a data reference component, wherein the data reference component creates a data structure that references locations of the two or more data items within the reserved space.

10. The system of claim 8, wherein the data estimation component estimates an amount of space of the storage device to reserve using characteristics of the two or more data items received by the data transfer component.

11. The system of claim 8, wherein the data estimation component further estimates an amount of space of the storage device to reserve using space on the storage device that does not already contain data items.

12. The system of claim 8, wherein the data allocation component frees any unused space from the reserved space after the data transfer component transfers the two or more data items to the storage device.

13. The system of claim 8, wherein the data allocation component allocates additional space to the reserved space when the reserved space is full and the data transfer component has not transferred all the data items to the storage device.

14. A non-transitory computer-readable storage medium whose contents are capable of causing a computing system to perform a method of storing data to a data storage medium, the method comprising:

determining an estimated total amount of storage space for storing multiple data files on the data storage medium by estimating at least a total size of the multiple data files;

reserving an amount of contiguous storage space more than the estimated total amount of storage space on the data storage medium by instructing a file system that a single file is to be stored in the contiguous storage space;

allocating the reserved storage space on the data storage medium; and

transferring the multiple data files to be stored to the allocated contiguous storage space on the data storage medium.

15. The non-transitory computer-readable storage medium of claim 14, the method further comprising:

creating a data structure representing the allocated contiguous storage space, wherein the data structure stores information referencing the multiple data files within the allocated contiguous space.

16. The non-transitory computer-readable storage medium of claim 14, wherein allocating more than the estimated amount of storage space on the data storage medium includes allocating two discrete contiguous spaces within the data storage medium.

17. The non-transitory computer-readable storage medium of claim 14, the method further comprising:

removing any space not containing transferred data from the allocated contiguous space, such that the removed space is available to be written to by other data.