Method and system for efficient generation of storage reports

• US 7,552,115 B2
• Filed: 04/15/2005
• Issued: 06/23/2009
• Est. Priority Date: 04/15/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Matev, Georgi M., Oltean, Paul Adrian, Kalach, Ran, Benton, James R.
• Primary Examiner(s): Breene; John E
• Assistant Examiner(s): Sweeney; Patrick E
• Application Number:
  US11/107,977
• Publication Number:
  US 20060236069A1
• Time in Patent Office: 1,530 Days
• Field of Search: 707/3, 707/4, 707/104.1
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Described is a method and system by which reports of storage usage in computer systems are generated in an efficient manner by consolidating multiple requests for reports into a minimal number of volume scans, including by intelligently selecting a scanning method (e.g., of file system metadata versus find-first/find-next) and by performing parallel scans on different volumes. Namespace consolidation scans namespaces together, so as to generate multiple reports from the same set of files, reducing the number of volumes scans required to collect the data. Each volume scan may be a find-first, find next directory-based scan, or a volume metadata database scan. Time consolidation groups independent storage report generations together, such as storage report requests received within an administrator-specified interval. Parallel scans of different volumes may be performed, subject to I/O and processing resource limitations, and so that volumes partitioned on the same spindle are not scanned in parallel.

Claims

1. In a computing environment comprising at least a processor and a memory, a method comprising:

receiving a first request for a first storage report, the first request including an indication of information that is to be determined by scanning at least a portion of a namespace on a storage volume;

receiving a second request for a second storage report, the second request including an indication of information that is to be determined by scanning the same portion of the namespace on the storage volume;

consolidating the first request and the second request, such that the storage volume information indicated in the first request and the storage volume information indicated in the second request are both simultaneously ascertainable from a single storage volume scan;

planning a storage volume scan based on;

the consolidated requests for storage reports corresponding to the namespace, the storage volume scan being configured to scan the namespace for both the storage volume information indicated in the first request and the storage volume information indicated in the second request simultaneously in a single scan; and

one or more activity hints;

overriding the storage volume scan based on the consolidated requests and the activity hints, such that the storage volume scan is based on one or more user inputs;

performing the storage volume scan as indicated by the user inputs to simultaneously ascertain both the storage volume information indicated in the first request and the storage volume information indicated in the second request; and

generating a plurality of reports based on the scanned information, each report in the plurality of reports corresponding to a request for information including at least one of the first request and the second request.

2. The method of claim 1 wherein planning the volume scan comprises planning a single scan of file metadata of the storage volume.

3. The method of claim 2 wherein planning the single scan of the file system metadata of the storage volume comprises evaluating a number of files of each directory against a total number of files on the storage volume.

4. The method of claim 3 further comprising, accessing previous information to estimate the number of files of each directory and the total number of files on the storage volume.

5. The method of claim 1 wherein planning the volume scan comprises planning a find-first, find-next scan of at least two different directories of the storage volume.

6. The method of claim 1 further comprising, waiting for an interval to receive the second request after receiving the first request.

7. The method of claim 1 further comprising, planning the volume scan with parallel scanning of at least one other volume.

8. At least one computer-readable storage medium having computer-executable instructions stored thereon, which upon execution perform the method of claim 1.

13. The method of claim 1, wherein the first request for the first storage report requests a first type of storage volume snapshot information and the second request for the second storage report requests a second, different type of storage volume snapshot information.

14. The method of claim 1, further comprising:

receiving an indication that all storage report requests received within a certain time frame are to be consolidated for processing in single scan;

automatically consolidating those storage report requests received within the time frame, such that the consolidated requests can be simultaneously processed in a single scan.

15. The method of claim 1, wherein planning a storage volume snapshot scan further comprises analyzing various sets of information related to the scan including at least one of a namespace list, volume-to-disk mapping data for scheduling parallel scans and previous activity hints.

16. The method of claim 15, wherein previous activity hints comprise information from at least one previous scan to better optimize the forthcoming scan, the hints including at least one of the total number of files and directories on a snapshot volume, and the number of files and directories in a requested namespace or set of namespaces and the amount of fragmentation of the storage volume snapshot.

19. The method of claim 1, wherein the user input comprises an indication of one or more criteria that are to be used as the basis of the scan.

20. The method of claim 19, wherein the criteria indicates that a file system metadata scan is to be performed based on the criteria.

9. At least one computer-readable storage medium having computer-executable instructions stored thereon, which upon execution perform steps, comprising:

receiving a first request for a first storage report, the first request including an indication of information that is to be determined by scanning at least a portion of a namespace on a snapshot of a storage volume;

receiving a second request for a second storage report, the second request including an indication of information that is to be determined by scanning the same portion of the namespace on the snapshot of the storage volume;

consolidating the first request and the second request, such that the storage volume snapshot information indicated in the first request and the storage volume snapshot information indicated in the second request are both simultaneously ascertainable from a single storage volume snapshot scan;

planning a storage volume snapshot scan based on the consolidated requests for storage reports corresponding to the namespace, the storage volume snapshot scan being configured to scan the namespace for both the storage volume snapshot information indicated in the first request and the storage volume snapshot information indicated in the second request simultaneously in a single scan;

performing the planned storage volume snapshot scan to simultaneously ascertain both the storage volume snapshot information indicated in the first request and the storage volume snapshot information indicated in the second request; and

generating a plurality of reports based on the scanned information, each report in the plurality of reports corresponding to a request for information including at least one of the first request and the second request.

10. The computer-readable storage medium of claim 9 wherein planning the scan of the snapshot volume set comprises performing at least one step of a set of steps, the set including, performing namespace consolidation, determining for at least one storage volume snapshot whether to scan file system metadata or scan via a find-first, find-next scan, and scanning at least two volume snapshots in parallel with one another.

11. The computer-readable storage medium of claim 10 wherein performing namespace consolidation comprises scanning across common namespaces, common snapshot volumes and/or by waiting to receive two different requests for storage reports.

12. The computer-readable storage medium of claim 10 wherein determining whether to scan file system metadata or scan via a find-first, find-next scan comprises evaluating information maintained from a previous scan.

17. The computer-readable storage medium of claim 9, further comprising:

determining that a storage volume snapshot scan has been planned based on one or more activity hints; and

overriding the plan based on the activity hints, such that the storage volume snapshot scan is based on one or more user inputs.

18. In a computing environment comprising at least a processor and a memory, a method for combining multiple storage volume snapshot scan requests, such that the requests can be processed simultaneously in a single storage volume snapshot scan, the method comprising:

receiving a plurality of storage volume snapshot scan requests requesting information about data stored on a snapshot of a storage volume;

consolidating the scan requests, such that the requests can be processed simultaneously in a single storage volume snapshot scan;

accessing information including activity hints from at least one previous storage volume snapshot scan;

based on the accessed information, determining, from a list of available volume scans, which snapshot volume scan is most efficient;

performing the determined storage volume snapshot scan;

generating intermediate data during the scan indicating information corresponding to that information requested in the storage volume snapshot scan request; and

rendering one or more storage volume snapshot reports in a user-selected format based on the generated intermediate data.