Method and system for efficient generation of storage reports

• US 20060236069A1
• Filed: 04/15/2005
• Published: 10/19/2006
• Est. Priority Date: 04/15/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Oltean, Paul Adrian, Kalach, Ran, Benton, James R., Matev, Georgi M.
• Granted Patent: US 7,552,115 B2
• Time in Patent Office: Days
• US Class Current: 711/173
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Described is a method and system by which reports of storage usage in computer systems are generated in an efficient manner by consolidating multiple requests for reports into a minimal number of volume scans, including by intelligently selecting a scanning method (e.g., of file system metadata versus find-first/find-next) and by performing parallel scans on different volumes. Namespace consolidation scans namespaces together, so as to generate multiple reports from the same set of files, reducing the number of volumes scans required to collect the data. Each volume scan may be a find-first, find next directory-based scan, or a volume metadata database scan. Time consolidation groups independent storage report generations together, such as storage report requests received within an administrator-specified interval. Parallel scans of different volumes may be performed, subject to I/O and processing resource limitations, and so that volumes partitioned on the same spindle are not scanned in parallel.

Claims

1. In a computing environment, a method comprising:

receiving a first request for a first storage report corresponding to a namespace to scan on a storage volume;

receiving a second request for a second storage report corresponding to a namespace to scan on the storage volume; and

planning a volume scan based on a consolidation of the namespaces.

2. The method of claim 1 wherein planning the volume scan comprises planning a single scan of file metadata of the storage volume.

3. The method of claim 2 wherein the namespace of the first request and the namespace of the second request correspond to different directories on the storage volume.

4. The method of claim 3 wherein planning the single scan of the file system metadata of the storage volume comprises evaluating a number of files of each directory against a total number of files on the storage volume.

5. The method of claim 4 further comprising, accessing previous information to estimate the number of files of each directory and the total number of files on the storage volume.

6. The method of claim 1 wherein planning the volume scan comprises planning a find-first, find-next scan of at least two different directories of the storage volume.

7. The method of claim 1 further comprising, waiting for an interval to receive the second request after receiving the first request.

8. The method of claim 1 further comprising, planning the volume scan with parallel scanning of at least one other volume.

9. At least one computer-readable medium having computer-executable instructions, which when executed perform the method of claim 1.

10. In a computing environment, a system comprising:

means for receiving requests for storage reports of storage volumes;

planning means for consolidating a plurality of requests to reduce or avoid repetitive scans of a storage volume of a set of one or more storage volumes;

scanning means for scanning each storage volume of the set based on a plan from the planning means;

data gathering means for collecting data received from the scanning means; and

rendering means for outputting a storage report based on the data collected by the data gathering means.

11. The system of claim 10 wherein the planning means consolidates namespace information to produce the plan.

12. The system of claim 10 wherein the planning means consolidates a plurality of requests received at different times to produce the plan.

13. The system of claim 10 wherein the planning means produces the plan such that at least two storage volumes are to be scanned in parallel with one another, each volume scanned in parallel corresponding to a different disk, and wherein the planning means includes means for limiting the number of disks that may be scanned in parallel.

14. The system of claim 10 wherein at least one storage volume has file system metadata, and wherein the planning means determines that a scan of that storage volume comprises a scan of the file system metadata.

15. The system of claim 10 wherein the planning means determines that a scan of a particular storage volume comprises a find-first, find-next scan of at least one directory of the particular storage volume.

16. At least one computer-readable medium having computer-executable instructions, which when executed perform steps, comprising:

receiving at least one request for a storage report, each request corresponding to a volume set of at least one storage volume from which the storage report is to be generated, and each request associated with information corresponding to at least one subset of files for which the report is to be generated; and

planning a scan of the volume set in which at least two subsets of files are evaluated in a single volume scan.

17. The computer-readable medium of claim 16 wherein planning the scan of the volume set comprises performing at least one step of a set of steps, the set including, performing namespace consolidation, determining for at least one storage volume whether to scan file system metadata or scan via a find-first, find-next scan, and scanning at least two volumes in parallel with one another.

18. The computer-readable medium of claim 17 wherein performing namespace consolidation comprises scanning across common namespaces, common volumes and/or by waiting to receive two different requests for storage reports.

19. The computer-readable medium of claim 17 wherein determining whether to scan file system metadata or scan via a find-first, find-next scan comprises evaluating information maintained from a previous scan.

20. The computer-readable medium of claim 17 wherein scanning at least two volumes in parallel with one another comprises determining whether a volume is on a common disk with another volume to avoid scanning volumes in parallel that are on a common disk, and limiting a number of disks that can be scanned in parallel with one another based on computing resource limitations.