Hierarchical data storage management

• US 6,330,572 B1
• Filed: 07/15/1999
• Issued: 12/11/2001
• Est. Priority Date: 07/15/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Imation Corporation (Glassbridge Enterprises, Inc.)
• Inventors: Sitka, Larry
• Primary Examiner(s): Alam, Hosain T.
• Assistant Examiner(s): TRUONG, CAM Y T
• Application Number:
  US09/354,280
• Time in Patent Office: 880 Days
• Field of Search: 707/104, 707/100, 707/201, 707/205, 707/203-204
• US Class Current: 707/608
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/941 Human sciences
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99956 File allocation

Abstract

A system and method for managing the storage of files within an HSM system incorporate an architecture and methodology that facilitate the storage and retrieval of large image files as part of an overall image processing workflow. In particular, the system and method may find ready application in a workflow that involves the processing of groups of images associated with particular customers, projects, or transactions, and may act as a storage server for a client application that implements the workflow. The system and method may be useful, for example, in handling the storage of images uploaded from scanned photographic film, or digital images submitted to a photo-processing shop by amateur or professional photographers. In this case, the client application can be a photo-processing application that could provide for various media formats, sizes, and quantities of image reproductions for a consumer. As another example, the system and method may be useful in handling the storage of medical diagnostic images associated with a particular medical patient or study. In this case, the client application could be a picture archival communication system (PACS) that manages the archival of imagery for viewing by physicians. Further, the system and method may be useful in handling the storage of images associated with particular printing jobs, e.g., for publishers, advertising customers, and the like. In this case, the client application could be a digital prepress workflow application.

Claims

1. A method for storing files among a hierarchy of data storage media in a hierarchical storage management system, the method comprising:

assigning the files to filesets, each of the files assigned to a respective one of the filesets sharing one or more common attributes with the other files assigned to the respective fileset;

storing the files assigned to the respective fileset together on a common data storage medium at a first level in the hierarchy of data storage media; and

moving the files assigned to the respective fileset together from the common data storage medium to another common data storage medium at a second level in the hierarchy of data storage media, such that the files assigned to the respective fileset are stored together on one of the data storage media at substantially all times.

2. The method of claim 1, wherein the one or more common attributes include at least one of a common entity and a common project associated with each of the files assigned to the respective fileset.

3. The method of claim 1, wherein the one or more common attributes include a common transaction associated with each of the files assigned to the respective fileset.

4. The method of claim 1, wherein the files include medical diagnostic imagery, and the one or more common attributes include at least one of a common patient and a common medical practitioner associated with each of the files assigned to a respective one of the filesets.

5. The method of claim 1, wherein the files include advertising imagery, and the one or more common attributes include a common advertising customer associated with each of the files assigned to the respective fileset.

6. The method of claim 1, further comprising retrieving the files assigned to the respective fileset together as a group.

7. The method of claim 1, further comprising deleting the files assigned to the respective fileset together as a group.

8. The method of claim 1, further comprising, in response to a user request to take action with respect to one of the files in one of the filesets, taking the requested action with respect to all of the files in the respective fileset.

9. The method of claim 8, wherein the requested action includes deletion of the respective file, and taking the requested action includes deleting all of the files in the respective fileset.

10. The method of claim 8, wherein the requested action includes retrieval of the respective file, and taking the requested action includes retrieving all of the files in the respective fileset.

11. The method of claim 1, wherein the first level of data storage media includes magnetic hard disk media and the second level of data storage media includes one of optical data storage media and magnetic tape media.

12. A computer-implemented system for storing files among a hierarchy of data storage media in a hierarchical storage management system, the file storage system comprising a processor that is programmed to:

assign the files to filesets, each of the files assigned to a respective one of the filesets sharing one or more common attributes with the other files assigned to the respective fileset;

control storage of the files assigned to the respective fileset together on a common data storage medium at a first level in the hierarchy of data storage media; and

control movement of the files assigned to the respective fileset together from the common data storage medium to another common data storage medium at a second level in the hierarchy of data storage media, such that the files assigned to the respective fileset are stored together on one of the data storage media at substantially all times.

13. The system of claim 12, wherein the one or more common attributes include at least one of a common entity and a common project associated with each of the files assigned to the respective fileset.

14. The system of claim 12, wherein the one or more common attributes include a common transaction associated with each of the files assigned to the respective fileset.

15. The system of claim 12, wherein the files include medical diagnostic imagery, and the one or more common attributes include at least one of a common patient and a common medical practitioner associated with each of the files assigned to a respective one of the filesets.

16. The system of claim 12, wherein the files include advertising imagery, and the one or more common attributes include a common advertising customer associated with each of the files assigned to the respective fileset.

17. The system of claim 12, wherein the processor is further programmed to control retrieval of the files assigned to the respective fileset together as a group.

18. The system of claim 12, wherein the processor is further programmed to control deletion of the files assigned to the respective fileset together as a group.

19. The system of claim 12, wherein the processor is further programmed to receive a user request to take action with respect to one of the files in one of the filesets and, in response to the request, take the requested action with respect to all of the files in the respective fileset.

20. The system of claim 19, wherein the requested action includes deletion of the respective file, and the processor is programmed to take the requested action by deleting all of the files in the respective fileset.

21. The system of claim 19, wherein the requested action includes retrieval of the respective file, and the processor is programmed to take the requested action by retrieving all of the files in the respective fileset.

22. The system of claim 12, wherein the first level of data storage media includes magnetic hard disk media and the second level of data storage media includes one of optical data storage media and magnetic tape media.

23. A method for storing files, the method comprising:

storing the files on data storage media volumes;

writing, to each of the media volumes, volume metadata that uniquely describes the respective volume;

writing, with each of the files stored on the media volumes, file metadata that uniquely describes the respective file; and

moving one or more of the files from one of the media volumes to another of the media volumes, wherein each of the moved files carries with it the file metadata for the respective moved file.

24. The method of claim 23, wherein each of the volume metadata and the file metadata is based on a globally unique identifier (guid).

25. The method of claim 23, wherein storing each of the files includes storing a data file containing the contents of the respective file and storing a metadata file containing the file metadata.

26. The method of claim 23, further comprising:

constructing original file location database entries specifying the arrangement of the files on each of the media volumes; and

in the event the original database entries become corrupted, reconstructing the database entries using the file metadata written with each of the files stored on the respective media volume.

27. The method of claim 26, wherein the file metadata for the files stored on the respective media volume provides sufficient information to reconstruct the database entries access to the original database entries.

28. A computer-implemented system for storing files, the system composing a processor that is programmed to:

control storage of the files on data storage media volumes;

write, to each of the media volumes, volume metadata that uniquely describes the respective volume;

write, with each of the files stored on the media volumes, file metadata that uniquely describes the respective file; and

control movement of one or more of the files from one of the media volumes to another of the media volumes, wherein each of the moved files carries with it the file metadata for the respective moved file.

29. The system of claim 28, wherein each of the volume metadata and the file metadata is based on a globally unique identifier (guid).

30. The system of claim 28, wherein storing each of the files includes storing a data file containing the contents of the respective file and storing a metadata file containing the file metadata.

31. The system of claim 28, further comprising:

constructing original file location database entries specifying the arrangement of the files on each of the media volumes; and

in the event the original database entries become corrupted, reconstructing the database entries using the file metadata written with each of the files stored on the respective media volume.

32. The system of claim 31, wherein the file metadata for the files stored on the respective media volume provides sufficient information to reconstruct the database entries access to the original database entries.