Method and apparatus for hierarchical storage management based on data value and user interest

• US 7,177,883 B2
• Filed: 07/15/2004
• Issued: 02/13/2007
• Est. Priority Date: 07/15/2004
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Yagawa, Yuichi
• Primary Examiner(s): Wong; Don
• Assistant Examiner(s): Le; Michael
• Application Number:
  US10/891,511
• Publication Number:
  US 20060015529A1
• Time in Patent Office: 943 Days
• Field of Search: 707/104.1, 707/2, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A hierarchical storage data management system includes application servers, a metadata server interconnected with the application servers through a local area network, storage devices interconnected through data flow paths, and a storage network connecting the storage devices to the application servers and to the metadata server. The metadata server including a metadata management element, a data value management unit, and a hierarchical storage management element calculates a data value for each of stores data objects in the system, assigns a storage cost value for each of storage areas in the system, normalizes calculated data values and assigned storage costs to an identical value range, compares normalized data values with normalized storage costs thereby determining whether to relocate the data objects to different storage areas, and relocates data objects to storage areas with storage cost values identical with data values of the data objects.

Claims

1. A hierarchical storage data management system, comprising:

a plurality of application servers;

a metadata server interconnected with said plurality of application servers through a local area network;

a plurality of storage devices being interconnected through a plurality of data flow paths; and

a storage network connecting said plurality of storage devices to said plurality of application servers and to said metadata server,

wherein said metadata server comprises a metadata management element, a data value management element for assigning a data value to each of a plurality of data objects of a user corresponding to interest of the user, and a hierarchical storage management element,

wherein said data value management element includes;

means for receiving a plurality of user interest indicators for a plurality of data objects from said metadata management element;

means for analyzing said plurality of user interest indicators;

means for determining a user interest level for each of said data objects; and

means for assigning to each of said data objects a data value corresponding to said user interest level,

said user interest indicators include bibliographic information, words extracted from said data object and an assigned lifetime value of said data object as assigned by the user with respect to a lifetime of said data object from being stored to deleted,

wherein said hierarchical storage management element includes;

means for receiving a plurality of storage cost indicators from said metadata management element;

means for analyzing said plurality of storage cost indicators;

means for determining a storage cost level for each of a plurality of storage areas in said storage devices;

means for assigning to said each storage area a storage cost value corresponding to said storage cost level, means for planning metadata adjustment by balancing between said storage cost values and metadata values assigned to metadata objects, and means for adjusting storage areas of said metadata objects as planned by said means for planning metadata adjustment,

said storage cost indicators include average performance and reliability of said each storage area, and

wherein said metadata server normalizes assigned data values by said data value management element and assigned storage cost values by said hierarchical storage management element by setting a range of the assigned data values to be identical with a range of the assigned storage cost values, and compares normalized data values with normalized storage cost values to determine whether to relocate data objects to different storage areas.

2. The hierarchical storage data management system according to claim 1, wherein said plurality of storage devices comprises at least one of

a plurality of online storage devices,

a plurality of near-line storage devices, and

a plurality of offline storage devices.

3. The hierarchical storage data management system according to claim 2, wherein said storage cost value decreases depending upon that said each storage area is located in an online storage device, a near-line storage device, or an offline storage device.

4. The hierarchical storage data management system according to claim 1, wherein the application servers are inter-connected through the local area network, the application servers access the storage devices through the storage network, a link is established between the storage network and each of the storage devices.

5. The hierarchical storage data management system according to claim 1, wherein each of the storage devices has links to directly access other storage devices, said each of the storage devices manages metadata therein and accesses other storage devices if requested data are not available therein.

6. The hierarchical storage data management system according to claim 1, wherein said hierarchical storage management element stores or relocates said data objects into a plurality of storage areas in said plurality of storage devices based on said data values and a storage profile pertaining to each of said storage areas.

7. The hierarchical storage data management system according to claim 1, wherein said user interest indicators include a user'"'"'s data access records of said data object.

8. The hierarchical storage data management system according to claim 7, wherein said user'"'"'s access records are numbers of access times provided for a pre-determined time period or a lifetime of said data object from being stored to deleted.

9. The hierarchical storage data management system according to claim 8, wherein said numbers of access times are weighted with access frequency.

10. The hierarchical storage data management system according to claim 7, wherein said user'"'"'s access records are classified by a plurality of command types, and said data value is a sum of said user'"'"'s access records of some or all of said command types.

11. The hierarchical storage data management system according to claim 7, wherein the metadata management element has an interface that answers requests for said data access records from the data value management element.

12. The hierarchical storage data management system according to claim 1, wherein said user interest indicators include indexes.

13. The hierarchical storage data management system according to claim 1, wherein said bibliographic information includes author, title, keywords, and production date.

14. The hierarchical storage data management system according to claim 1, wherein said metadata server uses the maximum data value of said data values as a maximum value for the storage cost values.

15. The hierarchical storage data management system according to claim 1, wherein said metadata server uses the maximum storage cost value of said storage cost values as a maximum value of said data values.

16. A hierarchical storage data management method, comprising:

providing a plurality of data objects and a plurality of storage areas in storage devices connected via a storage network;

calculating a data value for each of the data objects thereby assigning the data value to each of the data objects of a user corresponding to interest of the user, the calculating step involving;

collecting a plurality of user interest indicators;

analyzing said plurality of user interest indicators;

determining a user interest level for each of said data objects; and

assigning to each of said data objects a data value corresponding to said user interest level;

assigning a storage cost value for each of the storage areas, the assigning step involving;

collecting a plurality of storage cost indicators;

analyzing said plurality of storage cost indicators;

determining a storage cost level for each of said storage areas; and

assigning to each of said storage areas a storage cost value corresponding to said storage cost level;

normalizing calculated data values and assigned storage cost values to an identical value range;

comparing normalized data values with normalized storage cost values thereby determining whether to relocate said data objects to different storage areas;

relocating data objects to storage areas with storage cost values identical with data values of said data objects;

planning metadata adjustment by balancing between said storage cost values and metadata values assigned to metadata objects; and

adjusting storage areas of said metadata objects as planned by said means for planning metadata adjustment,

wherein the collecting step in the calculating step involves collecting bibliographic information, words extracted from said data object and an assigned lifetime value of said data object as assigned by the user with respect to a lifetime of said data object from being stored to deleted, and

wherein the collecting step in the assigning step involves collecting data of average performance and reliability of each of said storage areas.

17. The hierarchical storage data management method according to claim 16, wherein the collecting step involves collecting indexes.

18. The hierarchical storage data management method according to claim 16, wherein said bibliographic information includes author, title, keywords, and production date.

19. The hierarchical storage data management method according to claim 16, wherein the assigning step involves decreasing said storage cost values depending upon each of said storage areas is located in an online storage device, a near-line storage device, or an offline storage device.

20. The hierarchical storage data management method according to claim 16, wherein the normalizing step involves using the maximum data value of said data values as a maximum value for the storage cost values.

21. The hierarchical storage data management method according to claim 16, wherein the normalizing step involves using the maximum storage cost value of said storage cost values as a maximum value of said data values.

22. The hierarchical storage data management method according to claim 16, wherein the collecting step in the calculating step involves adding numbers of access times provided for a pre-determined time period or a lifetime of each of said data objects from being stored to being deleted.

23. The hierarchical storage data management method according to claim 22, wherein access to each of said data objects includes a plurality of command types for access, and each of said data values is a sum of said numbers of access times of some or all of said command types.

24. The hierarchical storage data management method according to claim 22, wherein said numbers of access times are weighted with access frequency.