Method and apparatus of hierarchical storage management based on data value

• US 20060015529A1
• Filed: 07/15/2004
• Published: 01/19/2006
• Est. Priority Date: 07/15/2004
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Yagawa, Yuichi
• Granted Patent: US 7,177,883 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A hierarchical storage data management system includes application servers, a metadata server interconnected with the application servers through a local area network, storage devices interconnected through data flow paths, and a storage network connecting the storage devices to the application servers and to the metadata server. The metadata server including a metadata management element, a data value management unit, and a hierarchical storage management element calculates a data value for each of stores data objects in the system, assigns a storage cost value for each of storage areas in the system, normalizes calculated data values and assigned storage costs to an identical value range, compares normalized data values with normalized storage costs thereby determining whether to relocate the data objects to different storage areas, and relocates data objects to storage areas with storage cost values identical with data values of the data objects.

Claims

22. A hierarchical storage data management method, comprising:

providing a plurality of data objects and a plurality of storage areas in storage devices connected via a storage network;

calculating a data value for each of the data objects;

assigning a storage cost value for each of the storage areas;

normalizing calculated data values and assigned storage costs to an identical value range;

comparing normalized data values with normalized storage costs thereby determining whether to relocate said data objects to different storage areas; and

relocating data objects to storage areas with storage cost values identical with data values of said data objects.

1. A hierarchical storage data management system, comprising:

a plurality of application servers;

a metadata server interconnected with said plurality of application servers through a local area network;

a plurality of storage devices being interconnected through a plurality of data flow paths; and

a storage network connecting said plurality of storage devices to said plurality of application servers and to said metadata server,

wherein said metadata server comprises a metadata management element, a data value management unit, and a hierarchical storage management element.

2. The hierarchical storage data management system according to claim 1, wherein said plurality of storage devices comprises at least one of

a plurality of online storage devices,

a plurality of near-line storage devices, and

a plurality of offline storage devices.

3. The hierarchical storage data management system according to claim 1, wherein the application servers are inter-connected through the local area network, the application servers access the storage devices through the storage network, a link is established between the storage network and each of the storage devices.

4. The hierarchical storage data management system according to claim 1, wherein each of the storage devices has links to directly access other storage devices, said each of the storage devices manages metadata therein and accesses other storage devices if requested data are not available therein.

5. The hierarchical storage data management system according to claim 1, wherein said data value management unit includes:

means for receiving a plurality of user interest indicators for a plurality of data objects from said metadata management element;

means for analyzing said plurality of user interest indicators;

means for determining a user interest level for each of said data objects; and

means for assigning to each of said data objects a data value corresponding to said user interest level.

6. The hierarchical storage data management system according to claim 1, wherein said hierarchical storage management element stores or relocates said data objects into said plurality of storage devices based on said data values and a storage profile pertaining to each of said storage areas.

7. The hierarchical storage data management system according to claim 5, wherein said user interest indicators include a user'"'"'s access records of said data object.

8. The hierarchical storage data management system according to claim 7, wherein said user'"'"'s access records are numbers of access times provided for a pre-determined time period or a lifetime of said data object from being stored to deleted.

9. The hierarchical storage data management system according to claim 7, wherein said user'"'"'s access records are classified by a plurality of commend types, and said data value is a sum of said user'"'"'s access records of some or all said commend types.

10. The hierarchical storage data management system according to claim 8, wherein said numbers of access times are weighted with access frequency.

11. The hierarchical storage data management system according to claim 5, wherein said user interest indicators include at least one of indexes, bibliographic information, and words extracted from said data object.

12. The hierarchical storage data management system according to claim 11, wherein said bibliographic information includes author, title, keywords, and production date.

13. The hierarchical storage data management system according to claim 5, wherein said data value includes an assigned lifetime value of said data object with respect to a lifetime of said data object from being stored to deleted.

14. The hierarchical storage data management system according to claim 7, wherein the metadata management element has an interface that answers requests for said data access records from the data value management element.

15. The hierarchical storage data management system according to claim 2, wherein said hierarchical storage management element includes:

means for receiving a plurality of storage cost indicators from said metadata management element;

means for analyzing said plurality of storage cost indicators;

means for determining a storage cost level for said storage area; and

means for assigning to said storage area a storage cost value corresponding to said storage cost level.

16. The hierarchical storage data management system according to claim 15, wherein said storage cost indicators include average performance and reliability of said storage area.

17. The hierarchical storage data management system according to claim 15, wherein said storage value varies from high to low depending upon that said storage area is located in an online storage device, a near-line storage device, and an offline storage device.

18. The hierarchical storage data management system according to claim 1, wherein said metadata server normalizes data values of a plurality of data objects calculated by said data value management unit and storage cost values of a plurality of storage areas in the storage devices assigned by said hierarchical storage management element by setting a range of the data values to be identical with a range of the storage cost values.

19. The hierarchical storage data management system according to claim 18, wherein said metadata server uses the maximum data value of said data values as a maximum value for the storage cost values, and adjusts other storage cost values accordingly.

20. The hierarchical storage data management system according to claim 18, wherein said metadata server uses the maximum storage cost value of said storage cost values as a maximum value of said data values, and adjusts other data values accordingly.

21. The hierarchical storage data management system according to claim 18, wherein said metadata server compares normalized data values with normalized storage cost values to determine whether to relocate said data objects to difference storage areas.

23. The hierarchical storage data management method according to claim 22, wherein the calculating step involves:

collecting a plurality of user interest indicators;

analyzing said plurality of user interest indicators;

determining a user interest level for each of said data object; and

assigning to each of said data objects a data value corresponding to said user interest level.

24. The hierarchical storage data management method according to claim 23, wherein access to each of said data objects includes a plurality of commend types for access, and each of said data values is a sum of said numbers of access times of some or all said commend types.

25. The hierarchical storage data management method according to claim 23, wherein said numbers of access times are weighted with access frequency.

26. The hierarchical storage data management method according to claim 22, wherein the collecting step involves collecting at least one of indexes, bibliographic information, and words extracted from each of said data objects.

27. The hierarchical storage data management method according to claim 26, wherein said bibliographic information includes author, title, keywords, and production date.

28. The hierarchical storage data management method according to claim 22, wherein the collecting step involves assigning a lifetime value to each of said data objects with respect to a lifetime of each of said data objects from being stored to deleted.

29. The hierarchical storage data management method according to claim 21, wherein the assignment step involves:

collecting a plurality of storage cost indicators;

analyzing said plurality of storage cost indicators;

determining a storage cost level for each of said storage areas; and

assigning to each of said storage areas a storage cost value corresponding to said storage cost level.

30. The hierarchical storage data management method according to claim 29, wherein the collecting step involves collecting data of average performance and reliability of each of said storage areas.

31. The hierarchical storage data management method according to claim 29, wherein the assigning step involves varying said storage values from high to low depending upon each of said storage areas is located in an online storage device, a near-line storage device, and an offline storage device.

32. The hierarchical storage data management method according to claim 21, wherein the normalizing step involves using the maximum data value of said data values as a maximum value for the storage cost values, and adjusting other storage cost values accordingly.

33. The hierarchical storage data management method according to claim 21, wherein the normalizing step involves using the maximum storage cost value of said storage cost values as a maximum value of said data values, and adjusting other data values accordingly.

23-1. The hierarchical storage data management method according to claim 22, wherein the collecting step involves adding numbers of access times provided for a pre-determined time period or a lifetime of each of said data objects from being stored to deleted.