Data placement optimization

• US 10,120,891 B2
• Filed: 02/11/2013
• Issued: 11/06/2018
• Est. Priority Date: 02/11/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aslot, Vishal C., Bello, Adekunle, Boss, Gregory J.
• Primary Examiner(s): Waldron, Scott A.
• Assistant Examiner(s): Cheung, Hubert
• Application Number:
  US13/764,064
• Publication Number:
  US 20140229447A1
• Time in Patent Office: 2,094 Days
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity

Abstract

For improving data placement optimization in a computing storage environment, an application layer and/or a user are allowed to attach anticipated usage metadata and directive metadata to a file.

Claims

1. A method for improving data placement optimization by a processor device in a computing storage environment, the method comprising:

attaching anticipated usage metadata and directive metadata to a file by an application layer, the anticipated usage metadata containing a plurality of attributes describing characteristics of the file and the directive metadata containing directives comprising storage instructions for the file;

wherein a storage system uses the characteristics of the file from the anticipated usage metadata and the storage instructions from the directive metadata in lieu of a general storage policy to determine a file placement of the file within the storage system;

monitoring for a file creation event, wherein the file is created;

attaching thereby embedding the anticipated usage metadata and the directive metadata within data content of the file by the application layer during the creation of the file;

updating the anticipated usage metadata and the directive metadata in real time according to a frequency of access of the file;

adding a set of core metadata to the file at the time of the file creation, the core metadata comprising at least a bandwidth requirement for processing the file and an importance factor of the file as assigned to the attached anticipated usage metadata and the directive metadata such that the importance factor signifies how essential the anticipated usage metadata and the directive metadata are to be followed when determining the file placement;

wherein the core metadata further indicates the importance factor to a storage controller of the storage system for performing resource allocation operations to process the file by the storage controller so as to allocate enough resources by the storage controller to the file to ensure the bandwidth requirement for processing the file is satisfied during file access;

scanning the directive metadata for determining when to execute those directives contained in the directive metadata; and

executing the directive metadata.

2. The method of claim 1, wherein the anticipated usage metadata includes at least one of time, scope, and duration for the file.

3. The method of claim 1, wherein the directive metadata is one of a local directive that is executed on a current system and a cascading directive that is sent to a next destination enroute to a final destination and allow for automatically executing each of an appropriate action on each destination at an appropriate time.

4. The method of claim 3, further including transitioning from the cascading directive to the local directive after the directive metadata arrives at a target system.

5. The method of claim 1, further including anticipating future usage patterns by using the anticipated usage metadata and the directive metadata for proactively moving data to one of a plurality of storage tiers.

6. The method of claim 1, further including optimizing the file placement using the anticipated usage metadata and the directive metadata.

7. The method of claim 6, further including extending the optimization of the file placement beyond a storage system infrastructure into an edge-caching network using the anticipated usage metadata and the directive metadata.

8. The method of claim 1, further including performing one of:

reading the anticipated usage metadata and the directive metadata,

scheduling a send file operation for sending the file to a next destination according to the anticipated usage metadata and the directive metadata,

scheduling a send metadata operation for sending metadata to a next destination according to the anticipated usage metadata and the directive metadata.

9. The method of claim 1, further including performing one of:

setting up default metadata as system tunable by a file type,

attaching the default metadata on the file type,

modifying the default metadata that is attached to the file, and

using one of a plurality of defined application programming interfaces (API) for attaching to the anticipated usage metadata and the directive metadata.

10. A system for improving data placement optimization in a computing storage environment, the system comprising:

at least one processor device operable in the computing storage environment, wherein the at least one processor device;

attaches anticipated usage metadata and directive metadata to a file by an application layer, the anticipated usage metadata containing a plurality of attributes describing characteristics of the file and the directive metadata containing directives comprising storage instructions for the file;

wherein a storage system uses the characteristics of the file from the anticipated usage metadata and the storage instructions from the directive metadata in lieu of a general storage policy to determine a file placement of the file within the storage system;

monitors for a file creation event, wherein the file is created;

attaches thereby embedding the anticipated usage metadata and the directive metadata within data content of the file by the application layer during the creation of the file;

updates the anticipated usage metadata and the directive metadata in real time according to a frequency of access of the file;

adds a set of core metadata to the file at the time of the file creation, the core metadata comprising at least a bandwidth requirement for processing the file and an importance factor of the file as assigned to the attached anticipated usage metadata and the directive metadata such that the importance factor signifies how essential the anticipated usage metadata and the directive metadata are to be followed when determining the file placement;

wherein the core metadata further indicates the importance factor to a storage controller of the storage system for performing resource allocation operations to process the file by the storage controller so as to allocate enough resources by the storage controller to the file to ensure the bandwidth requirement for processing the file is satisfied during file access;

scans the directive metadata for determining when to execute those directives contained in the directive metadata; and

executes the directive metadata.

11. The system of claim 10, wherein the anticipated usage metadata includes at least one of time, scope, and duration for the file.

12. The system of claim 10, wherein the directive metadata is one of a local directive that is executed on a current system and a cascading directive that is sent to a next destination enroute to a final destination and allow for automatically executing each of an appropriate action on each destination at an appropriate time.

13. The system of claim 12, wherein the at least one processor device transitions from the cascading directive to the local directive after the directive metadata arrives at a target system.

14. The system of claim 10, wherein the at least one processor device anticipates future usage patterns by using the anticipated usage metadata and the directive metadata for proactively moving data to one of a plurality of storage tiers.

15. The system of claim 10, wherein the at least one processor device optimizes the file placement using the anticipated usage metadata and the directive metadata.

16. The system of claim 15, wherein the at least one processor device extends the optimization of the file placement beyond a storage system infrastructure into an edge caching network using the anticipated usage metadata and the directive metadata.

17. The system of claim 10, wherein the at least one processor device performs one of:

reading the anticipated usage metadata and the directive metadata,

scheduling a send file operation for sending the file to a next destination according to the anticipated usage metadata and the directive metadata,

scheduling a send metadata operation for sending metadata to a next destination according to the anticipated usage metadata and the directive metadata.

18. The system of claim 10, wherein the at least one processor device performs one of:

setting up default metadata as system tunable by a file type,

attaching the default metadata on the file type,

modifying the default metadata that is attached to the file, and

using one of a plurality of defined application programming interfaces (API) for attaching to the anticipated usage metadata and the directive metadata.

19. A computer program product for reducing data loss in a computing storage environment by a processor device, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion that attaches anticipated usage metadata and directive metadata to a file by an application layer, the anticipated usage metadata containing a plurality of attributes describing characteristics of the file and the directive metadata containing directives comprising storage instructions for the file;

wherein a storage system uses the characteristics of the file from the anticipated usage metadata and the storage instructions from the directive metadata in lieu of a general storage policy to determine a file placement of the file within the storage system;

monitors for a file creation event, wherein the file is created;

attaches thereby embedding the anticipated usage metadata and the directive metadata within data content of the file by the application layer during the creation of the file;

updates the anticipated usage metadata and the directive metadata in real time according to a frequency of access of the file;

adds a set of core metadata to the file at the time of the file creation, the core metadata comprising at least a bandwidth requirement for processing the file and an importance factor of the file as assigned to the attached anticipated usage metadata and the directive metadata such that the importance factor signifies how essential the anticipated usage metadata and the directive metadata are to be followed when determining the file placement;

wherein the core metadata further indicates the importance factor to a storage controller of the storage system for performing resource allocation operations to process the file by the storage controller so as to allocate enough resources by the storage controller to the file to ensure the bandwidth requirement for processing the file is satisfied during file access;

scans the directive metadata for determining when to execute those directives contained in the directive metadata; and

executes the directive metadata.

20. The computer program product of claim 19, wherein the anticipated usage metadata includes at least one of time, scope, and duration for the file.

21. The computer program product of claim 20, further including a second executable portion that transitions from the cascading directive to the local directive after the directive metadata arrives at a target system.

22. The computer program product of claim 19, wherein the directive metadata is one of a local directive that is executed on a current system and a cascading directive that is sent to a next destination enroute to a final destination and allow for automatically executing each of an appropriate action on each destination at an appropriate time.

23. The computer program product of claim 19, further including a second executable portion that anticipates future usage patterns by using the anticipated usage metadata and the directive metadata for proactively moving data to one of a plurality of storage tiers.

24. The computer program product of claim 19, further including a second executable portion that optimizes the file placement using the anticipated usage metadata and the directive metadata.

25. The computer program product of claim 24, further including a third executable portion that extends the optimization of the file placement beyond a storage system infrastructure into an edge caching network using the anticipated usage metadata and the directive metadata.

26. The computer program product of claim 19, further including a second executable portion that performs one of:

reading the anticipated usage metadata and the directive metadata,

scheduling a send file operation for sending the file to a next destination according to the anticipated usage metadata and the directive metadata,

scheduling a send metadata operation for sending metadata to a next destination according to the anticipated usage metadata and the directive metadata.

27. The computer program product of claim 19, further including a second executable portion that performs one of:

setting up default metadata as system tunable by a file type,

attaching the default metadata on the file type,

modifying the default metadata that is attached to the file, and

using one of a plurality of defined application programming interfaces (API) for attaching to the anticipated usage metadata and the directive metadata.