Intelligent data routing and storage provisioning

• US 10,394,766 B2
• Filed: 11/03/2016
• Issued: 08/27/2019
• Est. Priority Date: 04/02/2013
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Dejana, Ryan G., Deluca, Lisa Seacat, Goodman, Brian D., Krook, Daniel C.
• Primary Examiner(s): Willoughby, Alicia M
• Application Number:
  US15/342,426
• Publication Number:
  US 20170060859A1
• Time in Patent Office: 1,027 Days
• Field of Search: None
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Approaches for routing data to storage are provided. An approach includes determining implicit metadata from explicit metadata received with a request from a user to store a file. The approach also includes determining a storage resource based on the explicit metadata, the implicit metadata, and a registry of storage resources. The approach additionally includes routing data of the file to the determined storage resource.

Claims

1. A method for routing data to storage comprising:

receiving, from a remote computing device, a user request to store a file and the file to be stored;

parsing both the request and the file to be stored into data comprising raw data and explicit metadata describing both the request and the file to be stored;

determining implicit metadata from the explicit metadata;

validating the explicit metadata and the implicit metadata by comparing to schema stored in storage resources that include predefined configurations of processors, memories, storage devices and data structures;

determining a storage resource based on the explicit metadata, the implicit metadata, and a registry of the storage resources,

wherein the registry of storage resources includes information describing characteristics of available storage resources and the determining comprises comparing the implicit metadata and the explicit metadata to the characteristics of the available storage resources, wherein the characteristics of the available storage resources are regularly updated, and wherein the explicit metadata comprises information provided by the file to be stored and information provided by the user in the request;

routing the file to be stored to the determined storage resource; and

tracking access history and patterns to re-determine a storage location based on the access history and patterns,

wherein the determining implicit metadata and the determining the storage resource are performed by a computing device comprising at least one processor.

2. The method of claim 1, wherein the determining the implicit metadata comprises determining information about the user, the request, and the file.

3. The method of claim 2, wherein the determining implicit metadata uses one or more predefined policies corresponding to the explicit metadata.

4. The method of claim 1, wherein the determining the storage resource comprises selecting one of a plurality of storage resources from the registry of storage resources.

5. The method of claim 1, wherein at least two of the plurality of storage resources are provided by remote storage services operated by different entities.

6. The method of claim 1, wherein the determining the storage resource comprises dynamically provisioning a new storage resource based on the implicit metadata and the explicit metadata, the provisioning including:

selecting a platform template from a group of platform templates, the platform template having characteristics that suit the explicit metadata and the implicit metadata; and

configuring at least one of the available storage resources based on the selected platform template to generate the new storage resource.

7. The method of claim 6, wherein the provisioning the new storage resource comprises selecting the at least one of the available storage resource from a pool of available storage resource elements.

8. The method of claim 6, wherein the provisioning the new storage resource comprises determining available storage resources which can be converted and converting the available storage resources which can be converted to generate the new storage resource.

9. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports a computer infrastructure that determines the implicit metadata, determines the storage resource, and routes the data of the file.

10. The method of claim 1, wherein steps of claim 1 are provided by a service provider on a subscription, advertising, and/or fee basis.

11. The method of claim 1, wherein:

the explicit metadata comprises a plurality of explicit metadata and the implicit metadata comprises a plurality of implicit metadata;

the registry of storage resources comprises predefined information describing characteristics of available storage resources; and

the determining the storage resource comprises comparing the plurality of explicit metadata and the plurality of implicit metadata to the predefined information describing characteristics of available storage resources.

12. The method of claim 1, wherein:

the explicit metadata is selected from the group consisting of;

file name, file type, file size, file creation date, file modification data, file owner, file author, storage costs, redundancy type, quality type, durability type, confidentiality type, legal type, user-defined exceptions and combinations thereof;

the implicit metadata determined from the explicit metadata is selected from the group consisting of;

data type, data size, confidentiality type, legal type, location type, access type, user type, device type and combinations thereof; and

the determining the implicit metadata from the explicit metadata comprises utilizing rules stored in a controller engine that associate one or more types of explicit metadata with implicit metadata.

13. The method of claim 1, wherein the determining implicit metadata comprises utilizing stored policies associating one or more types of explicit metadata with types of implicit metadata to derive the implicit metadata from the explicit metadata.

14. The method of claim 13, wherein the implicit metadata is selected from the group consisting of:

legal types, confidentiality types and combinations thereof.

15. The method of claim 14, wherein the implicit metadata comprises confidentiality types selected from the group consisting of:

confidential type, secret type, privileged type, personal type, and combinations thereof.

16. A system for routing data to storage, the system comprising:

a central processing unit (CPU), a computer readable memory and a computer readable storage media;

program instructions to receive a storage request to store a file and the file to be stored;

program instructions to parse both the storage request and the file to be stored into data comprising raw data and explicit metadata describing the storage request and the file to be stored;

program instructions to check storage resources and locations to determine what storage resources or locations are no longer in use or are currently available by identifying the storage resources and the locations that have been explicitly released;

program instruction to identify the storage resources and the locations that have not been accessed in a given period of time;

program instruction to identify new resources or locations that are available to the system;

program instructions to determine that an appropriate storage resource for storing the file is unavailable from at least one of a predefined pool of remote storage providers based on the explicit metadata received with the storage request, implicit metadata derived from the explicit metadata based on rules associating one or more types of explicit metadata with implicit metadata, and a registry of storage resources, wherein the implicit metadata is different from the explicit metadata and includes data type, data size, confidentiality type, legal type, location type, access type, user type and device type;

program instructions to provision a new remote storage resource when it is determined that the appropriate storage resource is unavailable;

program instructions to store the file in the new remote storage resource; and

program instructions to de-provision the new remote storage resource to meet demands of different user data,

wherein each of the program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

17. The system of claim 16, further comprising:

program instructions to iteratively monitor a plurality of registered storage resources;

program instructions to determine that one or more of the plurality of registered storage resources can be recycled; and

program instructions to store identifiers of elements of the one or more of the plurality of registered storage resources in a resource pool and mark the elements as available.

18. The system of claim 16, wherein:

the explicit metadata is selected from the group consisting of;

file name, file type, file size, file creation date, file modification data, file owner, file author, storage costs, redundancy type, quality type, durability type, user-defined exceptions and combinations thereof; and

the system further comprises program instructions to derive the implicit metadata from the explicit metadata utilizing the rules stored in a controller engine.

19. The system of claim 16, wherein the implicit metadata comprises confidentiality types selected from the group consisting of:

confidential type, secret type, privileged type, personal type, and combinations thereof.

20. The system of claim 16, wherein the provisioning comprises:

selecting a platform template from a group of platform templates, the platform template having characteristics that suit the explicit metadata and the implicit metadata; and

configuring at least one of the available storage resources based on the selected platform template to generate the new storage resource.