Multi-tier data storage using burst buffer appliance for workload transformation based on performance characteristics of at least one tier
First Claim
1. A multi-tier storage system, comprising:
- at least first and second storage tiers comprising respective disjoint subsets of storage for storing data from at least one compute cluster comprised of a plurality of compute nodes, wherein said first storage tier comprises at least one burst buffer appliance, wherein said at least one burst buffer appliance comprises a memory for storing data and at least one processing device to transform at least a portion of said data for storage on said second storage tier based on a mapping of a plurality of said burst buffer appliances to at least one corresponding stripe on said second storage tier, wherein the transformation of the data by the at least one processing device of the at least one burst buffer reorganizes an input-output workload into a network stream having an improved alignment, relative to an alignment prior to said transformation, for transferring said data in parallel from said first storage tier to corresponding objects in said second storage tier.
3 Assignments
0 Petitions
Accused Products
Abstract
Improved multi-tier storage techniques are provided for storing data, such as checkpoints or other bursty data, in parallel computing environments. A burst buffer appliance is provided for use in a first storage tier of a multi-tier storage system comprising at least the first storage tier and a second storage tier. The exemplary burst buffer appliance comprises a memory for storing data; and at least one processing device to transform at least a portion of the data for storage on the second storage tier based on one or more performance characteristics of the second storage tier. In at least one embodiment, the at least one processing device is further configured to perform at least one function on the at least the portion of the data on behalf of the second storage tier. The performance characteristics of the second storage tier comprise, for example, a stripe size and/or network topology information.
8 Citations
20 Claims
-
1. A multi-tier storage system, comprising:
at least first and second storage tiers comprising respective disjoint subsets of storage for storing data from at least one compute cluster comprised of a plurality of compute nodes, wherein said first storage tier comprises at least one burst buffer appliance, wherein said at least one burst buffer appliance comprises a memory for storing data and at least one processing device to transform at least a portion of said data for storage on said second storage tier based on a mapping of a plurality of said burst buffer appliances to at least one corresponding stripe on said second storage tier, wherein the transformation of the data by the at least one processing device of the at least one burst buffer reorganizes an input-output workload into a network stream having an improved alignment, relative to an alignment prior to said transformation, for transferring said data in parallel from said first storage tier to corresponding objects in said second storage tier. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
8. A burst buffer appliance for use in a first storage tier of a multi-tier storage system comprising at least said first storage tier and a second storage tier, said burst buffer appliance comprising:
-
a memory for storing data; and at least one processing device to transform at least a portion of said data for storage on said second storage tier based on a mapping of a plurality of said burst buffer appliances to at least one corresponding stripe on said second storage tier, wherein the transformation of the data by the at least one processing device of the at least one burst buffer reorganizes an input-output workload into a network stream having an improved alignment, relative to an alignment prior to said transformation, for transferring said data in parallel from said first storage tier to corresponding objects in said second storage tier. - View Dependent Claims (9, 10, 11, 12, 13)
-
-
14. A method performed by a burst buffer appliance in a first storage tier of a multi-tier storage system, comprising the steps of:
-
obtaining data in said multi-tier storage system, wherein said multi-tier storage system comprises at least said first storage tier and a second storage tier for storing said data; and transforming at least a portion of said data for storage on said second storage tier using at least one processing device based on a mapping of a plurality of said burst buffer appliances to at least one corresponding stripe on said second storage tier, wherein the transformation of the data by the at least one processing device of the at least one burst buffer reorganizes an input-output workload into a network stream having an improved alignment, relative to an alignment prior to said transformation, for transferring said data in parallel from said first storage tier to corresponding objects in said second storage tier. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification