Wear leveling and management in an electronic environment

• US 9,450,876 B1
• Filed: 03/13/2013
• Issued: 09/20/2016
• Est. Priority Date: 03/13/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Marr, Michael David
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Ye, Zi
• Application Number:
  US13/799,333
• Time in Patent Office: 1,287 Days
• Field of Search: 709/224, 709/223, 709/226, 711/103, 365/148
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/1036 Life time enhancement
• CPC Class Codes: G06F 2212/7204 Capacity control, e.g. partitioning, end-of-life degradation
• CPC Class Codes: G06F 2212/7208 Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 2212/7211 Wear leveling
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering

Abstract

Workloads can be intelligently placed across a group of resources in order to attempt to balance or otherwise manage the level of wear among various components of those resources. Devices such as solid state drives or other NAND-type devices can have a limited number of operations that can be performed before those devices become unreliable, such that it can be desirable to monitor the wear level of each of these devices. As it can be easier to manage resources with similar wear levels for large groups of resources, it can be desirable to attempt to level the relative amount of wear among at least groups of these resources. Attempts can be made to level across a fleet or resources, within pools of resources, and/or within the resources themselves, such as where a server includes multiple devices with potentially different wear levels, such as multiple NAND-type devices.

Claims

1. A computer-implemented method of distributing workloads, comprising:

receiving a request to process a workload using at least one of a plurality of host machines operating in a multi-tenant environment;

analyzing at least one of information for the request or information for a source of the request to determine an expected amount of wear resulting from processing the workload, wherein the wear corresponds to a reduction of available read-write cycles in a memory device;

analyzing each of at least a portion of the plurality of host machines to determine an average wear level of one or more memory devices installed in the host machine, the average wear level for the one or more memory devices determined by analyzing information provided by the one or more memory devices installed in the host machine, each of the one or more memory devices having a finite number of read-write cycles;

balancing the average wear levels of the one or more memory devices across at least the analyzed host machines based at least in part upon the average wear level of each of one or more memory devices installed in the analyzed host machines and the expected amount of wear resulting from processing the workload;

selecting a host machine based at least in part on the balancing and factors associated with the workload of the one or more memory devices;

sending information to process the workload in the request to the selected host machine of the analyzed host machines;

analyzing the processing of the workload;

determining that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least a determined threshold amount; and

moving at least a portion of the workload to a second host machine of the analyzed host machines based at least in part upon the determination that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least the determined threshold amount, wherein the second host machine is selected based at least in part on the balancing and factors associated with the portion of the workload.

2. The computer-implemented method of claim 1, further comprising:

allocating each of the plurality of host machines into one of a plurality of host groups, each host group comprising host machines with average memory device wear levels that are within a specified amount of wear deviation.

3. The computer-implemented method of claim 2, further comprising:

analyzing the average wear level of the one or more memory devices installed in the selected host machine after processing the information for the request; and

moving the selected host machine to a different host group when the average wear level of the one or more memory devices installed in the selected host machine falls outside the specified amount of wear deviation for a current host group of the selected host machine.

4. The computer-implemented method of claim 1, further comprising:

analyzing the processing of the workload on the selected host machine; and

moving a remaining portion of the processing of the workload to a different host machine if an amount of capacity needed to process the workload exceeds an expected amount of capacity needed to process the request.

5. A system, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, causes the system to;

obtain a workload to be processed using a selected resource of a set of resources;

determine an expected amount of wear resulting from processing the workload based at least in part upon information for one or more similar workloads previously processed using the set of resources, wherein the wear corresponds to a reduction of available read-write cycles in at least a portion of the set of resources;

determine a wear level of each of the at least a portion of the set of resources;

balance the wear levels of each of the at least a portion of the set of resources, the balancing based at least in part upon the relative wear of the selected resource and the expected amount of wear associated with processing the workload;

select a resource from the set of resources based at least in part on the balance of the wear levels and resource data received from the set of resources;

transmit information to process the workload in a request to the selected resource;

analyze the processing of the workload;

determine that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least a determined threshold amount; and

move at least a portion of the workload to a second resource of the set of resources based at least in part upon the determination that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least the determined threshold amount, wherein the second resource is selected based at least in part on the balancing and factors associated with the portion of the workload.

6. The system of claim 5, wherein the instructions when executed further cause the system to:

analyze the set of resources to attempt to locate a resource with the least amount or highest amount of wear to select as the selected resource.

7. The system of claim 5, wherein the instructions when executed further cause the system to:

group subsets of the resources by relative amounts of wear, wherein determining the selected resource includes determining a selected subset of the resources and analyzing the relative wear of the resources in the selected subset; and

place subsequent workloads on resources in order to reduce a number of the group subsets.

8. The system of claim 5, wherein the resource data comprises a temporal factor associated with a workload to the resource, a number of concurrent connections to the resource, a number of workloads being currently processed by each resource, or an age of each resource.

9. The system of claim 5, wherein the instructions when executed further cause the system to:

normalize wear data received from the set of resources, the set of resources capable of including resources of different types that report different types of wear data.

10. The system of claim 5, wherein the resources comprise at least one of host machines including one or more memory devices known to wear over time, or one or more memory devices known to wear over time that are installed within a host machine.

11. The system of claim 5, wherein the information for one or more similar workloads includes information for similar workloads submitted by a customer associated with the workload to be processed.

12. The system of claim 11, wherein the instructions when executed further cause the system to:

monitor the processing of the workload using the selected resource or the second resource;

determine that a number of operations required to process the workload exceeds an expected number of operations to process the workload based at least in part on the monitoring; and

move a remaining portion of the workload to another resource based at least in part on the determination that the number of operations required to process the workload exceeds the expected number of operations to process the workload.

13. The system of claim 5, wherein the selected resource is determined in order to minimize the variation in wear across the set of resources or accelerate the wear of an identified resource.

14. The system of claim 5, wherein the set of resources includes at least one of solid state drives, flash memory devices, or NAND devices having a finite number of read-write cycles.

15. The system of claim 5, wherein the instructions when executed further cause the system to:

generate an alarm when a resource or a subset of the resources approaches zero remaining lifetime.

16. The system of claim 5, wherein the instructions when executed further cause the system to:

generate an alarm when a variation in wear level across the resources exceeds an allowable amount of variation.

17. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computer system, cause the computer system to:

determine historical information for a user associated with a workload to be processed;

determine an expected amount of wear associated with processing the workload based at least in part upon the historical information for the user, wherein the wear is associated with a reduction of available read-write cycles in a memory resource;

determine a wear level of each of at least a portion of a set of memory resources;

balance an average wear level across at least the portion of the set of memory resources based at least in part upon the wear level of each of at least the portion of the set of memory resources and the expected amount of wear associated with processing the workload;

determine a selected memory resource from the set of memory resources based at least in part on the balance of the wear level and one or more factors associated with the set of memory resources;

transmit information to process the workload in a request to the selected memory resource;

analyze the processing of the workload;

determine that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least a determined threshold amount; and

move at least a portion of the workload to a second memory resource of the set of memory resources based at least in part upon the determination that the actual wear resulting from the processing of the workload exceeds the expected amount of wear by at least the determined threshold amount, wherein the second memory resource is selected based at least in part on the balancing and factors associated with the portion of the workload.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the computer system to:

normalize wear data received from the set of resources, the set of resources including resources of different types that report different types of wear data.

19. The non-transitory computer-readable storage medium of claim 17, wherein the resources comprise at least one of host machines including one or more memory devices known to wear over time, or one or more memory devices known to wear over time that are installed within a host machine.

20. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the computer system to:

analyze the set of resources to attempt to locate a resource with the least amount or highest amount of wear to select as the selected resource.

21. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the computer system to:

group subsets of the resources by relative amounts of wear, wherein determining the selected resource includes determining a selected subset of the resources and analyzing the relative wear of the resources in the selected subset.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed further cause the computer system to:

attempt to place subsequent workloads to reduce a number of the group subsets.

23. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the computer system to:

monitor the processing of the workload using the selected memory resource or the second memory resource;

determine that a number of operations required to process the workload exceeds an expected number of operations to process the workload based at least in part on the monitoring; and

move a remaining portion of the workload to another memory resource based at least in part on the determination that the number of operations required to process the workload exceeds the expected number of operations to process the workload.

24. The non-transitory computer-readable storage medium of claim 17, wherein the set of resources includes at least one of solid state drives, flash memory devices, or NAND devices having a finite number of read-write cycles.

25. The non-transitory computer-readable storage medium of claim 17, wherein the one or more factors comprises a temporal factor associated with a workload to the memory resource, a number of concurrent connections to the memory resource, or a number of workloads being currently processed by each memory resource.