Method and apparatus for managing storage of data

• US 8,020,047 B2
• Filed: 01/09/2007
• Issued: 09/13/2011
• Est. Priority Date: 01/17/2006
• Status: Expired due to Fees

• Current Assignee: Xyratex Technology Limited (Seagate Technology Holdings Plc)
• Original Assignee: Xyratex Technology Limited (Seagate Technology Holdings Plc)
• Inventors: Courtney, Timothy E. G.
• Primary Examiner(s): Beausoliel, Jr.; Robert W
• Assistant Examiner(s): Mehrmanesh; Elmira
• Application Number:
  US11/650,975
• Publication Number:
  US 20070180337A1
• Time in Patent Office: 1,708 Days
• Field of Search: 714/54, 714/52, 714/6.24, 711/158, 711/160
• US Class Current: 714/54
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G11B 20/18 Error detection or correction; Testing , e.g. of drop-outs
• CPC Class Codes: H03M 13/35 Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics

Abstract

In a method and apparatus for storage of data, data is stored. The stored data is checked, for example periodically and/or on data access, for development of a fault in the stored data. If a fault is detected, at least one of: (i) increased protection and (ii) increased error correction is applied to at least some of the remaining fault-free data.

Claims

1. A method of managing storage of data comprising at least one data set, the method comprising:

classifying data within a data set according to its importance in rendering the data set usable;

storing the data according to the importance of the data in the data set;

checking the stored data for development of a fault in the stored data; and

,

wherein, after a fault is detected in a data set, the data set comprises faulty data and fault-free data and the method further comprises;

analysing at least some of the remaining fault-free data in the data set and reassessing its classification according to its importance in providing a usable data set from the fault-free data alone; and

applying, in dependence upon the classification of the remaining fault-free data in the data set, to at least some of the remaining fault-free data in the data set at least one of;

(i) increased protection and (ii) increased error correction.

2. A method according to claim 1, wherein the checking of the stored data for development of a fault in the stored data is carried out periodically.

3. A method according to claim 1, wherein the checking of the stored data for development of a fault in the stored data is carried out as the stored data is accessed.

4. A method according to claim 3, wherein the checking of the stored data for development of a fault in the stored data is additionally carried out periodically if the stored data has not been accessed for a certain period of time.

5. A method according to claim 1, wherein said data is stored in at least one data storage device, said at least one of (i) increased protection and (ii) increased error correction being applied to at least some of the data that is physically adjacent the faulty data in said at least one data storage device.

6. A method according to claim 1, wherein said at least one of (i) increased protection and (ii) increased error correction is applied to at least some of the data that is adjacent the faulty data in a file that is made up of said data.

7. A method according to claim 1, wherein said at least one of (i) increased protection and (ii) increased error correction is applied to at least some of the data that is logically temporally adjacent the faulty data in a data stream that is made up of said data.

8. A method according to claim 1, wherein data that is classed as being of relatively low importance is stored with a relatively low level of protection or no protection, and data that is classed as being of relatively high importance is stored with a relatively high level of protection.

9. A method according to claim 1, wherein data that is classed as being of relatively low importance is stored with a relatively low level of error correction or no error correction, and data that is classed as being of relatively high importance is stored with a relatively high level of error correction.

10. A method according to claim 1, wherein the data is stored in at least one storage device, and comprising, when a fault in the data stored in said at least one storage device is detected, said data storage device returning information about why the fault in the data occurred.

11. A method according to claim 1, wherein the data is stored in at least one storage device, and comprising, when a fault in the data stored in said at least one storage device is detected, determining the likelihood of said at least one data storage device suffering complete failure within a certain time period and moving the data from said at least one data storage device to another data storage device if said likelihood exceeds a threshold.

12. A method according to claim 1, comprising sending a prompt to a user to check that the remaining fault-free data provides a useable data set.

13. A method according to claim 1, wherein the faulty stored data is lost and not recovered.

14. A method of managing storage of data comprising at least one data set, the method comprising:

classifying data within a data set according to its importance in rendering the data set usable;

storing data in at least one storage device;

monitoring the at least one storage device to determine whether the at least one storage device has failed or is likely to fail within a certain time period; and

,

if it is determined that the at least one storage device has failed or is likely to fail within a certain time period such that data in a data set has become faulty or is likely to become faulty, analysing at least some of the remaining fault-free data in the data set and reassessing its classification according to its importance in rendering the data set usable from the fault-free data alone; and

,

maintaining or changing the manner in which the data is stored dependent upon the classification of the data in the data set in providing a usable data set.

15. A method according to claim 14, wherein the manner in which the data is stored is changed by increasing protection that is applied to at least some of the stored data.

16. A method according to claim 14, wherein the manner in which the data is stored is changed by increasing error protection that is applied to at least some of the stored data.

17. Apparatus for managing storage of data comprising at least one data set, the apparatus comprising:

a classifier arranged to classify data within a data set according to its importance in rendering the data set usable; and

a data storage manager arranged to store the data according to the importance of the data in the data set and to check the stored data for development of a fault in the stored data, wherein, after a fault is detected in a data set, the data set comprises faulty data and fault-free data and, the data storage manager being further arranged;

to analyse at least some of the remaining fault-free data in the data set and reassess its classification according to its importance in providing a usable data set from the fault-free data alone and to cause, in dependence upon the classification of the remaining fault-free data in the data set, application to at least some of the remaining fault-free data in the data set at least one of;

(i) increased protection and (ii) increased error correction.

18. Apparatus according to claim 17, wherein the data storage manager is arranged to check the stored data periodically for development of a fault in the stored data.

19. Apparatus according to claim 17, wherein the data storage manager is arranged to check the stored data as the stored data is accessed.

20. Apparatus according to claim 19, wherein the data storage manager is arranged additionally to check the stored data periodically if the stored data has not been accessed for a certain period of time.

21. Apparatus according to claim 17, wherein said data is stored in at least one data storage device, and wherein the data storage manager is arranged to cause said at least one of (i) increased protection and (ii) increased error correction to be applied to at least some of the data that is physically adjacent the faulty data in said at least one data storage device.

22. Apparatus according to claim 17, wherein the data storage manager is arranged to cause said at least one of (i) increased protection and (ii) increased error correction to be applied to at least some of the data that is adjacent the faulty data in a file that is made up of said data.

23. Apparatus according to claim 17, wherein the data storage manager is arranged to cause said at least one of (i) increased protection and (ii) increased error correction to be applied to at least some of the data that is logically temporally adjacent the faulty data in a data stream that is made up of said data.

24. Apparatus according to claim 17, wherein the data storage manager is arranged so that data that is classed as being of relatively low importance is stored with a relatively low level of protection or no protection, and data that is classed as being of relatively high importance is stored with a relatively high level of protection.

25. Apparatus according to claim 17, wherein the data storage manager is arranged so that data that is classed as being of relatively low importance is stored with a relatively low level of error correction or no error correction, and data that is classed as being of relatively high importance is stored with a relatively high level of error correction.

26. Apparatus according to claim 17, wherein the data is stored in at least one storage device, the data storage device being arranged so that;

when a fault in the data stored in said at least one storage device is detected, said data storage device returns information about why the fault in the data occurred.

27. Apparatus according to claim 17, wherein the data is stored in at least one storage device, the data storage device being arranged so that, when a fault in the data stored in said at least one storage device is detected, the likelihood of said at least one data storage device suffering complete failure within a certain time period is determined and so that the data is moved from said at least one data storage device to another data storage device if said likelihood exceeds a threshold.

28. Apparatus according to claim 17, wherein the data storage manager is arranged so that a prompt is sent to a user to check that the remaining fault-free data provides a useable data set.

29. Apparatus according to claim 17, wherein the faulty stored data is lost and not recovered.

30. Apparatus for managing storage of data comprising at least one data set, the apparatus comprising:

a classifier arranged to classify data within a data set according to its importance in rendering the data set usable;

at least one storage device in which data can be stored according to the importance of the data in the data set; and

,

a data storage manager arranged to monitor the at least one storage device to determine whether the at least one storage device has failed or is likely to fail within a certain time period, and, if it is determined that the at least one storage device has failed or is likely to fail within a certain time period, such that data in a data set has become faulty or is likely to become faulty; and

to analyse at least some of the remaining fault-free data in the data set and to reassess its classification according to its importance in rendering the data set usable from the fault-free data alone; and

to cause the manner in which the data is stored to be maintained or changed dependent upon the classification of the data in the data set in providing a usable data set.

31. Apparatus according to claim 30, wherein the data storage manager is arranged so that the manner in which the data is stored is changed by increasing protection that is applied to at least some of the stored data.

32. Apparatus according to claim 30, wherein the data storage manager is arranged so that the manner in which the data is stored is changed by increasing error protection that is applied to at least some of the stored data.

33. A method of managing storage of data comprising at least one data set, the method comprising:

classifying data within a data set according to its importance in rendering the data set usable;

storing the data according to the importance of the data in the data set; and

checking the stored data for development of a fault in the stored data;

wherein, after a fault is detected in a data set, the data set comprises faulty data and fault-free data and the method further comprises;

analysing at least some of the remaining fault-free data in the data set and reassessing its classification according to its importance in providing a usable data set from the fault-free data alone; and

determining, based upon said classification, whether measures are required to maintain a usable data set and, if so, applying, in dependence upon the classification of the remaining fault-free data in the data set, to at least some of the remaining fault-free data in the data set at least one of;

(i) protection or increased protection and (ii) error correction or increased error correction.

34. A method according to claim 33, wherein data that is classed as being of relatively low importance is stored with a relatively low level of protection or no protection, and data that is classed as being of relatively high importance is stored with a relatively high level of protection.

35. A method according to claim 33, wherein data that is classed as being of relatively low importance is stored with a relatively low level of error correction or no error correction, and data that is classed as being of relatively high importance is stored with a relatively high level of error correction.

36. An apparatus for managing storage of data comprising at least one data set, the apparatus comprising:

a classifier arranged to classify data within a data set according to its importance in rendering the data set usable;

a data storage manager arranged;

to store the data according to the importance of the data in the data set; and

to check the stored data for development of a fault in the stored data, wherein, after a fault is detected in a data set, the data set comprises faulty data and fault-free data, the classifier being arranged to analyse at least some of the remaining fault-free data and reassess its classification according to its importance in providing a usable data set from the remaining fault-free data alone after a fault has been detected and the data storage manager being further arranged to determine, based upon said classification, whether measures are required to maintain a usable data set and, if so, to cause application, in dependence upon the classification of the remaining fault-free data in the data set, to at least some of the remaining fault-free data at least one of;

(i) increased protection and (ii) increased error correction.

37. Apparatus according to claim 36, wherein the data storage manager is arranged so that data that is classed as being of relatively low importance is stored with a relatively low level of protection or no protection, and data that is classed as being of relatively high importance is stored with a relatively high level of protection.

38. Apparatus according to claim 36, wherein the data storage manager is arranged so that data that is classed as being of relatively low importance is stored with a relatively low level of error correction or no error correction, and data that is classed as being of relatively high importance is stored with a relatively high level of error correction.