Data recovery advisor

• US 10,248,483 B2
• Filed: 10/17/2008
• Issued: 04/02/2019
• Est. Priority Date: 10/19/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Dilman, Mark, Stewart, Michael James, Hu, Wei-Ming, Pareek, Alok, To, Jr., Lawrence HinLuen, Srihari, Vinay H.
• Primary Examiner(s): Allen, Brittany N
• Application Number:
  US12/253,897
• Publication Number:
  US 20090106327A1
• Time in Patent Office: 3,819 Days
• Field of Search: 707999202, 707674, 707686, 707691, 714 25, 714 26
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/327 Alarm or error message display

Abstract

A computer is programmed to identify failures and perform recovery of data. Specifically, in several embodiments, the computer is programmed to automatically check integrity of data in a storage structure to identify a set of failures related to the storage structure. The computer is further programmed in some embodiments to identify, based on one failure in the set of failures, a group of repairs to fix that one failure. Each repair in the group of repairs is alternative to another repair in the group. The computer is also programmed in some embodiments to execute at least one repair in the group of repairs, so as to generate corrected data to fix the one failure. In certain embodiments, the corrected data is stored in non-volatile storage media of the computer.

Claims

1. A computer-implemented method to diagnose and fix problems in a data storage system, the data storage system being implemented at least partially by one or more computers, the method comprising:

checking integrity of one or more components of the data storage system;

wherein a data failure is related to corruption of data in a file, the data being read by or written by or read and written by a software program, and at least the corruption of the data is identified by said checking of integrity after an error is encountered by said software program which during normal functioning is unable to process the data due to the corruption of the data;

wherein said checking of integrity comprises checking for at least existence of said file;

identifying a type of repair based at least in part on using, with a map in a memory of a computer that maps failure types to repair types, a type of the data failure related to corruption;

wherein the type of repair identifies a group of alternative repairs each of which can fix the data failure related to corruption, such that each repair in the group is an alternative to another repair in the group, wherein at least one repair in the group uses a backup of the data;

checking feasibility of the group of alternative repairs at least by checking for existence of a backup of the data in a storage device, wherein at least said checking of feasibility is performed automatically, and a plurality of feasible repairs are selected by said checking of feasibility, from among the group of alternative repairs, and said at least one repair is excluded from the plurality of feasible repairs in response to the checking feasibility being unable to find a backup of the data;

consolidating multiple repairs in the plurality of feasible repairs, based on respective impacts of the multiple repairs, into one or more repair plans;

displaying the one or more repair plans;

receiving identification of a specific repair plan selected by user input from among the one or more repair plans displayed;

performing the specific repair plan selected by the user input, to obtain corrected data to fix the corruption in the data;

storing the corrected data in non-volatile storage media of the data storage system; and

said software program at least using the corrected data;

wherein at least said checking of feasibility, said consolidating, said performing and said storing are performed by one or more processors in the one or more computers.

2. The computer-implemented method of claim 1 further comprising:

displaying a plurality of data failures including the data failure related to corruption; and

displaying an impact of the data failure related to corruption, the impact identifying in a relational database stored in the data storage system, a table that is not currently available due to the data failure related to corruption.

3. The computer-implemented method of claim 2 further comprising:

receiving identification of the data failure related to corruption, selected by additional user input, from among the plurality of data failures displayed.

4. The computer-implemented method of claim 1 further comprising, prior to said identifying:

selecting the data failure related to corruption automatically, based on a highest priority thereof, in a set of failures.

5. The computer-implemented method of claim 1 wherein:

said file is a data file of a database stored in the data storage system; and

said checking of integrity is performed automatically on said data file, in response to occurrence of the error.

6. The computer-implemented method of claim 1 wherein:

an impact of a specific repair identifies, in a relational database stored in the data storage system, a table that will be offlined during performance of the specific repair.

7. The computer-implemented method of claim 1 wherein:

said checking of integrity comprises verifying that header information recorded in said file matches corresponding information recorded for said file within another file.

8. The computer-implemented method of claim 1 wherein:

said checking of integrity comprises detecting one or more corruptions in a disk image of a block of the data in a database stored in the data storage system, by at least checking for presence of at least one known field comprising a constant as a predetermined number therein.

9. The computer-implemented method of claim 1 wherein:

said checking of integrity further comprises using the file to check for existence of multiple files identified in the file.

10. The computer-implemented method of claim 1 wherein:

an impact of a specific repair comprises an estimate of duration to perform the specific repair; and

the computer-implemented method further comprises estimating the impact based on execution of a repair script corresponding to said specific repair.

11. The computer-implemented method of claim 1 wherein:

said checking of integrity comprises using one or more tests to check at least a structure of an item of data in the data storage system; and

among said one or more tests, a test checks the structure by determining validity of a value of an attribute of the item.

12. The computer-implemented method of claim 1 further comprising, prior to said performing:

determining an order of feasible repairs that are included in the specific repair plan and arranging or re-arranging the feasible repairs in the specific repair plan, in said order.

13. The computer-implemented method of claim 1 wherein:

an impact of a specific repair is no data loss; and

another impact of another repair comprises an estimate of repair duration.

14. The computer-implemented method of claim 1 wherein:

feasibility of one repair in the group of alternative repairs depends on another repair.

15. The computer-implemented method of claim 1 wherein:

feasibility of one repair in the group of alternative repairs depends on another failure.

39. The computer-implemented method of claim 1 wherein:

said at least one repair in the group of alternative repairs comprises replacing the file which contains corrupted data comprising the corruption, with another file that contains at least said backup of the data.

40. The computer-implemented method of claim 1 wherein:

the error is encountered by the software program in a first process distinct from a second process that uses the error to diagnose the data failure; and

the first process continues to execute after encountering the error.

41. The computer-implemented method of claim 1 wherein:

at least said checking of integrity is performed asynchronously without interfering with operation of the software program; and

the software program continues to operate after logging the error.

42. The computer-implemented method of claim 1 wherein:

one or more repairs in the group of alternative repairs are created automatically by instantiation of one or more respective repair templates stored in the memory of the computer.

16. One or more non-transitory computer-readable storage media storing a plurality of instructions which when executed by one or more processors in one or more computers cause at least:

checking integrity of one or more components of a data storage system;

wherein a data failure is related to corruption of data in a file, the data being read by or written by or read and written by a software program, and at least the corruption of the data is identified by said checking of integrity after an error is encountered by said software program which during normal functioning is unable to process the data due to the corruption of the data;

wherein said checking of integrity comprises checking for at least existence of said file;

identifying a type of repair based at least in part on using, with a map in a memory of a computer that maps failure types to repair types, a type of the data failure related to corruption;

wherein the type of repair identifies a group of alternative repairs each of which can fix the data failure related to corruption, such that each repair in the group is an alternative to another repair in the group, wherein at least one repair in the group uses a backup of the data;

checking feasibility of the group of alternative repairs at least by checking for existence of a backup of the data in a storage device, wherein at least said checking of feasibility is performed automatically, and a plurality of feasible repairs are selected by said checking of feasibility, from among the group of alternative repairs, and said at least one repair is excluded from the plurality of feasible repairs in response to the checking feasibility being unable to find a backup of the data;

consolidating multiple repairs in the plurality of feasible repairs, based on respective impacts of the multiple repairs, into one or more repair plans;

displaying the one or more repair plans;

receiving identification of a specific repair plan selected by user input from among the one or more repair plans displayed;

performing the specific repair plan selected by the user input, to obtain corrected data to fix the corruption in the data;

storing the corrected data in non-volatile storage media of the data storage system; and

said software program at least using the corrected data.

17. The one or more non-transitory computer-readable storage media of claim 16 wherein the plurality of instructions include instructions which when executed by the one or more processors further cause:

displaying a plurality of failures including the data failure related to corruption; and

displaying an impact of the data failure related to corruption, the impact identifying in a relational database stored in the data storage system, a table that is not currently available due to the data failure related to corruption.

18. The one or more non-transitory computer-readable storage media of claim 17 wherein the plurality of instructions include instructions which when executed by the one or more processors further cause:

receiving identification of the data failure related to corruption, selected by additional user input, from among the plurality of failures displayed.

19. The one or more non-transitory computer-readable storage media of claim 16 wherein the plurality of instructions include instructions which when executed by the one or more processors further cause, prior to said identifying:

selecting the data failure related to corruption automatically, based on a highest priority thereof, in a set of failures.

20. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said file is a data file of a database stored in the data storage system; and

said checking of integrity is performed automatically on said data file, in response to occurrence of the error.

21. The one or more non-transitory computer-readable storage media of claim 16 wherein:

an impact of a specific repair identifies, in a relational database stored in the data storage system, a table that will be offlined during performance of the specific repair.

22. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking of integrity comprises verifying that header information recorded in said file matches corresponding information recorded for said file within another file.

23. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking of integrity comprises detecting one or more corruptions in a disk image of a block of the data in a database stored in the data storage system, by at least checking for presence of at least one known field comprising a constant as a predetermined number therein.

24. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking of integrity comprises detecting corruption of a block of data in a database stored in the data storage system and responding to finding block corruption by checking if there are additional block corruptions within a predetermined address range around said block.

25. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking of integrity further comprises using the file to check for existence of multiple files identified in the file.

26. The one or more non-transitory computer-readable storage media of claim 16 wherein the plurality of instructions include instructions which when executed by the one or more processors further cause:

displaying the data failure related to corruption, and an impact identifying in a relational database stored in the data storage system, a table that is not currently available due to the data failure related to corruption.

27. The one or more non-transitory computer-readable storage media of claim 16 wherein:

an impact of a specific repair comprises an estimate of duration to perform the specific repair; and

the plurality of instructions include instructions which when executed by the one or more processors further cause estimating the impact based on execution of a repair script corresponding to said specific repair.

28. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking of integrity comprises using one or more tests to check at least a structure of an item of data in the data storage system; and

among said one or more tests, a test checks the structure by determining validity of a value of an attribute of the item.

29. The one or more non-transitory computer-readable storage media of claim 16 wherein the plurality of instructions include instructions which when executed by the one or more processors further cause, prior to said identifying:

determining an order of feasible repairs that are included in the specific repair plan and arranging or re-arranging the feasible repairs in the specific repair plan, in said order.

30. The one or more non-transitory computer-readable storage media of claim 16 wherein:

an impact of a specific repair is no data loss; and

another impact of another repair comprises an estimate of repair duration.

31. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said checking feasibility comprises checking whether a file is present that is needed for a repair in the group of alternative repairs.

43. The one or more non-transitory computer-readable storage media of claim 16 wherein:

said at least one repair in the group of alternative repairs comprises replacing the file which contains corrupted data comprising the corruption, with another file that contains at least said backup of the data.

44. The one or more non-transitory computer-readable storage media of claim 16 wherein:

the error is encountered by the software program in a first process distinct from a second process that uses the error to diagnose the data failure; and

the first process continues to execute after encountering the error.

45. The one or more non-transitory computer-readable storage media of claim 16 wherein:

at least said checking of integrity is performed asynchronously without interfering with operation of the software program; and

the software program continues to operate after logging the error.

46. The one or more non-transitory computer-readable storage media of claim 16 wherein:

one or more repairs in the group of alternative repairs are created automatically by instantiation of one or more respective repair templates stored in the memory of the computer.

32. A system, implemented at least partially by one or more computers comprising one or more processors and one or more memories, to diagnose and fix problems in a data storage system, wherein the one or more processors are configured to at least:

check integrity of one or more components of the data storage system;

wherein a data failure is related to corruption of data in a file, the data being read by or written by or read and written by a software program, and at least the corruption of the data is identified by said check of integrity after an error is encountered by said software program which during normal functioning is unable to process the data due to the corruption of the data;

wherein said check of integrity comprises checking for at least existence of said file;

identify a type of repair based at least in part on using, with a map in a memory of a computer that maps failure types to repair types, a type of the data failure related to corruption;

wherein the type of repair identifies a group of alternative repairs each of which can fix the data failure related to corruption, such that each repair in the group is an alternative to another repair in the group, wherein at least one repair in the group uses a backup of the data;

check feasibility of the group of alternative repairs at least by checking for existence of a backup of the data in a storage device, wherein at least said check of feasibility is performed automatically, and a plurality of feasible repairs are selected by said check of feasibility, from among the group of alternative repairs, and said at least one repair is excluded from the plurality of feasible repairs in response to said check of feasibility being unable to find a backup of the data;

consolidate multiple repairs in the plurality of feasible repairs, based on respective impacts of the multiple repairs, into one or more repair plans;

display the one or more repair plans;

receive identification of a specific repair plan selected by user input from among the one or more repair plans displayed;

perform the specific repair plan selected by the user input, to obtain corrected data to fix the corruption in the data;

store the corrected data in non-volatile storage media of the data storage system; and

said software program at least uses the corrected data.

33. The system of claim 32 wherein the one or more processors are further configured to at least:

display a plurality of failures including the data failure related to corruption;

display an impact of the data failure related to corruption, the impact identifying in a relational database, a table that is not currently available due to the data failure related to corruption; and

receive identification of the data failure related to corruption, selected by additional user input, from among the plurality of failures displayed.

34. The system of claim 32 wherein:

said file is a data file of a relational database stored in the data storage system;

said check of integrity is performed automatically on said data file, in response to occurrence of the error; and

an impact of a specific repair identifies, in the relational database, a table that will be offlined during performance of the specific repair.

35. The system of claim 32 wherein:

said check of integrity further comprises using the file to check for existence of multiple files identified in the file.

36. The system of claim 32 wherein the one or more processors are further configured to at least:

display the data failure related to corruption, and an impact identifying in a relational database stored in the data storage system, a table that is not currently available due to the data failure related to corruption.

37. The system of claim 32 wherein:

an impact of a specific repair comprises an estimate of duration to perform the specific repair; and

wherein the one or more processors are further configured to at least estimate the impact based on execution of a repair script corresponding to said specific repair.

38. The system of claim 32 wherein:

an impact of a specific repair is no data loss; and

another impact of another repair comprises an estimate of repair duration.