Timer-based apparatus and method for fault-tolerant booting of a storage controller

• US 7,523,350 B2
• Filed: 05/27/2005
• Issued: 04/21/2009
• Est. Priority Date: 04/01/2005
• Status: Expired due to Fees

• Current Assignee: Dot Hill Systems Corporation (Seagate Technology Holdings Plc)
• Original Assignee: Dot Hill Systems Corporation (Seagate Technology Holdings Plc)
• Inventors: Wang, Yuanru Frank, Lintz, Dwight Oliver Jr.
• Primary Examiner(s): DUNCAN, MARC M
• Application Number:
  US11/140,106
• Publication Number:
  US 20060236150A1
• Time in Patent Office: 1,425 Days
• Field of Search: 714/36, 714/51, 714/55
• US Class Current: 714/36
• CPC Class Codes: G06F 11/1417 Boot up procedures

Abstract

A fault tolerant storage controller having a processor, redundant copies of a stored program, and a timer that automatically runs when the processor is reset is disclosed. Selection logic selects a first copy of the program to boot on the processor. If the timer expires before the first copy successfully boots, the timer resets the processor and re-enables itself to run again. This time, selection logic selects a second copy of the stored program. In one embodiment, the program comprises separate loader and application programs, each having a redundant copy. The loader re-enables the timer when jumping to the first copy of the application code. If the timer expires before the first application copy successfully boots, the timer resets the processor and re-enables itself to run again. This time, the loader selects a second copy of the application program. In one embodiment, the redundant copies are stored in separate FLASH devices; in another, in distinct regions of the same FLASH device.

Claims

1. An apparatus for automatically selectively booting from redundant copies of a stored program in a storage-controller to tolerate a fault in up to all but one of the copies, the controller having a processor that begins fetching instructions of the stored program each time the processor is reset, the apparatus comprising:

a timer, coupled to the processor, configured to commence running when the processor is reset a first time, and to reset the processor a second time if said timer expires, wherein said timer is further configured to recommence runing when resetting the processor said second time, and to reset the processor a third time if said timer expires after re-commencing; and

selection logic, coupled to the processor, configured to select a first of the redundant copies for provision to the processor in response to said first reset, and to select a second of the redundant copies other than said first of the redundant copies for provision to the processor in response to said second reset, wherein said selection logic is further configured to select said first of the redundant copies for provision to the processor when said timer resets the processor said third time, wherein the storage controller comprises a redundant array of inexpensive disks (RAID) controller, wherein the stored program comprises a program for controlling a redundant array of inexpensive disks.

2. The apparatus as recited in claim 1, wherein said timer is configured to commence running when the processor is reset regardless of whether the processor executes any of the instructions of the stored program.

3. The apparatus as recited in claim 1, wherein said timer is further configured to set an indicator to indicate said first of the redundant copies has failed to boot if said timer expires, wherein said selection logic is configured to select said second of the redundant copies other than said first of the redundant copies for provision to the processor in response to said second reset based on said indicator.

4. The apparatus as recited in claim 3, wherein said indicator retains a value set by said timer after said timer resets the processor said second time, wherein said value indicates said first of the redundant copies has failed to boot.

5. The apparatus as recited in claim 1, wherein said selection logic is further configured to select a third of the redundant copies other than said first and second of the redundant copies for provision to the processor when said timer resets the processor said third time.

6. The apparatus as recited in claim 5, wherein said selection logic is further configured to continue to select subsequent of the redundant copies other than previously selected of the redundant copies for provision to the processor each time said timer resets the processor.

7. The apparatus as recited in claim 6, wherein said first, second and subsequent copies of the stored program are stored in respective first, second and subsequent non-volatile memories of the controller.

8. The apparatus as recited in claim 6, wherein said first, second and subsequent copies of the stored program are stored in a single non-volatile memory of the controller.

9. The apparatus as recited in claim 6, wherein said first, second and subsequent copies of the stored program are stored in a first number of non-volatile memories of the controller, wherein said first number is greater than one but fewer than a second number of said first, second, and subsequent copies of the stored program.

10. The apparatus as recited in claim 1, wherein the stored program includes instructions to disable said timer after successfully booting.

11. The apparatus as recited in claim 10, wherein said successfully booting comprises the stored program copying itself from a non-volatile memory to a random access memory (RAM) of the controller.

12. The apparatus as recited in claim 11, wherein the stored program comprises a loader program and an application program, wherein said successfully booting further comprises said loader program executing from said RAM and successfully copying said application program from said non-volatile memory to said RAM from which the processor may subsequently execute said application program.

13. The apparatus as recited in claim 12, wherein said successfully booting further comprises said application program successfully executing from said RAM.

14. The apparatus as recited in claim 1, wherein said first and second copies of the stored program are stored in a non-volatile memory of the controller in respective distinct first and second address ranges of said non-volatile memory.

15. The apparatus as recited in claim 14, wherein said selection logic is configured to generate a portion of an address for provision to said non-volatile memory, wherein said selection logic is configured to selectively generate a first predetermined value on said portion of said address to cause said first address range to be selected in response to said first reset, and to selectively generate a second predetermined value on said portion of said address to cause said second address range to be selected in response to said second reset.

16. The apparatus as recited in claim 14, wherein the stored program comprises a loader program and redundant copies of an application program, wherein said loader program is configured to selectively load one of said redundant copies of said application program into a random access memory (RAM) of the controller.

17. The apparatus as recited in claim 16, wherein first and second of said redundant copies of said application program are stored in said respective distinct first and second address ranges of said non-volatile memory.

18. The apparatus as recited in claim 14, wherein said non-volatile memory comprises a FLASH memory.

19. The apparatus as recited in claim 1, wherein said first and second copies of the stored program are stored in respective first and second non-volatile memories of the controller.

20. The apparatus as recited in claim 19, wherein said selection logic is configured to generate first and second reset signals to said respective first and second non-volatile memories to hold all but one of said first and second non-volatile memories in reset.

21. The apparatus as recited in claim 20, wherein said selection logic is configured to generate a true value on said first reset signal to said first non-volatile memory in response to said second reset of the processor, and to generate a true value on said second reset signal to said second non-volatile memory in response to said first reset of the processor.

22. The apparatus as recited in claim 20, wherein said first and second non-volatile memories are coupled to the processor by a shared bus.

23. The apparatus as recited in claim 22, wherein said first and second non-volatile memories comprise low pin count (LPC) FLASH memory devices coupled to the processor by said shared bus.

24. The apparatus as recited in claim 19, wherein the stored program comprises a loader program for selectively loading one of redundant copies of an application program into a random access memory (RAM) of the controller.

25. The apparatus as recited in claim 24, wherein first and second redundant copies of said application program are stored in said respective first and second non-volatile memories.

26. The apparatus as recited in claim 24, wherein said redundant copies of said application program are stored in a third non-volatile memory of the controller.

27. The apparatus as recited in claim 1, wherein the stored program comprises a loader program for selectively loading redundant copies of an application program into a random access memory (RAM) of the controller, wherein said loader program is configured to load a first of said redundant copies of said application program into said RAM, to subsequently disable and re-enable said timer, and to subsequently transfer control to said first of said redundant copies of said application program in said RAM.

28. The apparatus as recited in claim 27, wherein said timer comprises a control register, wherein said loader program is configured to atomically disable and re-enable said timer by a single write operation to said control register.

29. The apparatus as recited in claim 28, wherein said loader program is configured to load a second of said redundant copies of said application program into said RAM, rather than said first of said redundant copies of said application program, if said timer expired after said loader program re-enabled said timer.

30. The apparatus as recited in claim 28, wherein said timer is configured to reset the processor a third time if said timer expires after said loader program re-enables said timer, wherein said selection logic is configured to select said first of the redundant copies of said loader program for provision to the processor in response to said third reset if said first of the redundant copies of said loader program successfully booted after the processor was reset said first time.

31. The apparatus as recited in claim 30, wherein said selection logic is configured to select a second of redundant copies of said loader program for provision to the processor in response to said third reset if said second of the redundant copies of said loader program successfully booted after the processor was reset said second time.

32. The apparatus as recited in claim 1, wherein the processor is reset said first time by a power-on reset.

33. The apparatus as recited in claim 1, wherein the processor is reset said first time by an external reset of the controller.

34. The apparatus as recited in claim 1, wherein the stored program is stored in at least one non-volatile memory.

35. The apparatus as recited in claim 34, wherein said at least one non-volatile memory comprises at least one FLASH memory.

36. The apparatus as recited in claim 35, wherein said at least one FLASH memory is programmable during operation of the controller.

37. The apparatus as recited in claim 35, wherein said at least one FLASH memory is erasable during operation of the controller.

38. A storage controller for providing an improved probability of successfully booting, comprising:

first and second copies of a stored program;

a microprocessor, coupled to selectively boot said first and second copies of said stored program;

selection logic, coupling said microprocessor to said first and second stored program copies, wherein said selection logic initially selects said first stored program copy for said microprocessor to boot; and

a timer, coupled to the microprocessor, configured to reset said microprocessor a first time prior to said selection logic initially selecting said first stored program copy for said microprocessor to boot, said timer further configured to reset said microprocessor a second time if said microprocessor fails to boot said first stored program copy within a predetermined time and to update said selection logic to select said second stored program copy for said microprocessor to boot, said timer further configured to reset said microprocessor a third time if said microprocessor fails to boot said second stored program copy within a predetermined time and to update said selection logic to select said first stored program copy for said microprocessor to boot, wherein the storage controller comprises a redundant array of inexpensive disks (RAID) controller, wherein the stored program comprises a program for controlling a redundant array of inexpensive disks.

39. The storage controller as recited in claim 38, wherein said first and second stored program copies are stored in first and second non-volatile memory devices, respectively.

40. The storage controller as recited in claim 39, wherein said selection logic is configured to generate first and second resets for holding in a reset state one of said first and second non-volatile memory devices, respectively.

41. The storage controller as recited in claim 40, wherein said first and second non-volatile memory devices share a bus of said microprocessor, wherein said selection logic generates said second reset initially to enable said microprocessor to access said first non-volatile memory, and generates said first reset when said timer updates said selection logic if said microprocessor fails to boot said first stored program copy within said predetermined time to enable said microprocessor to access said second non-volatile memory.

42. The storage controller as recited in claim 39, wherein said first and second non-volatile memory devices comprise respective first and second FLASH memory devices.

43. The storage controller as recited in claim 38, wherein said first and second stored program copies are stored in respective first and second address regions of a non-volatile memory device.

44. The storage controller as recited in claim 43, wherein said selection logic is configured to generate one or more address signals to said non-volatile memory device.

45. The storage controller as recited in claim 44, wherein when said microprocessor accesses a predetermined address region within its address space, said selection logic generates said one or more address signals to enable said microprocessor to access said first region initially, and generates said one or more address signals to enable said microprocessor to access said second region when said timer updates said selection logic if said microprocessor fails to boot said first stored program copy within said predetermined time.

46. The storage controller as recited in claim 44, wherein said one or more address signals comprise a most-significant address signal of said non-volatile memory device.

47. The storage controller as recited in claim 43, wherein said non-volatile memory device comprises a FLASH memory device.

48. The storage controller as recited in claim 38, wherein said timer comprises a storage location for storing an indication of whether said microprocessor failed to boot said first stored program copy within said predetermined time, wherein initially said indication indicates said microprocessor has not failed to boot said first stored program copy within said predetermined time.

49. The storage controller as recited in claim 48, wherein said storage location retains its value after said timer resets said microprocessor.

50. The storage controller as recited in claim 38, wherein each of said first and second stored program copies includes instructions for disabling said timer when said microprocessor has successfully booted said respective stored program copy in less than said predetermined time.

51. The storage controller as recited in claim 38, wherein said timer comprises:

a status register, said status register comprising one or more bits for indicating whether said microprocessor failed to boot said first stored program copy within said predetermined time.

52. The storage controller as recited in claim 38, wherein said timer comprises:

a status register, said status register comprising one or more bits for indicating whether said microprocessor failed to boot said second stored program copy within said predetermined time.

53. The storage controller as recited in claim 38, wherein said timer comprises:

a status register, said status register comprising one or more bits for indicating whether said microprocessor failed more than once to boot said first stored program copy within said predetermined time.

54. The storage controller as recited in claim 38, wherein said timer comprises:

a status register, said status register comprising one or more bits for indicating whether said microprocessor failed more than once to boot said second stored program copy within said predetermined time.

55. The storage controller as recited in claim 38, wherein said timer comprises:

a count register for storing a value, said value specifying said predetermined time.

56. The storage controller as recited in claim 55 wherein in response to a reset of said timer, said count register is reset to a first predetermined said value.

57. The storage controller as recited in claim 56, wherein said value is further programmable by said stored program.

58. The storage controller as recited in claim 38, wherein said timer comprises:

a control register, said control register comprising one or more bits for specifying whether said timer is monitoring for a boot failure of a loader portion or an application portion of said stored program.

59. The storage controller as recited in claim 38, wherein said timer comprises:

a control register, said control register comprising one or more bits for specifying whether a loader portion of said stored program should attempt to boot a first or second copy of an application portion of said stored program.

60. The storage controller as recited in claim 38, further comprising:

a FLASH memory device, coupled to said microprocessor, for storing said first and second copies of said stored program;

wherein said timer comprises a control register, said control register comprising one or more bits for specifying whether accesses by said microprocessor to said FLASH memory device are to a loader or application portion of said stored program.

61. The storage controller as recited in claim 38, wherein said first and second stored program copies comprise different versions of said stored program that provide similar function.

62. A method for improving the probability of a microprocessor of a storage controller successfully booting a program stored therein, the method comprising:

starting a timer a first time in response to the microprocessor being reset a first time;

resetting the microprocessor a second time, after said starting the timer the first time, if the timer expired a first time before the microprocessor successfully boots a first copy of the stored program;

starting the timer a second time in response to said resetting the microprocessor the second time;

causing the microprocessor to attempt to boot a second copy of the stored program after said resetting the microprocessor the second time if the timer expired, the first time, wherein the storage controller comprises a redundant array of inexpensive disks (RAID) controller, wherein the program comprises a program for controlling a redundant array of inexpensive disks;

resetting the microprocessor a third time, after said starting the timer the second time, if the timer expired a second time before the microprocessor successfully boots the second copy of the stored program; and

causing the microprocessor to attempt to boot the first copy of the stored program after said resetting the microporcessor the third time.

63. The method as recited in claim 62, further comprising:

the first copy of the stored program disabling the timer prior to the timer expiring if the microprocessor successfully boots the first copy of the stored program.

64. The method as recited in claim 62, wherein the timer starts running independent of execution of the stored program by the microprocessor.

65. The method as recited in claim 62, wherein the timer is distinct from the microprocessor.

66. The method as recited in claim 62, wherein said causing the microprocessor to attempt to boot a second copy of the stored program comprises:

examining an indicator in a status register of the timer indicating the first copy failed to boot; and

selecting the second copy in response to said examining.

67. The method as recited in claim 66, wherein the first and second copies of the stored program are stored in respective first and second halves of a FLASH memory device of the controller, wherein said selecting the second copy comprises:

generating a predetermined value on an address bit to the FLASH memory device to select the second copy.

68. The method as recited in claim 67, further comprising:

generating an inverse of the predetermined value on the address bit to select the first copy for attempted booting, prior to said resetting the microprocessor.

69. The method as recited in claim 66, wherein the first and second copies of the stored program are stored in respective first and second FLASH memory devices of the controller sharing a bus to the microprocessor, wherein said selecting the second copy comprises:

generating a reset signal to the first FLASH memory device to select the second copy stored in the second FLASH memory device.

70. The method as recited in claim 69, further comprising:

generating a second reset signal to the second FLASH memory device to select the first copy stored in the first FLASH memory device for attempted booting, prior to said resetting the microprocessor the third time.

71. The method as recited in claim 62, wherein the first and second copies of the stored program comprise a loader program for loading an application program.

72. A method for improving the probability of a microprocessor of a storage controller successfully booting a program stored thereon, the method comprising:

attempting to boot a first copy of the stored program;

determining whether the first copy of the stored program failed to boot;

attempting to boot a second copy of the stored program, in response to determining the first copy of the stored program failed to boot, wherein the storage controller comprises a redundant array of inexpensive disks (RAID) controller, wherein the program comprises a program for controlling a redundant array of inexpensive disks;

determining whether the second copy of the stored program failed to boot; and

attempting to boot the first copy of hte stored program, in response to determining the second copy of the stored program failed to boot.

73. The method as recited in claim 72, further comprising:

determining the second copy failed to boot; and

attempting to boot the first copy, in response to said determining the second copy failed to boot.

74. The method as recited in claim 72, further comprising:

determining the second copy failed to boot; and

attempting to boot a third copy of the stored program, in response to said determining the second copy failed to boot.

75. The method as recited in claim 72, wherein said determining the first copy of the stored program failed to boot comprises detecting a timeout of a timer of the controller that commenced running substantially coincidentally with said attempting to boot the first copy of the stored program.

76. The method as recited in claim 72, wherein the first copy of the stored program comprises a first copy of a loader program and the second copy of the stored program comprises a second copy of the loader program, the method further comprising:

determining whether the second copy of the loader program successfully booted;

re-starting the timer, in response to determining the second copy of the loader program successfully booted; and

the second copy of the loader program attempting to boot a first copy of an application program of the stored program, after said re-starting the timer.

77. The method as recited in claim 76, further comprising:

determining whether the first-copy of the application program failed to boot;

re-booting the second copy of the loader program, in response to determining the first copy of the application program failed to boot; and

the second copy of the loader program attempting to boot a second copy of the application program, in response to said determining the first copy of the application program failed to boot.

78. The method as recited in claim 77 further comprising:

determining whether the second copy of the application program successfully booted; and

disabling the timer, in response to determining the second copy of the application program successfully booted.

79. The method as recited in claim 72, wherein the first copy of the stored program comprises a first copy of a loader program and the second copy of the stored program comprises a second copy of the loader program, the method further comprising:

re-starting the timer, in response to determining the first copy of the loader program successfully booted; and

the first copy of the loader program attempting to boot a first copy of an application program of the stored program, after said re-starting the timer.

80. The method as recited in claim 79, further comprising:

determining whether the first copy of the application program failed to boot;

re-booting the first copy of the loader program, in response to determining the first copy of the application program failed to boot; and

the first copy of the loader program attempting to boot a second copy of the application program, in response to said determining the first copy of the application program failed to boot.

81. The method as recited in claim 80, further comprising:

determining whether the second copy of the application program successfully booted; and

disabling the timer, in response to determining the second copy of the application program successfully booted.