Persistent and reliable delivery of event messages

• US 6,910,154 B1
• Filed: 08/18/2000
• Issued: 06/21/2005
• Est. Priority Date: 08/18/2000
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Schoenthal, Scott
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): LOHN, JOSHUA A
• Application Number:
  US09/642,064
• Time in Patent Office: 1,768 Days
• Field of Search: 714/18, 714/17, 714/16, 714/15, 714/4, 714/12, 714/48
• US Class Current: 714/18
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection

Abstract

The invention provides a method and system for persistent and reliable delivery of event messages. Those parts of the system responsible for delivering event messages are able to persistently maintain those event messages until the intended recipient of the event message confirms delivery of those event messages. Those parts of the system responsible for recovering from system crashes and other system errors are able to persistently maintain those event messages until delivery, even after recovery from system crashes or other system errors. The system includes a set of event message producers, and maintains an event-indication queue of those event messages provided by the event producers using a set of pre-allocated resources. An event-distribution engine distributes event messages to intended recipients and, after having received confirmation that the event messages were received, removes them from the event-indication queue. Recipients of event messages receive the event messages, acknowledge their receipt thereof, and might take action in response to the event message. The system includes persistent memory, initialization memory, and recipient persistent memories, and provides upon recovery from system crashes or other system error, an ability to replay event messages recorded in those memories, to re-present them as event messages. A cluster of file servers collectively forming a highly-available system shares persistent memories, so that upon a system crash or other system error, at least one other file server has a record of those event messages.

Claims

1. A method of a system handling an event message, comprising:

producing at least one event message;

storing said event message in a persistent memory in said system until said event message is handled; and

deleting said event message from said persistent memory after said event message is handled by said system;

wherein when said system is recovering from a system crash or error, said persistent memory is an initialization memory where said message is stored until said system recovers from said system crash or error.

2. A method as in claim 1, wherein said event message indicates an occurrence of a special event for said system.

3. A method as in claim 2, wherein said special event is said system crash or error.

4. A method as in claim 1, further comprising the step of handling said event message by delivering said event message to an intended recipient.

5. A method as in claim 4, wherein said event message is delivered to said intended recipient through a multiplexing recipient.

6. A method as in claim 5, wherein said multiplexing recipient further performs the steps of:

storing said event message in said multiplexing recipient'"'"'s persistent memory until said event message is handled by said multiplexing recipient; and

deleting said event message from said multiplexing recipient'"'"'s persistent memory after said event message is handled by said multiplexing recipient.

7. A method as in claim 4, wherein said intended recipient sends a confirmation to said system after delivery of said event message to said intended recipient.

8. A method as in claim 1, wherein storing said event message in said persistent memory further comprises storing said event message in an event indication queue, said event indication queue having resources pre-allocated before occurrence of an event associated with said event message.

9. A method as in claim 8, wherein said event indication queue is reliable even when said event message indicates that allocation of new resources is unstable.

10. A system comprising:

an event message producer;

a persistent memory in which at least one event message from said event message producer is stored until said system handles said event message; and

an event message distributor that handles said event message;

wherein said event message is deleted from said persistent memory after said event message is handled by said system; and

wherein when said system is recovering from a system crash or error, said persistent memory is an initialization memory where said message is stored until said system recovers from said system crash or error.

11. A system as in claim 10, wherein said event message indicates an occurrence of a special event for said system.

12. A system as in claim 11, wherein said special event is said system crash or error.

13. A system as in claim 10, wherein said event message distributor delivers said event message to an intended recipient.

14. A system as in claim 13, wherein said event message is delivered to said intended recipient through a multiplexing recipient.

15. A system as in claim 14, wherein said multiplexing recipient performs the steps of:

storing said event message in said multiplexing recipient'"'"'s persistent memory until said event message is handled by said multiplexing recipient; and

deleting said event message from said multiplexing recipient'"'"'s persistent memory after said event message is handled by said multiplexing recipient.

16. A system as in claim 13, wherein said intended recipient sends a confirmation to said system after delivery of said event message to said intended recipient.

17. A system as in claim 10, wherein said persistent memory further comprises an event indication queue, said event indication queue having resources pre-allocated before occurrence of an event associated with said event message.

18. A system as in claim 17, wherein said event indication queue is reliable even when said event message indicates that allocation of new resources is unstable.

19. A memory storing instructions executable by a system, said instructions comprising:

producing at least one event message;

storing said event message in a persistent memory in said system until said event message is handled; and

deleting said event message from said persistent memory after said event message is handled by said system;

wherein when said system is recovering from a system crash or error, said persistent memory is an initialization memory where said message is stored until said system recovers from said system crash or error.

20. A memory as in claim 19, wherein said event message indicates an occurrence of a special event for said system.

21. A memory as in claim 20, wherein said special event is said system crash or error.

22. A memory as in claim 19, wherein said instructions further capable of being interpreted to indicate handling said event message by delivering said event message to an intended recipient.

23. A memory as in claim 22, wherein said event message is delivered to said intended recipient through a multiplexing recipient.

24. A memory as in claim 23, wherein said multiplexing recipient further performs the steps of:

storing said event message in said multiplexing recipient'"'"'s persistent memory until said event message is handled by said multiplexing recipient; and

deleting said event message from said multiplexing recipient'"'"'s persistent memory after said event message is handled by said multiplexing recipient.

25. A memory as in claim 22, wherein said intended recipient sends a confirmation to said system after delivery of said event message to said intended recipient.

26. A memory as in claim 19, wherein storing said event message in said persistent memory further comprises storing said event message in an event indication queue, said event indication queue having resources pre-allocated before occurrence of an event associated with said event message.

27. A memory as in claims 26, wherein said event indication queue is sage indicates that allocation of new resources is unstable.