Logging incident manager
First Claim
1. A computer-implemented method associated with processing log messages from a plurality of handlers that are distributed among a plurality of hosts in an application cluster, the method comprising:
- by one or more handlers of the plurality of handlers;
(i) collecting a first set of log messages that include a coarse granularity level of detail regarding events that occur on one or more of the plurality of hosts, and storing the first set of log messages in a first buffer;
(ii) collecting a second set of log messages that include a fine granularity level of detail regarding the events, and storing the second set of log messages in a second buffer;
wherein the second buffer is configured as a circular buffer that includes a set number of storage slots and recycles the storage slots when full;
wherein storing in the circular buffer includes overwriting an oldest log message in the circular buffer with a new log message when the circular buffer is full to recycle the storage slots;
periodically transmitting to a logging appliance that includes at least a processor, the first set of log messages collected at the coarse granularity level from one or more of the plurality of handlers;
analyzing the first set of log messages having the coarse granularity level to determine whether an error has been encountered;
in response to detecting the error in the first set of log messages;
freezing the circular buffers in the plurality of handlers to preserve the second set of log messages stored therein and to stop storing new log messages in the circular buffers;
selecting a subset of the plurality of handlers to provide reports that include the second set of log messages collected at the fine granularity level from the circular buffers; and
scheduling the subset of the plurality of handlers to separately transmit the reports to the logging appliance;
in response to receiving the reports, combining the second set of log messages having the fine granularity level from the reports into a formal log.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems, methods, and other embodiments associated with intelligently processing log messages are described. In one embodiment, a computer-implemented method includes analyzing, by a logging appliance that includes at least hardware, communications received from a plurality of handlers to determine whether at least one of the communications indicates an error has been encountered by one of a plurality of components associated with the plurality of handlers. The method includes scheduling a subset of the plurality of handlers to provide reports that include a detailed set of log messages in response to detecting the error. The subset of the plurality of handlers includes handlers that are associated with the error.
-
Citations
26 Claims
-
1. A computer-implemented method associated with processing log messages from a plurality of handlers that are distributed among a plurality of hosts in an application cluster, the method comprising:
-
by one or more handlers of the plurality of handlers; (i) collecting a first set of log messages that include a coarse granularity level of detail regarding events that occur on one or more of the plurality of hosts, and storing the first set of log messages in a first buffer; (ii) collecting a second set of log messages that include a fine granularity level of detail regarding the events, and storing the second set of log messages in a second buffer; wherein the second buffer is configured as a circular buffer that includes a set number of storage slots and recycles the storage slots when full; wherein storing in the circular buffer includes overwriting an oldest log message in the circular buffer with a new log message when the circular buffer is full to recycle the storage slots; periodically transmitting to a logging appliance that includes at least a processor, the first set of log messages collected at the coarse granularity level from one or more of the plurality of handlers; analyzing the first set of log messages having the coarse granularity level to determine whether an error has been encountered; in response to detecting the error in the first set of log messages; freezing the circular buffers in the plurality of handlers to preserve the second set of log messages stored therein and to stop storing new log messages in the circular buffers; selecting a subset of the plurality of handlers to provide reports that include the second set of log messages collected at the fine granularity level from the circular buffers; and scheduling the subset of the plurality of handlers to separately transmit the reports to the logging appliance; in response to receiving the reports, combining the second set of log messages having the fine granularity level from the reports into a formal log. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computing system for processing log messages the contain data on events that occur on one or more of a plurality of host devices that execute a plurality of applications in an application cluster, comprising:
-
a plurality of handlers where each handler includes a first buffer and a second buffer, wherein the second buffer is configured as a circular buffer that includes a set number of storage slots and recycles the storage slots when the circular buffer is full; wherein each handler is configured to operate with one or more processors to; (i) collect a first set of log messages that include a coarse granularity level of detail regarding events that occur on one or more of the plurality of hosts, and store the first set of log messages in the first buffer; (ii) collect a second set of log messages that include a fine granularity level of detail regarding the events, and store the second set of log messages in the second buffer; wherein storing in the circular buffer includes overwriting an oldest log message in the circular buffer with a new log message when the circular buffer is full; wherein the plurality of handlers are configured to periodically transmit, to a logging appliance that includes at least a processor, the first set of log messages collected at the coarse granularity level from one or more of the plurality of handlers; incident manager logic stored in a non-transitory computer-readable medium as part of the logging appliance and including instructions that when executed by one or more processors of the logging appliance cause the one or more processors to analyze the first set of log messages having the coarse granularity level to determine whether at least one of the log messages indicates an error has been encountered; and wherein the incident management logic includes instructions that when executed by the one or more processors cause the one or more processors to, in response to detecting the error in the first set of log messages; (i) select a subset of the plurality of handlers to provide reports that include the second set of log messages collected at the fine granularity level from the circular buffers; (ii) freeze the circular buffers in the subset of the plurality of handlers to preserve the second set of log messages stored therein and to stop storing new log messages in the circular buffers; (iii) schedule the subset of the plurality of handlers to separately transmit the reports to the logging appliance; and (iv) in response to receiving the reports, combine the second set of log messages having the fine granularity level from the reports into a formal log. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A non-transitory computer-storage medium storing instructions that when executed, by one or more processors of a plurality of handlers that are computing devices for processing log messages from a plurality of hosts that are part of an application cluster, cause the one or more processors to:
-
collect a first set of log messages that include a coarse granularity level of detail regarding events that occur on one or more of the plurality of hosts, and store the first set of log messages in a first buffer; collect a second set of log messages that include a fine granularity level of detail regarding the events, and store the second set of log messages in a second buffer; wherein the second buffer is configured as a circular buffer that stores a set number of log messages and recycles storage slots by overwriting an oldest log message in the second buffer with a new log message when the second buffer is full; periodically transmit, to a logging appliance that includes at least a processor, the first set of log messages collected at the coarse granularity level from one or more of the plurality of handlers; analyze, by a processor of the logging appliance, the first set of log messages having the coarse granularity level to determine whether an error has been encountered; in response to detecting the error in the first set of log messages; (i) freeze the circular buffers in the plurality of handlers to preserve the second set of log messages stored therein and to stop storing new log messages in the circular buffers; (ii) select a subset of the plurality of handlers to provide reports that include the second set of log messages collected at the fine granularity level from the circular buffers; (iii) schedule the subset of the plurality of handlers to separately transmit the reports to the logging appliance; and (iv) in response to receiving the reports, combine the second set of log messages having the fine granularity level from the reports into a formal log. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26)
-
Specification