High availability event log collection in a networked system

• US 9,131,015 B2
• Filed: 10/08/2012
• Issued: 09/08/2015
• Est. Priority Date: 10/08/2012
• Status: Expired due to Fees

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Rittle, Loren J., Kuhlman, Douglas A., Singer, Gregory L.
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US13/646,798
• Publication Number:
  US 20140101110A1
• Time in Patent Office: 1,065 Days
• Field of Search: 707/654, 707/E17.005
• US Class Current: 1/1
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A method includes storing data received at a first server to a first file and storing data received at a second server to a second file, where the data in the two files are substantially similar. The two files are distinct, independent files. The first server provides a copy of the first file to the second server to store. The first server then stores additional data to the first file, while the second server maintains the copy of the first file unchanged. The second server may also provide a copy of the second file to the first server to store and continues to store additional data to the second file, while maintaining the copy of the second file on the first server unchanged. Event reports may then be generated using the various files, which provide high availability due to the above method and corresponding system arrangement.

Claims

1. A method comprising:

when a primary data collection server is accessible;

receiving, at the primary data collection server residing upon an associated physical hardware node, collected data via a data collection virtual Internet protocol address; and

transmitting a collected data copy of the collected data received at the primary data collection server from the primary data collection server to a backup data collection server through a data synchronization network connection operatively coupling the primary data collection server and the backup data collection server, the backup data collection server residing upon an associated physical hardware node and configured as a hot standby of the primary data collection server;

when the primary data collection server is inaccessible, receiving, at the backup data collection server, the collected data via the data collection virtual Internet protocol address;

generating a first file and a second file as two distinct, independent files, the first file being on a first server, the second file being on a second server, the first server and the second server operatively coupled to each of the primary data collection server and the backup data collection server;

storing, at the first sever, data associated with the collected data to the first file, the data stored to the first file being received at the first server from at least one of the primary data collection server or the backup data collection server;

storing, at the second server, data associated with the collected data to the second file, the data stored to the second file being received at the second server from at least one of the primary data collection server or the backup data collection server, the data stored to the second file being substantially similar to the data stored to the first file;

providing a first file copy of the first file to the second server by the first server and storing the first file copy of the first file by the second server; and

storing, at the first server, additional data to the first file, the additional data stored to the first file being received at the first server while maintaining the first file copy of the first file on the second server unchanged.

2. The method of claim 1 further comprising:

providing a second file copy of the second file to the first server by the second server and storing the second file copy of the second file by the first server; and

storing, at the second server, additional data to the second file, the additional data stored to the second file being received at the second server while maintaining the second file copy of the second file on the first server unchanged, the additional data received at the second server being substantially similar to the additional data received at the first server.

3. The method of claim 1 further comprising:

providing, by the first server, the first file and the second file copy of the second file to an analysis application on at least one analysis server;

comparing, by the analysis application, the first file with the second file copy of the second file and removing duplicate data; and

generating, by the analysis application, a first event report using the first file and the second file copy of the second file.

4. The method of claim 3 further comprising:

providing, by the second server, the second file and the first file copy of the first file to the analysis application on the at least one analysis server;

comparing, by the analysis application, the second file with the first file copy of the first file and removing duplicate data;

generating, by the analysis application, a second event report using the second file and the first file copy of the first file; and

comparing, by the analysis application, the first event report with the second event report to generate a third event report.

5. The method of claim 1 further comprising:

when the first server goes offline, receiving, at the second server, new additional data sent to the first server but not received by the first server;

storing, by the second server, the new additional data received by the second server to the second file; and

maintaining the first file copy of the first file on the second server unchanged.

6. The method of claim 5 further comprising:

when the first server goes back online, providing a new second file copy of the second file to the first server by the second server and storing the new second file copy of the second file by the first server.

7. The method of claim 1 wherein providing a first file copy of the first file to the second server by the first server comprises:

performing a server push from the first server to the second server.

8. The method of claim 1 wherein providing a first file copy of the first file to the second server by the first server comprises:

performing a client pull by the second server from the first server.

9. The method of claim 1, wherein the primary data collection server or the backup data collection server receives the collected data from one or more application servers via the data collection virtual internet protocol address, and the one or more application servers are associated with event recording.

10. The method of claim 9, wherein the one or more application servers comprise networked digital video recording servers.

11. An event log collection and reporting system comprising:

physical hardware nodes;

a primary data collection server configured to receive collected data via a data collection virtual Internet protocol address when the primary data collection server is accessible, the primary data collection server residing upon an associated physical hardware node;

a backup data collection server operatively coupled to the primary data collection server via a data synchronization network connection configured to provide a copy of the collected data received by the primary data collection server to the backup data collection server, the backup data collection server residing upon an associated physical hardware node and configured as a hot standby of the primary data collection server to receive the collected data via the data collection virtual Internet protocol address when the primary data collection server is inaccessible;

a first reporting server operatively coupled to a second reporting server, the first reporting server and the second reporting server operative to form a client-server connection there-between, the second reporting server configured as a hot standby of the first reporting server;

the first reporting server operatively coupled to the primary data collection server and the backup data collection server, the first reporting server operative to;

receive data associated with the collected data from at least one of—

the primary data collection server or the backup data collection server and store the data to a first file;

provide a first file copy of the first file to the second reporting server; and

store additional data to the first file, the additional data stored to the first file being received at the first reporting server, while the second reporting server maintains the first file copy of the first file on the second reporting server unchanged;

the second reporting server operatively coupled to the primary data collection server and the backup data collection server, the second reporting server operative to;

store the first file copy of the first file received from the first reporting server; and

receive the data associated with the collected data from at least one of the primary or backup data collection servers and store the data to a second file while maintaining the first file copy of the first file unchanged, the second file being distinct and independent from the first file and from the first file copy of the first file.

12. The event log collection and reporting system of claim 11 wherein the primary data collection server and the backup data collection server each include:

a message broker to facilitate master/slave operation between the primary data collection server and the backup data collection server.

13. The event log collection and reporting system of claim 12 wherein the primary data collection server and the backup data collection server each further include:

a routing software instance configured to provide the data collection virtual Internet protocol address.

14. The event log collection and reporting system of claim 13 wherein the first and second reporting servers are both operative to implement a secure communication protocol on the client-server connection there-between.

15. The event log collection and reporting system of claim 13 wherein the first and second reporting servers each include:

a routing software instance configured to provide a reporting virtual Internet protocol address for routing reports to at least one analysis server.

16. The event log collection and reporting system of claim 11, wherein the primary data collection server or the backup data collection server receives the collected data from one or more application servers via the data collection virtual internet protocol address, and the one or more application servers are associated with event recording.

17. The event log collection and reporting system of claim 16, wherein the one or more application servers comprise networked digital video recording servers.

18. A method comprising:

sending event data to a data collection virtual Internet Protocol (VIP) address by at least one application server and at least one recorder cluster manager;

when a first data collection server is accessible;

routing the event data over the data collection VIP address to the first data collection server residing upon an associated physical hardware node;

transmitting an event data copy of the event data received at the first data collection server to a second data collection server operatively coupled to the first data collection server, the second data collection server residing upon an associated physical hardware node and configured as a hot standby of the first data collection server;

when the first data collection server is inaccessible, routing the event data over the data collection VIP address to the second data collection server;

sending the event data from the first data collection server or from the second data collection server to a primary reporting server and to a backup reporting server, each reporting server operatively coupled to the first data collection server and the second data collection server;

creating a first file by the primary reporting server and providing a first file copy of the first file to the backup reporting server via a client-server connection;

creating a second file by the backup reporting server, the second file being substantially similar to the first file, and providing a second file copy of the second file to the primary reporting server via the client-server connection;

sending the first file and the second file copy of the second file to a reporting VIP address by the primary reporting server;

sending the second file and the first file copy of the first file to the reporting VIP address by the backup reporting server; and

generating at least one event report by at least one analysis server using the first file, the second file, the first file copy of the first file, and the second file copy of the second file.

19. The method of claim 18, wherein the at least one application server comprises a networked digital video recording server.