System having in-memory buffer service, temporary events file storage system and events file uploader service
First Claim
1. A system, comprising:
- a data store that is configured to store events;
an application server configured to run a client application that generates the events, wherein the application server comprises;
an in-memory buffer service;
a capture service configured to;
receive events generated by the client application running on the application server and to provide the events to the in-memory buffer service, wherein the in-memory buffer service is configured to temporarily hold the events in a bounded buffer that is implemented in physical memory at the application server, wherein the bounded buffer comprises a plurality of single-threaded buffer segments that are each configured to temporarily hold some of the events received from the capture service until a flush operation is performed, wherein each particular single-threaded buffer segment holds a plurality of events;
a consumer executor service configured to;
instantiate a plurality of consumer threads that each correspond to a particular single-threaded buffer segment, wherein each consumer thread comprises;
an instance of an events reader that is configured to read events flushed from a corresponding single-threaded buffer segment and to add an identifier to each the events; and
an instance of a direct events writer that is configured to attempt to directly write each event to the data store;
a temporary events file storage system (TEFSS);
wherein the application server comprises;
an indirect events writer that is invoked when the events are unable to be directly written to the data store, comprising;
a plurality of event capture threads each being configured to;
generate a particular events file that includes a plurality of events flushed from an in-memory buffer service, and write that particular events file to the TEFSS; and
an events file uploader service that is configured to read at least one events file from the TEFSS, and then write the events from each of the retrieved events files to the data store.
1 Assignment
0 Petitions
Accused Products
Abstract
Computer-implemented methods and systems are provided for writing events to a data store. An application server generates events, the data store that stores the events, and a temporary events file storage system (TEFSS) temporarily stores groups of events as events files. When events are unable to be written directly to the data store, an indirect events writer is invoked that includes event capture threads each being configured to generate a particular events file, and write it to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service. An events file uploader service reads the events file(s) from the TEFSS, and then writes the events from each of the events files to the data store.
-
Citations
28 Claims
-
1. A system, comprising:
-
a data store that is configured to store events; an application server configured to run a client application that generates the events, wherein the application server comprises; an in-memory buffer service; a capture service configured to;
receive events generated by the client application running on the application server and to provide the events to the in-memory buffer service, wherein the in-memory buffer service is configured to temporarily hold the events in a bounded buffer that is implemented in physical memory at the application server, wherein the bounded buffer comprises a plurality of single-threaded buffer segments that are each configured to temporarily hold some of the events received from the capture service until a flush operation is performed, wherein each particular single-threaded buffer segment holds a plurality of events;a consumer executor service configured to;
instantiate a plurality of consumer threads that each correspond to a particular single-threaded buffer segment, wherein each consumer thread comprises;
an instance of an events reader that is configured to read events flushed from a corresponding single-threaded buffer segment and to add an identifier to each the events; and
an instance of a direct events writer that is configured to attempt to directly write each event to the data store;a temporary events file storage system (TEFSS); wherein the application server comprises; an indirect events writer that is invoked when the events are unable to be directly written to the data store, comprising;
a plurality of event capture threads each being configured to;
generate a particular events file that includes a plurality of events flushed from an in-memory buffer service, and write that particular events file to the TEFSS; andan events file uploader service that is configured to read at least one events file from the TEFSS, and then write the events from each of the retrieved events files to the data store. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A non-transitory computer-readable medium including instructions, which when executed by a processor at an application server, are configurable to perform a method comprising:
-
receiving, at a capture service, events from a client application running on the application server, wherein the events are to be stored in a data store; storing, via the capture service, the events in an in-memory buffer service comprising;
a bounded buffer that is implemented in physical memory at the application server, wherein the bounded buffer comprises a plurality of single-threaded buffer segments that are each configured to temporarily store some of the events until a flush operation is performed;flushing the plurality of single-threaded buffer segments of the bounded buffer, wherein each particular single-threaded buffer segment includes the plurality of events; instantiating, via a consumer executor service, a plurality of consumer threads that each corresponds to a particular single-threaded buffer segment, wherein each consumer thread comprises;
an instance of an events reader and an instance of a direct events writer;reading, at the events reader of each consumer thread, the events flushed from the particular single-threaded buffer segment that corresponds to that consumer thread; adding, at each direct events writer, an identifier to each the events; at each direct events writer, attempting to directly write each event to the data store; generating at least one events file that comprises the plurality of events received from the in-memory buffer service when those events are unable to be directly written to the data store; writing each events file to a temporary events file storage system (TEFSS) that is configured to temporarily store each events file; reading each events file from the TEFSS; and writing the plurality of events from each events file that was read to the data store. - View Dependent Claims (23, 24, 25, 26, 27)
-
-
28. A computer-implemented method performed by an application server comprising an events file uploader service and an indirect events writer that is configured to instantiate a plurality of event capture threads each comprising an instance of an events file writer, the computer-implemented method comprising:
-
at a capture service;
receiving events generated by a client application running on the application server and to providing the events to an in-memory buffer service;at an in-memory buffer service;
temporarily holding the events in a bounded buffer that is implemented in physical memory at the application server, wherein the bounded buffer comprises a plurality of single-threaded buffer segments that are each configured to temporarily hold some of the events received from the capture service until a flush operation is performed, wherein each particular single-threaded buffer segment holds a plurality of events;at a consumer executor service; instantiating a plurality of consumer threads that each correspond to a particular single-threaded buffer segment, wherein each consumer thread comprises;
an instance of an events reader that is configured to read events flushed from a corresponding single-threaded buffer segment and to add an identifier to each the events; and
an instance of a direct events writer that is configured to attempt to directly write each event to the data store;invoking, when the events are unable to be directly written to the data store, an indirect events writer comprising;
a plurality of event capture threads;at an events file writer of each event capture thread; generating a particular events file that comprises the plurality of events received from the in-memory buffer service when those events are unable to be directly written to an event table at the data store; writing the particular events file to a temporary events file storage system (TEFSS) that is configured to temporarily store one or more events files for subsequent writing to the data store; invoking an events file uploader service that is configured to read at least one events file from the TEFSS, and then write the plurality of events from each of the retrieved events files to the event table maintained at the data store.
-
Specification