Stream-Oriented Database Machine and Method

• US 20090172014A1
• Filed: 08/18/2006
• Published: 07/02/2009
• Est. Priority Date: 08/23/2005
• Status: Abandoned Application

• Current Assignee: Position Networks Pty., Ltd.
• Original Assignee: Position Networks Pty., Ltd.
• Inventors: Huetter, Raymond John
• Application Number:
  US12/064,505
• Publication Number:
  US 20090172014A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2477 Temporal data queries

Abstract

An event stream processing device capable of processing larger numbers of events while simultaneously responding to queries. This is achieved through sequential storage of data, the maintenance in memory of information pertaining to the most recent events for each entity monitored and the aggregation of file read/write requests in a single thread which is capable of optimising the execution of those requests.

Claims

1. A high data throughput special purpose device;

said device comprising at least one processor in communication with an IO system, a memory and persistent storage in the form of at least one disk;

said device adapted to receive a substantially continuous stream of status data pertaining to the current state of a finite number of objects via said IO system;

said device keeping said current state of said finite number of said objects in memory while writing and reading an indefinite amount of indexed history sequentially stored on said at least one disk;

thereby to construct on said at least one disk a sequenced, time-ordered history of said status data extending back to a predetermined point in time.

2. The device of claim 1 wherein said device is adapted for keeping said current state of said finite number of said objects in memory while simultaneously writing and reading an indefinite amount of indexed history sequentially stored on said at least one disk.

3. The device of claim 1 wherein said device is a hybrid of memory-oriented and disk-oriented database systems.

4. The device of claim 1 wherein said status data includes at least a first parameter and a second parameter for each said object;

said first parameter comprising time data.

5. The device of claim 4 wherein said second parameter is location data pertaining to the location of said object at a given point in time.

6. The device of claim 1 comprising one or more central processing units (CPU'"'"'s), memory comprising one or more memory units, one or more persistent storage units, one or more communication sockets, and a clock.

7. The device of claim 1 programmatically arranged as an interconnected set of multi-threaded processing units (here within referred to as agents) executing a set of event processing, query processing, disk I/O, network I/O and housekeeping tasks.

8. The device claim 1 wherein said device is adapted for accepting one or more events streams comprising event data about events pertaining to objects.

9. The device of claim 8 wherein said device is adapted for grouping predetermined amounts of event data into tasks which represent work to be done.

10. The device of claim 9 wherein said device is adapted for keeping the current location and state of the objects in said memory, in concurrent data structures, said data structures indexed by at least the identity and location of respective said objects.

11. The device of claim 10 wherein said device is adapted for processing said tasks, thereby changing the location and state of said objects held in said memory.

12. The device of claim 11 wherein said device is adapted for writing a stream of time-ordered records of changes to said location and state data of said objects onto said persistent storage in a sequential manner, indexed by at least time, object identity and location, where said index is also written concurrently and sequentially with said records.

13. The device of claim 12 wherein said device is adapted for executing query tasks by retrieving relevant said location and state data about said objects from said memory or said persistent storage.

14. The device of claim 13 wherein said device is adapted for locating and retrieving said objects in said memory by either said identity or said location.

15. The device of claim 14 wherein said device is adapted for locating and retrieving said records in persistent storage by either said identity or said location or by time.

16. The device of claim 1 wherein said device is set to have a finite number of steps and an upper time-space processing limit to each step thereby to facilitate real time processing.

17. A device according to claim 4, wherein said status data is stored as a record, one for each said object for a unique value of said first parameter and wherein the fully processed records are collected in groups and each group given a sequence number to be recorded with it.

18. A method of processing and storing a substantially continuous stream of status data pertaining to the state of a finite number of objects;

said method comprising maintaining said current state of said finite number of said objects in memory while sequentially writing and reading an indefinite amount of indexed history of said status data to at least one disk;

thereby to provide current status of said objects from memory and history of said status data from said disk.

19. The method of claim 18;

said method comprising maintaining said current state of said finite number of said objects in memory while simultaneously sequentially writing and reading an indefinite amount of indexed history of said status data to at least one disk.

20-52. -52. (canceled)

53. A machine:

Comprising one or more central processing units (CPU'"'"'s), memory comprising one or more memory units, one or more persistent storage units, one or more communication sockets, and a clock;

Programmatically arranged as an interconnected set of multi-threaded processing units (here within referred to as agents) executing a set of event processing, query processing, disk I/O, network I/O and housekeeping tasks;

Accepting one or more events streams comprising event data about events pertaining to objects;

Grouping predetermined amounts of event data into tasks which represent work to be done;

Keeping the current location and state of the objects in said memory, in concurrent data structures, said data structures indexed by at least the identity and location of respective said objects;

Processing said tasks, thereby changing the location and state of said objects held in said memory;

Writing a stream of time-ordered records of changes to said location and state data of said objects onto said persistent storage in a sequential manner, indexed by at least time, object identity and location, where said index is also written concurrently and sequentially with said records;

Executing query tasks by retrieving relevant said location and state data about said objects from said memory or said persistent storage;

Locating and retrieving said objects in said memory by either said identity or said location; and

locating and retrieving said records in persistent storage by either said identity or said location or by time.