Method and system for collecting and retrieving time-series, real-time and non-real-time data

• US 6,651,030 B2
• Filed: 06/21/2002
• Issued: 11/18/2003
• Est. Priority Date: 06/22/2001
• Status: Expired due to Term

• Current Assignee: Schneider Electric Software, LLC (Schneider Electric SE)
• Original Assignee: Invensys Systems Incorporated (Invensys PLC)
• Inventors: Avergun, Mikhail, Victor, Hendrik Johannes, Knox-Davies, Llewellyn John
• Primary Examiner(s): SHAH, KAMINI S
• Application Number:
  US10/178,002
• Publication Number:
  US 20020198679A1
• Time in Patent Office: 515 Days
• Field of Search: 702/172, 702/176, 709/224, 709/201, 709/216, 707/500.1
• US Class Current: 702/172
• CPC Class Codes: G05B 2219/31462 Add time stamp to alarm message
• CPC Class Codes: G05B 23/0264 Control of logging system, e.g. decision on which data to store; time-stamping measurements

Abstract

Disclosed is a data collection and retrieval system that puts data produced by real-time and non-real-time data sources into parallel “streams” or data files. Real-time data points are stored with time stamps in data files and non-real-time data points with time stamps are stored in other data files. Real-time data points are stamped with the times at which their data were collected. Non-real-time data points are stored in time-sequential order, the sequence time being the time of relevance rather than the time of collection. By referencing the streams that contain data points relevant to a given period, header files facilitate coordinated retrieval of data from multiple parallel streams. The time stamps allow the data points to be merged together into one coordinated, time-sequential data stream for analysis. The parallel stream data collection and retrieval technique allows operators to add or delete data sources without shutting down the process being monitored.

Claims

1. In an environment with a real-time data source and with a non-real-time data source, a method for storing, in a time-coordinated fashion, data received from the real-time and non-real-time data sources, the method comprising:

receiving real-time data from the real-time data source;

storing the real-time data as entries in a real-time, streaming data file, the entries time-stamped with times associated with when the real-time data were produced;

receiving non-real-time data from the non-real-time data source;

storing the non-real-time data as entries in a non-real-time data file, the entries time-stamped with times of relevance of the non-real-time data; and

writing a header file that associates a time interval with those entries in the real-time and non-real-time data files whose time stamps fall within the time interval.

2. The method of claim 1 wherein storing data as entries comprises storing the entries in a data file in order of increasing time stamps.

3. The method of claim 1 wherein storing data as entries comprises storing in a data file only those entries whose time stamps fall within the time interval.

4. The method of claim 3 wherein associating the time interval with data entries in a data file comprises associating the time interval with the data file.

5. The method of claim 1 wherein the non-real-time data file comprises a non-real-time database and wherein the non-real-time data entries are keyed by their time stamps.

6. The method of claim 5 wherein the header file comprises a header database and wherein the header and the non-real-time databases are the same database.

7. The method of claim 1 wherein data entries are marked as associated with a data source.

8. The method of claim 7 wherein marking data entries as associated with a data source comprises storing only data received from the data source in a data file and associating the data file with the data source.

9. The method of claim 1 wherein time-stamping the non-real-time data entries with times of relevance comprises stamping a time at which input data were collected, wherein the collected input data were sent as inputs to a process, and wherein the non-real-time data received from the non-real-time data source are outputs of the process.

10. The method of claim 1 wherein writing a header file comprises associating the header file with the real-time and non-real-time data sources.

11. The method of claim 10 further comprising:

associating the header file with another data source, the other data source distinct from the real-time and non-real-time data sources;

wherein associating the header file with the other data source triggers receiving data from the other data source.

12. The method of claim 1 further comprising:

receiving data from another data source, the other data source distinct from the real-time and non-real-time data sources; and

storing the data received from the other data source as entries in a data file selected from the group consisting of;

the real-time data file, the non-real-time data file, and another data file.

13. The method of claim 1 further comprising:

storing at least one time-stamped snapshot entry in the real-time data file;

and wherein time-stamping a real-time data entry comprises stamping a period of time between the time stamp of a snapshot entry and a time when the real-time data of the entry were produced.

14. The method of claim 1 further comprising:

storing at least one time-stamped snapshot entry in the non-real-time data file;

and wherein time-stamping a non-real-time data entry comprises stamping a period of time between the time stamp of a snapshot entry and a time of relevance of the non-real-time data.

34. The method of claim 1 wherein at least some of the time-stamped real-time and non-real-time data entries measure aspects of an industrial process.

15. A computer-readable medium containing computer-executable instructions for performing, in an environment with a real-time data source and with a non-real-time data source, a method for storing, in a time-coordinated fashion, data received from the real-time and non-real-time data sources, the method comprising:

receiving real-time data from the real-time data source;

storing the real-time data as entries in a real-time, streaming data file, the entries time-stamped with times associated with when the real-time data were produced;

receiving non-real-time data from the non-real-time data source;

storing the non-real-time data as entries in a non-real-time data file, the entries time-stamped with times of relevance of the non-real-time data; and

writing a header file that associates the time interval with those entries in the real-time and non-real-time data files whose time stamps fall within the time interval.

16. A method for retrieving, in a time-coordinated fashion, data entries from a real-time, streaming data file and from a non-real-time data file, the method comprising:

selecting a time interval of interest;

accessing a header file that associates the time interval of interest with time-stamped data entries in the real-time and non-real-time data files whose time stamps fall within the time interval of interest;

retrieving from the real-time data file data entries associated by the header file with the time interval of interest;

retrieving from the non-real-time data file data entries associated by the header file with the time interval of interest; and

coordinating the retrieval of data entries from the real-time and non-real-time data files based on the time stamps of the data entries.

17. The method of claim 16 wherein all data entries in a data file have time stamps falling within the time interval of interest and wherein the header file associates the time interval of interest with the data file.

18. The method of claim 16 wherein the non-real-time data file comprises a non-real-time database and wherein the non-real-time data entries are keyed by their time stamps.

19. The method of claim 18 wherein the header file comprises a header database and wherein the header and the non-real-time databases are the same database.

20. The method of claim 16 wherein the header file further associates the time interval of interest with time-stamped data entries in another data file whose time stamps fall within the time interval of interest, the other data file distinct from the real-time and non-real-time data files;

the method further comprising;

retrieving from the other data file data entries associated by the header file with the time interval of interest; and

coordinating the retrieval of data entries from the other data file based on the time stamps of the data entries.

21. The method of claim 20 wherein at least one entry in the other data file contains a modification to an entry in the real-time data file and wherein retrieving an entry from the real-time data file comprises combining a data value stored with the real-time data entry with a data value stored with the entry in the other data file.

22. The method of claim 16 wherein a data file comprises at least one time-stamped snapshot entry.

23. The method of claim 16 wherein the header file is associated with a real-time data source and with a non-real-time data source;

wherein at least some of the real-time data entries in the real-time data file are based on data from the real-time data source;

wherein at least some of the non-real-time data entries in the non-real-time data file are based on data from the non-real-time data source;

the method further comprising;

selecting one or more data sources; and

retrieving only data entries based on data from the selected data sources.

29. The data structure of claim 23 further comprising a fourth data field containing data representing a time-stamped entry in another data file whose time stamp falls within the time interval, the other data file distinct from the real-time and non-real-time data files.

30. The data structure of claim 29 wherein the entry represented by data in the fourth data field contains a modification to the entry represented by data in the second data field.

35. The method of claim 16 wherein at least some of the time-stamped real-time and non-real-time data entries measure aspects of an industrial process.

24. A computer-readable medium containing computer-executable instructions for performing a method for retrieving, in a time-coordinated fashion, data entries from a real-time, streaming data file and from a non-real-time data file, the method comprising:

selecting a time interval of interest;

accessing a header file that associates the time interval of interest with time-stamped data entries in the real-time and non-real-time data files whose time stamps fall within the time interval of interest;

retrieving from the real-time data file data entries associated by the header file with the time interval of interest;

retrieving from the non-real-time data file data entries associated by the header file with the time interval of interest; and

coordinating the retrieval of data entries from the real-time and non-real-time data files based on the time stamps of the data entries.

25. A computer-readable medium having stored thereon a data structure, the data structure comprising:

a first data field containing data representing a time interval;

a second data field containing data representing a time-stamped entry in a real-time data file whose time stamp falls within the time interval; and

a third data field containing data representing a time-stamped entry in a non-real-time data file whose time stamp falls within the time interval.

26. The data structure of claim 25 wherein the data in the second data field represents a time-stamped entry by referring to the real-time data file that contains the time-stamped entry.

27. The data structure of claim 25 wherein the data in the third data field represents a time-stamped entry by referring to the non-real-time data file that contains the time-stamped entry.

28. The data structure of claim 25 further comprising a fourth data field containing data representing an association of the data structure with a data source.

36. The data structure of claim 25 wherein the time-stamped real-time and time data entries measure aspects of an industrial process.

31. In an environment with a real-time data source and with a non-real-time data source, a system for storing, in a time-coordinated fashion, data received from the real-time and non-real-time data sources, the system comprising:

a time interval;

a real-time, streaming data file;

a non-real-time data file;

a header file; and

a data storage coordinator for receiving data from the real-time data source, for storing the real-time data in the real-time data file as entries time-stamped with times associated with when the real-time data were produced, for receiving data from the non-real-time data source, for storing the non-real-time data in the non-real-time data file as entries time-stamped with times of relevance of the non-real-time data, and for associating the header file with those entries in the real-time and non-real-time data files whose time stamps fall within the time interval.

32. The system of claim 31 wherein the data storage coordinator associates the header file with the real-time and non-real-time data sources.

33. The system of claim 32 further comprising a user interface for associating the header file with another data source, the other data source distinct from the real-time and non-real-time data sources, and wherein the data storage coordinator receives data from the other data source and stores the data in a data file as time-stamped entries.

37. The system of claim 31 wherein at least some of the time-stamped real-time and non-real-time data entries measure aspect of an industrial process.