Method and system for data collection from remote sources

• US 6,691,116 B1
• Filed: 10/31/2001
• Issued: 02/10/2004
• Est. Priority Date: 10/31/2001
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Storability, Inc. (Oracle Corporation)
• Inventors: Bart, Eric S.
• Primary Examiner(s): Mizrahi; Diane D.
• Assistant Examiner(s): Mofiz; Apu M
• Application Number:
  US09/999,154
• Time in Patent Office: 832 Days
• Field of Search: 707/10,200,205 709/202,213,229,246 711/147 713/168
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/256 in federated or virtual databases
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

In accordance with one embodiment, a system is provided for collecting data from diverse data sources at a remote site. The system includes data collection daemons at the remote site, each for retrieving data from one of the data sources and for translating the data into a common format. A remote query agent collects data in the common format from the data collection daemons, and aggregates the data in a serialized data stream. A central query agent at a central collection site receives the data stream from the remote query agent. A database aggregator initiates data retrieval by the data collection daemons and deposits data received from the central query agent into a collection database. In accordance with another embodiment, a method is provided for remote data collection from diverse data sources. The method includes receiving a request at a remote collection site for collecting data from the data sources in accordance with a predetermined schedule; retrieving the data from the data sources in response to receiving the request; translating retrieved data from each data source into a common format; aggregating the data in the common format into a serialized a data stream; and transmitting the data stream to a central collection site for storage in a collection database.

Claims

1. A system for collecting data from diverse data sources at a remote site, comprising:

a plurality of data collection daemons at the remote site, each for retrieving data from one of the data sources and for translating the data into a common format;

a remote query agent for collecting data in the common format from the plurality of data collection daemons, and for aggregating the data in a serialized data stream;

a central query agent at a central collection site for receiving the data stream from the remote query agent;

a collection data base; and

a database aggregator for initiating data retrieval by the data collection daemons and for depositing data received from the central query agent into the collection database.

2. The system of claim 1 wherein the database aggregator initiates data collection by the data collection daemons by transmitting requests to the data collection daemons in accordance with a predetermined schedule.

3. The system of claim 1 wherein the common format comprises a vendor neutral data representation.

4. The system of claim 1 wherein the remote query agent transmits data to the central query agent over an encrypted communications link.

5. The system of claim 1 wherein the remote query agent transmits data to the central query agent a serialized encrypted data stream.

6. The system of claim 1 wherein the remote query agent transmits the data stream to the central query agent via at least one intermediate query agent.

7. The system of claim 6 wherein the at least one intermediate query agent collects data from another plurality of data collection daemons at another remote data collection site.

8. The system of claim 1 further comprising a redundant collection site for independently collecting data from the plurality of data collection daemons, the redundant collection site including a second central query agent, a second database aggregator and a second collection database.

9. A method for remote data collection from diverse data sources, comprising:

receiving a request at a remote collection site for collecting data from the data sources in accordance with a predetermined schedule;

retrieving the data from the data sources in response to receiving the request;

translating retrieved data from each data source into a common format;

aggregating the data in the common format into a serialized a data stream; and

transmitting the data stream to a central collection site for storage in a collection database.

10. The method of claim 9 wherein the common format comprises a vendor neutral data representation.

11. The method of claim 9 wherein the data stream is transmitted over an encrypted communications link.

12. The method of claim 9 wherein the data stream is transmitted to the central collection site via at least one intermediate query agent.

13. The method of claim 12 wherein the at least one intermediate query agent collects data from another plurality of data sources.

14. The method of claim 9 further comprising:

receiving a request at the remote collection site from a redundant central collection site for collecting data from the data sources;

retrieving the data from the data sources;

translating retrieved data into a common format;

aggregating the data in the common format into a serialized a data stream; and

transmitting the data stream to the redundant central collection site for storage in a collection database.

15. A method for collecting data at a central collection site from diverse data sources at a remote site, comprising:

generating a request at the central collection site for collecting data from the data sources at a remote collection site in accordance with a predetermined schedule;

transmitting the request to data collection devices associated with the data sources at the remote site; and

receiving data retrieved from the data sources, the data being translated into a common format and aggregated in a serialized data stream; and

storing data from the data stream in the collection database at the central collection site.

16. The method of claim 15 wherein the common format comprises a vendor neutral data representation.

17. The method of claim 15 wherein the data stream is received over an encrypted communications link.

18. The method of claim 15 wherein the data stream is transmitted to the central collection site via at least one intermediate query agent.

19. The method of claim 18 wherein the at least one intermediate query agent collects data from a plurality of other data sources.

20. The method of claim 15, further comprising:

generating a request at the central collection site for collecting data from data sources at another remote collection site in accordance with a predetermined schedule;

transmitting the request to data collection devices associated with the data sources at the another remote site; and

receiving data retrieved from the data sources at the another remote site, the data being translated into a common format and aggregated in a serialized data stream; and

storing data from the data stream in the collection database at the central collection site.

21. A system for collecting data from diverse data sources at a plurality of remote sites, comprising:

a plurality of data collection daemons at each remote site, each data collection daemon for retrieving data from one of the data sources an d for translating the data into a common format;

a remote query agent at each remote site for collecting data in the common format from the plurality of data collection daemons at the remote site, and for aggregating the data in a serialized data stream;

a central query agent at a central collection site for receiving the serialized data streams from the remote query agents at each remote site, and for aggregating the serialized data streams into a single data stream;

a collection data base; and

a database aggregator for initiating data retrieval by the data collection daemons and for depositing data received from the central query agent into the collection database.

22. The system of claim 21 wherein the database aggregator initiates data collection by the data collection daemons by transmitting requests to the data collection daemons in accordance with a predetermined schedule.

23. The system of claim 21 wherein the common format comprises a vendor neutral data representation.

24. The system of claim 21 wherein the remote query agent at each remote site transmits data to the central query agent over an encrypted communications link.

25. The system of claim 21 wherein at least some of the remote query agents transmit a data stream to the central query agent via at least one intermediate query agent.

26. The system of claim 25 wherein the at least one intermediate query agent collects data from another plurality of data collection daemons at another remote site.

27. The system of claim 21 further comprising a redundant collection site for independently collecting data from the plurality of data collection daemons, the redundant collection site including a second central query agent, a second database aggregator and a second collection database.

28. A system for remote data collection from diverse data sources, comprising:

means for receiving a request at a remote collection site for collecting data from the data sources in accordance with a predetermined schedule;

means for retrieving the data from the data sources;

means for translating retrieved data from each data source into a common format;

means for aggregating the data in the common format into a serialized a data stream; and

means for transmitting the data stream to a central collection site for storage in a collection database.

29. The system of claim 28 wherein the common format comprises a vendor neutral data representation.

30. A system for collecting data at a central collection site from diverse data sources at a remote site, comprising:

means for generating a request at the central collection site for collecting data from the data sources at a remote collection site in accordance with a predetermined schedule;

means for transmitting the request to data collection devices associated with the data sources at the remote site; and

means for receiving data retrieved from the data sources, the data being translated into a common format and aggregated in a serialized data stream; and

means for storing data from the data stream in the collection database at the central collection site.

31. The system of claim 30 wherein the common format comprises a vendor neutral data representation.