Method, system and computer product for continuously monitoring data sources for an event of interest

• US 20040164961A1
• Filed: 02/21/2003
• Published: 08/26/2004
• Est. Priority Date: 02/21/2003
• Status: Abandoned Application

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Sarkar, Abhinanda, Bal, Debasis, Subramanian, Gopi, Rajagopalan, Srikanth
• Application Number:
  US10/371,724
• Publication Number:
  US 20040164961A1
• Time in Patent Office: Days
• US Class Current: 345/163
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation

Abstract

Method, system and computer product for continuously monitoring data sources for an event of interest. A continuous stream of data from the plurality of data sources is received and the data that relates to the event of interest is extracted and stored in a historical database. Analytics is performed on the stored data that relates to the event of interest. In addition, the stored data is monitored according to the performed analytics to find data that relates to the event of interest.

Claims

1. A method for continuously monitoring a plurality of data sources for an event of interest, comprising;

receiving a continuous stream of data from the plurality of data sources;

extracting data that relates to the event of interest from the received data;

storing the extracted data that relates to the event of interest in a historical database;

performing analytics on the stored data that relates to the event of interest; and

monitoring the stored data according to the performed analytics to find data that relates to the event of interest.

2. The method of claim 1, wherein the plurality of data sources comprise data selected from the group of textual data, graphical data and numeric data.

3. The method of claim 1, wherein the receiving comprises automatically searching and downloading data from the plurality of data sources at predetermined time intervals.

4. The method of claim 1, wherein the extracting comprises automatically parsing the received data to extract data that relates to the event of interest.

5. The method of claim 4, wherein the parsing comprises validating and transforming the received data to generate a parsed data set.

6. The method of claim 5, wherein the parsing further comprises loading the parsed data set onto the historical database.

7. The method of claim 1, wherein the storing comprises creating a data storage schema dynamically to store the extracted data in the historical database.

8. The method of claim 1, wherein the performing comprises extracting a desired data set that relates to the event of interest from the historical database.

9. The method of claim 8, wherein the performing further comprises computing a parameter from the desired data set that relates to the event of interest.

10. The method of claim 9, further comprising periodically updating the parameter based on the analytics performed on the continuous stream of data.

11. The method of claim 1, wherein the monitoring comprises comparing the event of interest to a threshold value.

12. An Internet-based method for continuously monitoring a plurality of data sources for information that relates to an event of interest, comprising:

configuring an agent to search the plurality of data sources according to a specific search criteria;

using the agent to download a plurality of web pages containing information related to the plurality of data sources at predetermined time intervals;

parsing the data in the web pages;

storing the parsed data in a data repository;

extracting data from the data repository;

determining parameters of interest from the extracted data, wherein the parameters of interest relate to the event of interest; and

continuously monitoring the determined parameters of interest to find information that relates to the event of interest.

13. The Internet based method of claim 12, wherein the data sources comprise data selected from the group of textual data, graphical data and numeric data.

14. The Internet based method of claim 12, wherein the configuring comprises using a browser object with a search string embedded in it.

15. The Internet based method of claim 12, wherein the parsing comprises extracting data that relates to the event of interest from the plurality of web pages.

16. The Internet based method of claim 15, wherein the parsing further comprises validating and transforming the extracted data to generate the parsed data.

17. The Internet based method of claim 12, wherein the storing comprises loading the parsed data onto the data repository.

18. The Internet based method of claim 17, wherein the storing further comprises creating a data storage schema dynamically to load the parsed data set in the data repository.

19. The Internet based method of claim 12, wherein the extracting comprises performing analytics on the stored data to extract information relevant to the event of interest.

20. The Internet based method of claim 12, wherein the parameters of interest comprise corporate defaults, financial distress, corporate leasing decisions and weather forecasts.

21. A system for continuously monitoring a plurality of data sources for an event of interest, comprising;

an agent configured to automatically search and download data from the plurality of data sources at pre-determined time intervals;

a data extraction component configured to extract data that relates to the event of interest from the data downloaded by the agent;

a database configured to store the extracted data that relates to the event of interest; and

an analytics engine configured to perform analytics on the stored data.

22. The system of claim 21, wherein the plurality of data sources comprise data selected from the group of textual data, graphical data and numeric data.

23. The system of claim 21, wherein the agent comprises a browser object with a search string embedded in it.

24. The system of claim 21, wherein the data extraction component is configured to automatically parse the downloaded data to extract information that relates to the event of interest.

25. The system of claim 24, wherein the data extraction component is further configured to validate and transform the data to generate a parsed data set.

26. The system of claim 25, wherein the data extraction component is configured to load the parsed data set onto the database.

27. The system of claim 21, wherein the database is configured to create a data storage schema dynamically to store the extracted data in the historical database.

28. The system of claim 21, wherein the analytics engine is configured to extract a desired data set that relates to the event of interest from the database.

29. The system of claim 28, wherein the analytics engine is configured to compute a parameter from the desired data set, wherein, the parameter relates to the event of interest.

30. The system of claim 21, wherein the analytics engine is configured to continuously monitor the stored data to find information that relates to the event of interest.

31. A computer-readable medium storing computer instructions for instructing a computer system to continuously monitor a plurality of data sources for an event of interest, the computer instructions comprising;

receiving a continuous stream of data from the plurality of data sources;

extracting data that relates to the event of interest from the received data storing the extracted data that relates to the event of interest in a historical database;

performing analytics on the stored data that relates to the event of interest; and

monitoring the stored data according to the performed analytics to find data that relates to the event of interest.

32. The computer-readable medium of claim 31, wherein the plurality of data sources comprise data selected from the group of textual data, graphical data and numeric data.

33. The computer-readable medium of claim 31, wherein the receiving comprises receiving instructions for automatically searching and downloading data from the plurality of data sources at predetermined time intervals.

34. The computer-readable medium of claim 31, wherein the extracting comprises processing instructions for automatically parsing the received data to extract data that relates to the event of interest.

35. The computer-readable medium of claim 34, wherein the parsing further comprises instructions for validating and transforming the received data to generate a parsed data set.

36. The computer-readable medium of claim 35, wherein the parsing further comprises instructions for loading the parsed data set onto the historical database.

37. The computer-readable medium of claim 31, wherein the storing comprises instructions for creating a data storage schema dynamically to store the extracted data in the historical database.

38. The computer-readable medium of claim 31, wherein the performing comprises instructions for extracting a desired data set that relates to the event of interest from the historical database.

39. The computer-readable medium of claim 38, wherein the performing further comprises instructions for computing a parameter from the desired data set that relates to the event of interest.

40. The computer-readable medium of claim 39, further comprising instructions for periodically updating the parameter based on the analytics performed on the continuous stream of data.

41. The computer-readable medium of claim 31, wherein the monitoring comprises instructions for comparing the event of interest to a threshold value.

42. A computer-readable medium storing computer instructions for instructing a computer system to continuously monitor a plurality of data sources from the Internet for information that relates to an event of interest, comprising:

configuring an agent to search the plurality of data sources according to a specific search criteria;

using the agent to download a plurality of web pages containing information related to the plurality of data sources at predetermined time intervals;

parsing the data in the web pages;

storing the parsed data in a data repository;

extracting data from the data repository;

determining parameters of interest from the extracted data, wherein the parameters of interest relate to the event of interest; and

continuously monitoring the determined parameters of interest to find information that relates to the event of interest.

43. The computer-readable medium of claim 42, wherein the data sources comprise data selected from the group of textual data, graphical data and numeric data.

44. The computer-readable medium of claim 42, wherein the configuring comprises instructions for using a browser object with a search string embedded in it.

45. The computer-readable medium of claim 42, wherein the parsing comprises instructions for extracting data that relates to the event of interest from the plurality of web pages.

46. The computer-readable medium of claim 45, wherein the parsing further comprises instructions for validating and transforming the extracted data to generate the parsed data.

47. The computer-readable medium of claim 42, wherein the storing comprises instructions for loading the parsed data onto the data repository.

48. The computer-readable medium of claim 47, wherein the storing further comprises instructions for creating a data storage schema dynamically to load the parsed data set in the data repository.

49. The computer-readable medium of claim 42, wherein the extracting comprises instructions for performing analytics on the stored data to extract information relevant to the event of interest.

50. The computer-readable medium of claim 42, wherein the parameters of interest comprise corporate defaults, financial distress, corporate leasing decisions and weather forecasts.

51. A method in a computer system for displaying a plurality of pages to enable a user to view information related to quoted financial assets of an entity to determine the financial health of the entity, comprising:

displaying an input screen for permitting the user to input a request to search for the entity;

displaying a query screen for permitting a user to query for data related to the financial assets for the searched entity; and

displaying an alert screen for permitting the user to view a recent change to the financial health of at least one entity.

52. The method of claim 51, wherein the financial health is computed based on online analysis of the quoted financial assets pertaining to the entity.

53. The method of claim 52, wherein the financial health further comprises detection of default frequency behavior of the entity.

54. The method of claim 51, wherein the input screen for permitting the user to input a request to search for the entity comprises querying for the entity by means of a user input search string.

55. The method of claim 51, wherein the input screen for permitting the user to input a request to search for the entity further comprises searching for the entity by means of a drop down menu selection.

56. The method of claim 51, wherein the query screen for permitting a user to query data related to the financial assets for the searched entity comprises displaying a screen with a start date, an end date and a desired interval frequency for viewing transitions to the financial health of the searched entity over a time window.

57. The method of claim 56, wherein viewing the transitions in the financial health of the searched entity over the time window comprises displaying a screen with an updated value of the financial health computed over the time window as per the desired interval frequency;

and wherein the updated value of the computed financial health is traced over the time window numerically and graphically.

58. The method of claim 57, wherein the financial health reflects a risk of default of the entity over the time window.

59. The method of claim 51, wherein the query screen for permitting a user to query data related to the financial assets further comprises displaying a screen for viewing the quoted financial assets pertaining to the entity.

60. The method of claim 51, wherein the alert screen for permitting the user to view a recent change in the financial health for at least one entity comprises displaying a screen for viewing a date of change in the financial health of the entity and the effect of the recent change on the financial health of the entity.

61. The method of claim 60, wherein the recent change in the financial health of the entity represents the current financial health of the entity on the date of change.

62. A system for continuously monitoring a plurality of data sources for an event of interest, comprising:

a processing unit configured to download, extract and analyze a continuous stream of data from the plurality of data sources, the processing unit comprising;

a data fetch entity configured to automatically search and download the continuous stream of data from the plurality of data sources;

a data extraction entity configured to extract data that relates to the event of interest from the continuous stream of data; and

an analytics engine configured to compute and monitor a parameter of interest related to the event of interest from the extracted data;

a memory unit configured to store a plurality of data used by the processing unit; and

a user interface configured to interface the processing unit with a user.

63. The system of claim 62, wherein the data fetch entity of the processing unit comprises a data seeking portion configured to connect to and download the continuous stream of data from the plurality of data sources.

64. The system of claim 62, wherein the data extraction entity of the processing unit comprises:

a rules processing portion configured to process rules to extract data from the continuous stream of data that relates to the event of interest;

a data parsing portion configured to validate and transform the extracted data to generate a parsed data set; and

a data loading portion configured to load the parsed data set onto a data repository.

65. The system of claim 62, wherein the analytics engine of the processing unit comprises:

a rules generation portion configured to classify the parameter into pre-defined threshold ranges;

a data processing portion configured to compute the parameter that relates to the event of interest from the extracted data; and

a data loading portion configured to load the computed parameter onto a data repository.

66. The system of claim 62, wherein the memory unit comprises:

a data seeker memory portion configured to store the continuous stream of data from the plurality of data sources;

a rules memory portion configured to store rules for data extraction and threshold values for the parameter;

a parser memory portion configured to store the extracted data that relates to the event of interest; and

a data process memory portion configured to store information related to the computed parameter.

67. The system of claim 62, wherein the user interface comprises displaying a plurality of pages to enable a user to view information related to the event of interest.