Method and apparatus for intelligent data assimilation

• US 6,847,974 B2
• Filed: 07/25/2001
• Issued: 01/25/2005
• Est. Priority Date: 03/26/2001
• Status: Expired due to Fees

• Current Assignee: Cricket Operating Company LLC
• Original Assignee: US Search Com Incorporated
• Inventors: Wachtel, David C.
• Primary Examiner(s): Corrielus, Jean M.
• Application Number:
  US09/916,243
• Publication Number:
  US 20020194181A1
• Time in Patent Office: 1,280 Days
• Field of Search: 707/100, 707/101, 707/104.1, 707/6, 707/3, 707/9, 707/10, 707/103, 715/513, 715/500.1, 709/202, 709/231, 709/232, 709/217, 709/220, 717/144, 717/114, 717/149
• US Class Current: 717/143
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: Y10S 707/955 Object-oriented
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

An intelligent data assimilation system including an ontology description, workflows, and logical search objects. The logical search objects operably connect to external and internal data providers and return search results using an ontology describing atomic data objects and semantic objects. The semantic objects are grouped into larger semantic structures by workflows to create customized services that return search results termed data products. Services are accessed through an application server capable of responding to service requests from different types of data clients. Graphical user interfaces provide facilities for creating logical search objects and aggregating logical search objects into workflows and services.

Claims

1. A method of fulfilling a data service request, the method comprising:

providing an ontology description of a data service, wherein the ontology description of the data service comprises a semantic description of atomic objects associated with the data service;

providing a first workflow;

providing a plurality of logical search objects operably coupled via a respective one of a plurality of communications links to a respective one of a plurality of data providers and operably coupled to the first workflow;

transmitting by the logical search objects to the data providers via the communications links a plurality of search requests, the search requests generated by the first workflow from the data service request;

receiving by the logical search objects from the data providers via the communications link a plurality of data sets in response to the search requests;

transmitting by the logical search objects to the first workflow the data sets; and

generating by the first workflow a knowledge instance from the data sets using the ontology description of the data service.

2. The method of claim 1 wherein at least one of the communications links is adapted for communications with a database server.

3. The method of claim 1 wherein at least one of the communications links is adapted for communications with a FTP server.

4. The method of claim 1 wherein at least one of the communications links is adapted for communications with a Web server.

5. The method of claim 1 wherein at least one of the communications links is adapted for communications with a file system.

6. The method of claim 1 wherein at least one of the communications links is adapted for communication with a human data provider.

7. The method of claim 1 wherein at least one of the communications links is adapted for communication with a communications protocol proprietary to the data provider.

8. The method of claim 1 wherein the data service request is included in a XML document.

9. The method of claim 1, further comprising:

providing an application server operably coupling a data client to the first workflow via a service request communications link, and

receiving by the application server from the data client via the service request communications link a data service request message, the data service request message including the data service request;

transmitting by the application server to the first workflow the data service request message; and

transmitting by the first workflow to at least one of the logical search objects the data service request message.

10. The method of claim 9, wherein the service request communications link is adapted for communications using SMTP.

11. The method of claim 9, wherein the service request communications link is adapted for communications using JMS.

12. The method of claim 9, wherein the service request communications link is adapted for communications using HTTP.

13. The method of claim 9, wherein the service request communications link is adapted for communications using RMI.

14. The method of claim 9, wherein the first workflow is specified by the application server using the service request message.

15. The method of claim 9 further comprising:

providing a formatter; and

formatting by the formatter the data set encapsulated in the knowledge instance into a format requested by the data client.

16. The method of claim 1 further comprising providing a second workflow operably coupled to the first workflow.

17. The method of claim 1, wherein the logical search objects are specified by the first workflow.

18. A method for accessing by a software object a data provider via a communications link, comprising:

receiving by the software object from a second software object a search request message document;

generating by the software object a data request for the data provider from the search request message document;

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set, including;

providing a parser adaptor operably coupled to the software object;

providing a parser semantic description of the data set for use by the parser adaptor;

providing a semantic object semantic description;

generating by the parser adaptor extracted data from the data set using the parser semantic description; and

generating by the parser adaptor the semantic object using the extracted data according to the semantic object semantic description.

19. A method for accessing by a software object a data provider via a communications link, comprising:

receiving by the software object from a second software object a search request message document;

generating by the software object a data request for the data provider from the search request message document, including;

providing a request builder operably coupled to the software object;

providing a native object operably coupled to the request builder, the native object encapsulating implementation details of a data request for the data provider;

providing a native semantic description including ontology information describing a data structure used by the request builder to build the data request for the data provider;

transmitting by the request builder to the native object the search request; and

generating by the native object the data request from the search request using the native semantic description;

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set.

20. A data processing object system adapted to fulfill a data service request, comprising:

a processor; and

a memory operabiy coupled to the processor and having program instructions stored therein, the processor being operable to execute the program instructions, the program instructions including;

providing an ontology description of a data service, wherein the ontology description of the data service comprises a semantic description of atomic objects associated with the data service;

providing a first workflow;

providing a plurality of logical search objects operably coupled via a respective one of a plurality of communications links to a respective one of a plurality of data providers and operably coupled to the first workflow;

transmitting by the logical data providers via the communications links a plurality of search requests, the search requests generated by the first workflow from the data service search objects to the description request;

receiving by the logical search objects from the data providers via the communications link a plurality of data sets in response to the search requests;

transmitting by the logical search objects to the first workflow the data sets; and

generating by the first workflow a knowledge instance from the data sets using the ontology description of the data service.

21. The data processing system of claim 20 at least one of the communications links is adapted for communications with a database server.

22. The data processing system of claim 20 wherein at least one of the communications links is adapted for communications with a FTP server.

23. The data processing system of claim 20 wherein at least one of the communications links is adapted for communications with a Web server.

24. The data processing system of claim 20 wherein at least one of the communications links is adapted for communications with a file system.

25. The data processing system of claim 20 wherein at least one of the communications links is adapted for communication with a human data provider.

26. The data processing system of claim 20 wherein at least one of the communications links is adapted for communication with a communications protocol proprietary to the data provider.

27. The data processing system of claim 20 wherein the data service request is included in a XML document.

28. The data processing system of claim 20, the program instructions further including:

providing an application server operably coupling a data client to the first workflow via a service request communications link, and

receiving by the application server from the data client via the service request communications link a data service request message, the data service request message including the data service request;

transmitting by the application server to the first workflow the data service request message; and

transmitting by the first workflow to at least one of the logical search objects the data service request message.

29. The data processing system of claim 28, wherein the service request communications link is adapted for communications using SMTP.

30. The data processing system of claim 28, wherein the service request communications link is adapted for communications using JMS.

31. The data processing system of claim 28, wherein the service request communications link, is adapted for communications using HTTP.

32. The data processing system of claim 28, wherein the service request communications line is adapted for communications using RMI.

33. The data processing system of claim 28, wherein the logical search objects are specified by the first workflow.

34. The data processing system of claim 28, wherein the first workflow is specified by the application server using the service request message.

35. The data processing system of claim 28, the program instructions further including:

providing a formatter; and

formatting by the formatter the data set encapsulated in the knowledge instance into a format requested by the data client.

36. The data processing system of claim 28, the program instructions further including providing a second workflow operably coupled to the first workflow.

37. A data processing system adapted to access a data provider via a communications link, comprising:

a processor; and

a memory operably coupled to the processor and having program instructions stored therein, the processor being operable to execute the program instructions, the program instructions including;

receiving by a software object a search request message document;

generating by the software object a data request for the data provider from the search request message document;

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set, including;

providing a parser adaptor operably coupled to the software object;

providing a parser semantic description of the data set for use by the parser adaptor;

providing a semantic object semantic description;

generating by the parser adaptor extracted data, from the data set using the parser semantic description; and

generating by the parser adaptor the semantic object using the extracted data according to the semantic object semantic description.

38. A data processing system adapted to access a data provider via a communications link, comprising:

a processor; and

a memory operably coupled to the processor and having program instructions stored therein, the processor being operable to execute the program instructions, the program instructions including;

receiving by a software object a search request message document;

generating by the software object a data request for the data provider from the search request message document, including;

providing a request builder operably coupled to the software object;

providing a native object operably coupled to the request builder, the native object encapsulating implementation details of a data request for the data provider;

providing a native semantic description including ontology information describing a data structure used by the request builder to build the data request for the data provider;

transmitting by the request builder to the native object the search request; and

generating by the native object the data request from the search request using the native semantic description;

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set.

39. A computer readable media embodying program instructions for execution by a computer, the computer program instructions adapting a computer to fulfill a data service request, the program instructions comprising:

providing an ontology description of a data service, wherein the ontology description of the data service comprises a semantic description of atomic objects associated with the data service;

providing a first workflow;

providing a plurality of logical search object a operably coupled via a respective one of a plurality of communications links to a respective one of a plurality of data providers and operably coupled to the first workflow;

transmitting by the logical search objects to the data providers via the communications links a plurality of search requests the search requests generated by the first workflow from the data service request;

receiving by the logical search objects from the data providers via the communications link a plurality of data sets in response to the search requests;

transmitting by the logical search objects to the first workflow the data sets; and

generating by first workflow a knowledge instance from the data sets using the ontology description of the service.

40. The computer readable media of claim 39 wherein at least one of the communications links is adapted for communications with a database server.

41. The computer readable media of claim 39 wherein at least one of the communications links is adapted for communications with a FTP server.

42. The computer readable media of claim 39 wherein at least one of the communications links is adapted for communications with a Web server.

44. The computer readable media of claim 39 wherein at least one of the communications links is adapted for communication with a human data provider.

45. The computer readable media of claim 39 wherein at least one of the communications links is adapted for communication with a communications protocol proprietary to the data provider.

46. The computer readable media of claim 39 wherein the data service request is included in a XML document.

47. The computer readable media of claim 39, the program instructions further comprising:

providing an application server operably coupling a data client to the first workflow via a service request communications link, and

receiving, by the application server from the data client via the service request communications link a data service request message, the data service request message including the data service request;

transmitting by the application server to the first workflow the data service request message; and

transmitting by the first workflow to at least one of the logical search objects the data service request message.

48. The computer readable media of claim 47 wherein the service request communications link is adapted for communications using SMTP.

49. The computer readable media of claim 47 wherein the service request communications link is adapted for communications using JMS.

50. The computer readable media of claim 47 wherein the service request communications link is adapted for communications using HTTP.

51. The computer readable media of claim 47 wherein the service request communications link is adapted for communications using RMI.

52. The computer readable media of claim 47, wherein at least one of the logical search objects is specified by the first workflow.

53. The computer readable media of claim 47, wherein the first workflow is specified by the application server using the service request message.

54. The computer readable media of claim 47, the program instructions further comprising:

providing a formatter; and

formatting by the formatter the data set encapsulated in the knowledge instance into a format requested by the data client.

55. The computer readable media of claim 47, the program instructions further comprising providing a second workflow operably coupled to the first workflow.

43. The computer readable media of aim 39 wherein at least one of the communications links is adapted for communications with a file system.

56. A computer readable media embodying program instructions for execution by a computer, the computer program instructions adapting a computer to access a data provider via a communications link, the program instructions comprising:

receiving by a software object a search request message document;

generating by the software object a data request for the data provider from the search request message document;

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set, including;

providing a parser adaptor operably coupled to the software object;

providing a parser semantic description of the data set for use by the parser adaptor;

providing a semantic object semantic description;

generating by the parser adaptor extracted data from the data set using the parser semantic description; and

generating by the parser adaptor the semantic object using the extracted data according to the semantic object semantic description.

57. A computer readable media embodying program instructions for execution by a computer, the computer program instructions adapting a computer to access a data provider via a communications link, the program instructions comprising:

receiving by a software object a search request message document;

generating by the software object a data request for the data provider from the search request message document;

providing a request builder operably coupled to the software object;

providing a native object operably coupled to the request builder, the native object encapsulating implementation details of a data request for the data provider;

providing a native semantic description including ontology information describing a data structure used by the request builder to build the data request for the data provider;

transmitting by the request builder to the native object the search request; and

generating by the native object the data request from the search request using the native semantic description.

transmitting by the software object to the data provider the data request via the communications link;

receiving by the software object from the data provider a data set via the communications link; and

generating by the software object a semantic object from the data set.

58. A method of building a knowledge instance for a data service, comprising:

providing a plurality of logical search objects, each logical search object operable to retrieve a data set from one or more data providers;

instantiating by a first workflow one or more logical search objects selected from the plurality of logical search objects;

receiving by the one or more logical search objects data sets from the one or more data providers;

transmitting by the one or more logical search objects to the workflow the data sets; and

generating by the workflow the knowledge instance from the data seta using an ontology description of a data service, wherein the ontology description of the data service comprises a semantic description of atomic objects associated with the data service.

59. The method of claim 58, wherein at least one of the data providers is a database server.

60. The method of claim 58, wherein at least one of the data providers is a FTP server.

61. The method of claim 58, wherein at least one of the data providers is a Web server.

62. The method of claim 58, wherein at least one of the data providers is a file system.

63. The method of claim 58, wherein at least one of the data providers is a human data provider.

64. The method of claim 58 wherein at least one of the logical search objects is coupled to at least one data provider via a communications link adapted for communications using a communications protocol proprietary to the data provider.

65. The method of claim 58 further comprising providing a second workflow operably coupled to the first workflow.

66. The method of claim 58, further comprising:

providing an application server operably coupling a data client to the first workflow via a service request communications link, and

receiving by the application server from the data client via the service request communications link a data service request message, the data service request message including a data service request;

transmitting by the application server to the first workflow the data service request message; and

transmitting by the first workflow to at least one of the logical search objects the data service request message.

67. The method of claim 66, wherein the first workflow is specified by the application server using the service request message.

68. The method of claim 66, further comprising:

providing a formatter; and

formatting by the formatter the knowledge instance into a format requested by the data client.

69. The method of claim 66, wherein the service request communications link is adapted for communications using SMTP.

70. The method of claim 66, wherein the service request communications link is adapted for communications using JMS.

71. The method of claim 66, wherein the service request communications link is adapted for communications using HTTP.

72. The method of claim 66, wherein the service request communications link is adapted for communications using RMI.

73. A data processing system adapted to build a knowledge instance for a data service, comprising:

a processor; and

a memory operablv coupled to the processor and having program instructions stored therein, the processor being operable to execute the program instructions, the program instructions including;

providing a plurality of logical search objects, each logical search object operable to retrieve a data set from one or more data providers;

instantiating by a first workflow one or more logical search objects selected from the plurality of logical search objects;

receiving by the one or more logical search objects data sets from the one or more data providers;

transmitting by the one or more logical search objects to the workflow the data sets; and

generating by the workflow the knowledge instance from the data seta using an ontology description of a data service, wherein the ontology description of the data service comprises a semantic description of atomic objects associated with the data service.

74. The data processing system of claim 73, wherein at least one of the data providers is a database server.

75. The data processing system of claim 73, wherein at least one of the data providers is a FTP server.

76. The data processing system of claim 73, wherein at least one of the data providers is a Web server.

77. The data processing system of claim 73, wherein at least one of the data providers is a file system.

78. The data processing system of claim 73, wherein at least one of the data providers is a human data provider.

79. The data processing system of claim 73, wherein at least one of the logical search objects is coupled to at least one data provider via a communications link adapted for communications using a communications protocol proprietary to the data provider.

80. The data processing system of claim 73, the program instructions further including providing a second workflow operably coupled to the first workflow.

81. The data processing system of claim 77, the program instructions further including:

providing an application server operably coupling a data client to the first workflow via a service request communications link, and

receiving by the application server from the data client via the service request communications link a data service request message, the data service request message including a data service request;

transmitting by the application server to the first workflow the data service request message; and

transmitting by the first workflow to at least one of the logical search objects the data service request message.

82. The data processing system of claim 81, wherein the first workflow is specified by the application server using the service request message.

83. The data processing system of claim 81, further comprising:

providing a formatter; and

formatting by the formatter the knowledge instance into a format requested by the data client.

84. The data processing system of claim 81, wherein the service request communications link is adapted for communications using SMTP.

85. The data processing system of claim 81, wherein the service request communications link is adapted for communications using JMS.

86. The data processing system of claim 81, wherein the service request communications link is adapted for communications using HTTP.

87. The data processing system of claim 81, wherein the service request communications link is adapted for communications using RMI.