Implicit dialog approach for creating conversational access to web content

• US 10,824,658 B2
• Filed: 08/02/2018
• Issued: 11/03/2020
• Est. Priority Date: 08/02/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bakis, Raimo, Feng, Song, Ganhotra, Jatin, Gunasekara, Chulaka, Nahamoo, David, Polymenakos, Lazaros, Shashidhara, Sunil D, Wu, Cheng, Zhu, Li
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US16/053,718
• Publication Number:
  US 20200042642A1
• Time in Patent Office: 824 Days
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/328 Management therefor
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A method, apparatus and computer program product for creating a dialog system for web content is described. Knowledge is extracted from a target web application for the dialog system. The knowledge includes an organizational structure of the target web application and domain knowledge pertinent to the target web application. A deep learning process associates the domain knowledge with the organization structure of the target application. A plurality of knowledge sources of different respective types are created from the domain knowledge and the organizational structure. Each of the knowledge sources is used for providing answers to user queries to the dialog system. As part of the invention, a semantic matcher is provided to select among the answers provided by the plurality of knowledge sources for a best answer to a user query.

Claims

1. A method for creating a dialog system for web content comprising:

extracting knowledge from a target web application for the dialog system, the knowledge including an organizational structure of the target web application and domain knowledge pertinent to the target web application;

using a deep learning process, associating the domain knowledge with the organization structure of the target application;

using the associated domain knowledge, creating a plurality of knowledge sources of different respective data storage types from the domain knowledge and the organizational structure, wherein ones of the plurality of knowledge sources are selected from the group of a relational database, a knowledge graph and a non-relational database, each of the knowledge sources for providing answers to user queries to the dialog system; and

providing a semantic matcher to select among the answers provided by the plurality of knowledge sources for a best answer to a user query.

2. The method as recited in claim 1, wherein the semantic matcher includes a deep learning process and the method further comprises training the semantic matcher using the domain knowledge.

3. The method as recited in claim 1, further comprising:

pointing a crawler to the target web application;

identifying a set of web pages used with the target web application;

extracting a set of natural language sentence and path pairs from the set of web pages; and

creating a set of triples from the set of natural language and path pairs, each triple containing a pair of sentence elements and a relationship between the sentence elements.

4. The method as recited in claim 3, further comprising indexing the set of triples and the set of natural language and path pairs using the organizational structure.

5. The method as recited in claim 1, further comprising providing a query generator for creating respective queries formatted for each of the plurality of knowledge sources in response to a single user input.

6. The method as recited in claim 1, wherein the plurality of knowledge sources comprise a frequently asked question database, a knowledge graph and a relational database, and the method further comprises:

using a dialog question and answer construction module to create the frequently asked question database;

using a triple construction module to create the knowledge graph; and

using a table construction module to create a table in the relational database.

7. The method as recited in claim 1, further comprising providing a domain trained natural language processing service which includes the semantic matcher for determining entities and user intent in a natural language user input using the domain knowledge.

8. The method as recited in claim 1, wherein the semantic matcher selects among a plurality of answers respectively provided by different ones of the plurality of knowledge sources for the best answer to a user query according to the domain knowledge.

9. The method as recited in claim 1, wherein the associating uses at least some unstructured domain knowledge from a source outside the organization structure.

10. The method as recited in claim 1, wherein the plurality of knowledge sources includes a relational database, a knowledge graph and a non-relational base.

11. Apparatus, comprising:

a processor;

computer memory holding computer program instructions executed by the processor for creating a dialog system for web content, the computer program instructions comprising;

program code, operative to extract knowledge from a target web application for the dialog system, the knowledge including an organizational structure of the target web application and domain knowledge pertinent to the target web application;

program code, operative to use a deep learning process, associating the domain knowledge with the organization structure of the target application;

program code, operative to use the associated domain knowledge to create a plurality of knowledge sources of different respective data storage types from the domain knowledge and the organizational structure, wherein ones of the plurality of knowledge sources are selected from the group of a relational database, a knowledge graph and a non-relational database, each of the knowledge sources for providing answers to user queries to the dialog system; and

program code, operative to provide a semantic matcher to select among the answers provided by the plurality of knowledge sources for a best answer to a user query.

12. The apparatus as recited in claim 11, wherein the semantic matcher includes a deep learning process and the method further comprises training the semantic matcher using the domain knowledge.

13. The apparatus as recited in claim 11, further comprising:

program code, operative to point a crawler to the target web application;

program code, operative to identify a set of web pages used with the target web application;

program code, operative to extract a set of natural language sentence and path pairs from the set of web pages; and

program code, operative to create a set of triples from the set of natural language and path pairs, each triple containing a pair of sentence elements and a relationship between the sentence elements.

14. The apparatus as recited in claim 13, further comprising program code, operative to index the set of triples and the set of natural language and path pairs using the organizational structure.

15. The apparatus as recited in claim 11, further comprising program code, operative to provide a query generator for creating respective queries formatted for each of the plurality of knowledge sources in response to a single user input.

16. The apparatus as recited in claim 13, wherein the plurality of knowledge sources comprise a frequently asked question database, a knowledge graph and a relational database, and the computer program instructions further comprises:

program code, operative to use a dialog question and answer construction module to create the frequently asked question database;

program code, operative to use a triple construction module to create the knowledge graph; and

program code, operative to use a table construction module to create a table in the relational database.

17. A computer program product in a non-transitory computer readable medium for use in a data processing system, the computer program product holding computer program instructions executed by the data processing system for creating a dialog system for web content, the computer program instructions comprising:

program code, operative to extract knowledge from a target web application for the dialog system, the knowledge including an organizational structure of the target web application and domain knowledge pertinent to the target web application;

program code, operative to use a deep learning process, associating the domain knowledge with the organization structure of the target application;

program code, operative to use the associated domain knowledge to create a plurality of knowledge sources of different respective data storage types from the domain knowledge and the organizational structure, wherein ones of the plurality of knowledge sources are selected from the group of a relational database, a knowledge graph and a non-relational database, each of the knowledge sources for providing answers to user queries to the dialog system; and

program code, operative to provide a semantic matcher to select among the answers provided by the plurality of knowledge sources for a best answer to a user query.

18. The computer program product as recited in claim 17, wherein the semantic matcher includes a deep learning process and the method further comprises training the semantic matcher using the domain knowledge.

19. The computer program product as recited in claim 17, further comprising:

program code, operative to point a crawler to the target web application;

program code, operative to identify a set of web pages used with the target web application;

program code, operative to extract a set of natural language sentence and path pairs from the set of web pages; and

program code, operative to create a set of triples from the set of natural language and path pairs, each triple containing a pair of sentence elements and a relationship between the sentence elements.

20. The computer program product as recited in claim 17, further comprising program code, operative to index the set of triples and the set of natural language and path pairs using the organizational structure.

21. The computer program product as recited in claim 17, further comprising program code, operative to provide a query generator for creating respective queries formatted for each of the plurality of knowledge sources in response to a single user input.

22. The computer program product as recited in claim 17, wherein the plurality of knowledge sources comprise a frequently asked question database, a knowledge graph and a relational database, and the computer program instructions further comprises:

program code, operative to use a dialog question and answer construction module to create the frequently asked question database;

program code, operative to use a triple construction module to create the knowledge graph; and

program code, operative to use a table construction module to create a table in the relational database.