Providing a natural language based application program interface

• US 10,719,667 B1
• Filed: 08/05/2015
• Issued: 07/21/2020
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Roy, Howard Scott
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US14/818,579
• Time in Patent Office: 1,812 Days
• Field of Search: 704 9, 704275, 704277, 704235, 704260, 704231, 717140, 717168, 717106, 717769, 717803, 705 30, 705 32, 705 34, 381 56
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing a natural language based program interface to software applications. One of the methods includes, obtaining, via a natural language front end, a natural language query or a natural language update statement issued by a software application; converting the natural language query or natural language update statement into structured operations to be performed on APIs of a knowledge base; performing the structured operations on the APIs to produce a natural language output statement; and providing, via a natural language output interface, the natural language output statement to the software application. The knowledge base stores entity information according to a data schema and has structured APIs for use by software applications to query the knowledge base; the software applications are limited to communicating with the knowledge base through the interfaces provided by the natural language front end.

Claims

1. A system comprising:

a knowledge base and a natural language front end, wherein the knowledge base and the natural language front end are implemented on one or more computers and one or more storage devices;

wherein the knowledge base stores structured content items according to a data schema and provides a structured application program interface (API) and not a natural language interface for receiving queries of and updates to the knowledge base from software applications;

wherein the natural language front end is configured to (i) enable multiple software applications to interact with the knowledge base using a natural language interface provided by the natural language front end and (ii) interact with the knowledge base using the structured API of the knowledge base, and wherein the natural language front end includes;

a natural language input interface to receive, from the software applications, natural language queries and natural language update statements;

a conversion database and a conversion subsystem that are configured to;

convert natural language queries to structured API query operations of the structured API of the knowledge base;

convert natural language update statements to structured API update operations of the structured API of the knowledge base;

perform the structured API query operations on the structured API of the knowledge base to obtain one or more structured content items from the knowledge base;

perform the structured API update operations on the structured API of the knowledge base to cause the knowledge base to update structured content items of the knowledge base;

convert structured content items obtained by performing the structured API query operations to natural language output statements; and

perform at least one of (i) augmenting at least one of the natural language queries by adding, to the natural language query, an identifier for one or more objects in the knowledge base that are referenced in the natural language query, or (ii) providing, with at least one of the natural language output statements, one or more identifiers that are recognized by the software application that provided the natural language query that corresponds to the natural language output statement; and

a natural language output interface to provide the natural language output statements to the software applications.

2. The system of claim 1, wherein the conversion database and conversion subsystem are configured to:

obtain one or more respective data identifiers for one or more data items in structured content items obtained by the structured API query operations; and

include the one or more data identifiers to identify terms in the natural language output statements.

3. The system of claim 1, wherein the natural language front end is further configured to provide non-natural language output statements corresponding to the natural language output statements.

4. The system of claim 1, wherein each software application in the multiple software applications includes a natural language query generation module to generate a natural language query according to a user interaction that is other than providing the natural language query with a software application.

5. The system of claim 1, wherein:

the conversion database includes a respective application schema for each of one or more of the software applications, wherein each application schema defines mappings between (i) terms in the natural language queries and the natural language update statements received from the software application corresponding to the respective application schema, and (ii) terms included in the data schema of the knowledge base; and

the conversion database includes the data schema of the knowledge base.

6. The system of claim 1, wherein:

the knowledge base is a structured query language (SQL) database; and

the structured API operations are SQL statements.

7. The system of claim 1, wherein the conversion database and the conversion subsystem are configured to augment at least one of the natural language queries by adding, to the natural language query, an identifier for one or more objects in the knowledge base that are referenced in the natural language query.

8. The system of claim 1, wherein the conversion database and the conversion subsystem are further configured to receive contextual data, from the multiple software applications, that indicates a description, separate from user input, of a context of a user device when the natural language query was issued by a user of the user device;

and

wherein the conversion database and the conversion subsystem are configured to convert the natural language queries to the structured API query operations based on the contextual data.

9. The system of claim 1, wherein the conversion database and the conversion subsystem are configured to provide, with at least one of the natural language output statements, one or more identifiers that are recognized by the software application that provided the natural language query that corresponds to the natural language output statement.

10. A method performed by a natural language front end running on a system of one or more computers, the method comprising:

receiving, by a natural language interface of the natural language front end, from multiple software applications, natural language queries and natural language update statements directed to a knowledge base that stores structured content items according to a data schema and provides a structured application program interface (API) and not a natural language interface for receiving queries of and updates to the knowledge base from software applications, wherein the natural language front end is configured to (i) enable multiple software applications to interact with the knowledge base using a natural language interface provided by the natural language front end and (ii) interact with the knowledge base using the structured API of the knowledge base;

performing, by a conversion subsystem using a conversion database, operations comprising;

converting the natural language queries to one or more structured API query operations of the structured API of the knowledge base;

converting the natural language update statements to structured API update operations of the structured API of the knowledge base;

performing the structured API query operations on the structured API of the knowledge base to obtain one or more structured content items from the knowledge base;

performing the structured API update operations on the structured API of the knowledge base to cause the knowledge base to update structured content items of the knowledge base;

converting structured content items obtaining by performing the structured API query operations to natural language output statements;

performing at least one of (i) augmenting at least one of the natural language queries by adding, to the natural language query, an identifier for one or more objects in the knowledge base that are referenced in the natural language query, or (ii) providing, with at least one of the natural language output statements, one or more identifiers that are recognized by the software application that provided the natural language query that corresponds to the natural language output statement; and

providing, by a natural language output interface of the natural language front end, the natural language output statements to the software applications.

11. The method of claim 10, wherein each software application in the software applications includes a natural language query generation module to generate a natural language query according to a user interaction that is other than providing the natural language query with a software application.

12. The method of claim 10, further comprising:

obtaining one or more respective data identifiers for one or more data items in structured content items obtained by the structured API query operations; and

including the one or more data identifiers to identify terms in the natural language output statements.

13. The method of claim 10, further comprising:

providing non-natural language output statements corresponding to the natural language output statements.

14. The method of claim 10, wherein:

the conversion database includes a respective application schema for each of one or more of the software applications, wherein each application schema defines mappings between (i) terms in the natural language queries and the natural language update statements received from the software application corresponding to the respective application schema, and (ii) terms included in the data schema of the knowledge base; and

the conversion database includes the data schema of the knowledge base.

15. The method of claim 10, wherein:

the knowledge base is a SQL database; and

the structured API operations are SQL statements.

16. The method of claim 10, wherein the natural language queries and natural language update statements are received through a communication network from software applications on different user devices.

17. The method of claim 10, wherein the natural language front end receives and responds to the natural language queries and natural language update statements from the multiple applications (i) without revealing to the applications syntax, input parameters, and output parameters of the APIs of the knowledge base to the software applications, and (ii) without revealing to the applications a form in which the knowledge base stores the obtained one or more structured content items.

18. One or more non-transitory computer storage media encoded with a computer program, the computer program comprising instructions that when executed by a natural language front end running on a system of one or more computers cause the natural language front end to perform operations comprising:

receiving, from multiple software applications, natural language queries of a knowledge base and natural language update statements to the knowledge base, wherein the knowledge base stores structured content items according to a data schema and provides a structured application program interface (API) and not a natural language interface for receiving queries of and updates to the knowledge base from software applications;

converting the natural language queries into one or more structured API query operations for the structured API of the knowledge base;

converting the natural language update statements to structured API update operations of the structured API of the knowledge base;

performing the structured API query operations on the structured API of the knowledge base to obtain one or more structured content items from the knowledge base;

performing the structured API update operations on the structured API of the knowledge base to cause the knowledge base to update structured content items of the knowledge base;

converting structured content items obtained by performing the structured API query operations to natural language output statements;

performing at least one of (i) augmenting at least one of the natural language queries by adding, to the natural language query, an identifier for one or more objects in the knowledge base that are referenced in the natural language query, or (ii) providing, with at least one of the natural language output statements, one or more identifiers that are recognized by the software application that provided the natural language query that corresponds to the natural language output statement; and

providing the natural language output statements to the multiple software applications;

wherein the natural language front end is configured to (i) enable multiple software applications to interact with the knowledge base using a natural language interface provided by the natural language front end and (ii) interact with the knowledge base using the structured API of the knowledge base.

19. The one or more non-transitory computer storage media of claim 18, wherein each software application in the software applications includes a natural language query generation module to generate a natural language query according to a user interaction that is other than providing the natural language query with a software application.

20. The one or more non-transitory computer storage of claim 18, the operations further comprising:

obtaining one or more respective data identifiers for one or more data items in structured content items obtained by the structured API query operations; and

including the one or more data identifiers to identify terms in the natural language output statements.

21. The one or more non-transitory computer storage medium of claim 18, the operations further comprising:

providing non-natural language output statements corresponding to the natural language output statements.

22. The one or more non-transitory computer storage of claim 18, wherein:

the natural language front end includes a conversion database that includes a respective application schema for each of one or more of the software applications, wherein each application schema defines mappings between (i) terms in the natural language queries and the natural language update statements received from the software application corresponding to the respective application schema, and (ii) terms included in the data schema of the knowledge base; and

the conversion database includes the data schema of the knowledge base.

23. The one or more non-transitory computer storage of claim 18, wherein:

the knowledge base is a SQL database; and

the structured API operations are SQL statements.