QA based on context aware, real-time information from mobile devices

• US 10,387,409 B2
• Filed: 06/06/2013
• Issued: 08/20/2019
• Est. Priority Date: 06/06/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Calo, Seraphin Bernard, Freimuth, Douglas M, Ganti, Raghu Kiran, Fan, James J., Ye, Fan
• Primary Examiner(s): Hu, Jensen
• Application Number:
  US13/912,147
• Publication Number:
  US 20140365518A1
• Time in Patent Office: 2,266 Days
• Field of Search: 707760
• CPC Class Codes: G06F 16/243 Natural language query formulation

Abstract

A common infrastructure collects data from a plurality of mobile devices and traditional sensors at Internet scale to respond to natural language queries received at different applications. The infrastructure includes a semantic interpreter to translate the natural language query to a data request specification that is processed by the data collection system. The data collection system includes a phenomenon layer that expresses data and information needs in a declarative fashion and coordinates data collection and processing for queries. An edge layer manages devices, receives collection requirements from the backend layer, configures and instructs devices for data collection, and conducts aggregation and primitive processing of data. This layer contains network edge nodes, such as base stations in a cellular network. Each node manages a set of local data generating networked devices. The device agent data layer using common agents on the networked devices receives data collection instructions and performs data collection.

Claims

1. A question answering system comprising:

an application layer comprising a plurality of applications running on a plurality of computing systems, each application configured to generate a natural language query describing at least one phenomenon comprising an event occurring in real time at a given physical location within a time window that extends over a given duration of real time;

mobile edge capture and analytics middleware in communication with the plurality of applications running on the plurality of computing systems, the plurality of applications communicating the plurality of natural language queries to common mobile edge capture and analytics middleware and the mobile edge capture and analytics middleware comprising;

a semantic interpreter in communication with the applications and configured to translate each natural language query into a data request specification, each data request specification comprising an identification of types of data to be obtained that are responsive to the natural language query and that indicate an occurrence of the at least one phenomenon;

a phenomenon layer configured to receive the data request specification and to generate data collection requirements; and

an edge layer in communication with the phenomenon layer and configured to receive the data collection requirements and to identify raw data required to satisfy the data collection requirements, the raw data comprising time data within the time window, physical location data and physical sensor data, the mobile edge capture and analytics middleware combining the time data, physical location data and physical sensor data to identify occurrences of phenomena indicated by the raw data and storing the time data, physical location data, physical sensor data and identified phenomena for use in responding to duplicate data collection requirements;

a plurality of data generating network devices disposed at the physical location during the time window, each data generating networked device comprising physical sensors; and

a plurality of instances of a common software agent, each instance of the common software agent executing on one of the plurality of data generating networked devices and in communication with the mobile edge capture and analytics middleware, instances of the common software agent obtaining the identified raw data required to satisfy the data collection requirements from the physical sensors disposed in the given physical location during the given time duration and communicating the raw data to the mobile edge capture and analytic middleware.

2. The system of claim 1, wherein the semantic interpreter is disposed in the phenomenon layer.

3. The system of claim 1, wherein the semantic interpreter is further configured to:

obtain profile information for an entity using the application to submit the natural language query; and

use the obtained profile information in identifying the types of data to be obtained that are responsive to the natural language query.

4. The system of claim 1, wherein the semantic interpreter comprises a plurality of subject matter domain templates, each subject matter domain template comprising:

a pre-define list of types of data relevant to a given subject matter domain; and

a set of parameters for each type of data in the pre-defined list of types of data, the set of parameters comprising adjustable qualities and quantities for the types of data.

5. The system of claim 4, wherein the semantic interpreter is further configured to:

identify a subject matter domain associated with the natural language query;

identify a subject matter domain template from the plurality of subject matter domain templates corresponding to the subject matter domain associated with the natural language query; and

use the corresponding subject matter domain template to create the data request specification.

6. The system of claim 4, wherein the semantic interpreter is further configured to:

receive inputs from the application regarding preferred qualities and quantities for each type of data; and

adjust the qualities and quantities of each type of data in accordance with the received inputs.

7. The system of claim 4, wherein the common software agents are further configured to obtain the identified raw data in accordance with the adjusted qualities and quantities of each type of data.

8. The system of claim 4, wherein the semantic interpreter further comprises a plurality of data collection policies, each data collection policy comprising a predetermined constraint on data collection.

9. The system of claim 8, wherein the common software agents are further configured to obtain the identified raw data in accordance with the identified data collection policies.

10. The system of claim 8, wherein each data collection policy is associated with one of the subject matter domains.

11. The system of claim 8, wherein the data collection policies comprise limitations on the data generating networked devices that can be used to obtain the raw data, temporal limitations on data collection and spatial limitations on data collection.

12. The system of claim 1, wherein the semantic interpreter further comprises a knowledge based expert system comprising a plurality of subject matter domain experts, each subject matter domain expert corresponding to a given subject matter domain and configured to create the data request specification for natural language queries encompassing that subject matter domain.

13. The system of claim 1, wherein:

the system further comprises an ontology builder in communication with the semantic interpreter, the ontology builder configured to specify relationships among concepts and attributes of those concepts; and

the semantic interpreter is further configured to;

identify a concept in the natural language query, the concept comprising at least a portion of the natural language query;

use the ontology builder to build an ontology corresponding to the concept; and

use the ontology to describe the data request specification.

14. The system of claim 13, wherein the concept comprises a physical object or a physical phenomenon.

15. The system of claim 13, wherein the semantic interpreter is further configured to:

use the ontology to identify attributes of the concept; and

identify the types of data available to obtain information about the identified attributes.

16. The system of claim 15, wherein:

the edge layer is further configured to identify information about the identified attributes that has not been obtained; and

the common software agents are further configured to use the identified types of data to obtain the raw data corresponding to the identified information about the identified attributes that has not been obtained.

17. The system of claim 1, wherein the system further comprises a raw data translator in communication with the edge layer and the application and configured to:

process the raw data and identified occurrences of phenomena into natural language responses; and

communicate the natural language responses to the application.

18. The system of claim 1, wherein the data collection requirements comprise raw data and phenomena.

19. The system of claim 1, wherein the phenomenon layer comprises:

a collection task manager configured to provide an interface for applications running on the computing system to submit the natural language query, to provide an interface for the semantic interpreter to submit the data request specifications and to forward the generated data collection requirements to the edge layer;

a backend metadata manager configured to maintain metadata regarding phenomena capable of being generated by different edge nodes at the edge layer and a physical location coverage associated with each edge node; and

a backend data manager configured to receive raw data obtained from the data generating networked devices and phenomena generated in the edge layer using the raw data, to aggregate the received raw data and phenomena and to communicate the raw data and phenomena to the applications that submit data request specifications.

20. The system of claim 19, wherein the collection task manager is further configured to maintain state information for data collection tasks initiated at the edge layer to satisfy the data collection requirements.