Providing answers to questions using multiple models to score candidate answers

• US 8,738,617 B2
• Filed: 09/23/2011
• Issued: 05/27/2014
• Est. Priority Date: 09/28/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Eric W., Ferrucci, David A., Murdock, James W. IV
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/242,447
• Publication Number:
  US 20120078891A1
• Time in Patent Office: 977 Days
• Field of Search: None
• US Class Current: 707/731
• CPC Class Codes: F16H 1/28 with gears having orbital motion
• CPC Class Codes: F16H 2200/0034 the gear ratios comprising two forward speeds
• CPC Class Codes: F16H 2200/0039 the gear ratios comprising three forward speeds
• CPC Class Codes: F16H 2200/2005 with one sets of orbital gears
• CPC Class Codes: F16H 2200/2035 with two engaging means
• CPC Class Codes: F16H 3/006 power being selectively transmitted by either one of the parallel flow paths
• CPC Class Codes: F16H 3/091 including a single countershaft
• CPC Class Codes: F16H 3/54 one of the central gears being internally toothed and the other externally toothed
• CPC Class Codes: F16H 37/02 comprising essentially only toothed or friction gearings
• CPC Class Codes: F16H 37/0813 with only one input shaft differentials for four wheel drive vehicles B60K17/346
• CPC Class Codes: F16H 57/035 Gearboxes for gearing with endless flexible members
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method, system and computer program product for generating answers to questions. In one embodiment, the method comprises receiving an input query; conducting a search to identify candidate answers to the input query, and producing a plurality of scores for each of the candidate answers. For each of the candidate answers, one, of a plurality of candidate ranking functions, is selected. This selected ranking function is applied to the each of the candidate answers to determine a ranking for the candidate answer based on the scores for that candidate answer. One or more of the candidate answers is selected, based on the rankings for the candidate answers, as one or more answers to the input query. In an embodiment, the ranking function selection is performed using information about the question. In an embodiment, the ranking function selection is performed using information about each answer.

Claims

1. A system for generating answers to questions, comprising:

a computer device comprising;

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory device storing a plurality of candidate scoring models; and

at least one processor coupled to the memory device and operative for;

receiving an input query;

using a candidate answer generation module to conduct a search, based on said query, in one or more data sources to identify candidate answers to the input query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring models based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring models, and using said selected one of the candidate scoring models to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query.

2. The system according to claim 1, wherein said selected one of the candidate scoring functions is selected based on the input query.

3. The system according to claim 2, wherein:

the conducting a search includes extracting defined information about the input query; and

the selected one of the candidate scoring functions is selected based on said defined information.

4. The system according to claim 1, wherein the selected one of the candidate scoring functions is selected based on said each candidate answer.

5. The system according to claim 4, wherein:

the producing a plurality of scores for each of the candidate answers includes extracting defined information about each of the candidate answers; and

the selected one of the candidate scoring functions is selected based on said extracted defined information.

6. The system according to claim 1, wherein the selecting one of the candidate scoring models includes:

determining a defined measure of a grammatical complexity of the input query; and

using the measure of the grammatical complexity of the input query to select one of the candidate scoring models to determine the second classification scores for all of the candidate answers identified for the input query.

7. An article of manufacture, comprising:

at least one tangible computer readable device having computer readable program code logic tangibly embodied therein to generate answers to questions, and for use with a memory device storing a plurality of candidate scoring models, the computer readable program code logic, when executing, performing the following;

receiving an input query;

using a candidate answer generation module to conduct a search, based on said query, in one or more data sources to identify candidate answers to the input query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring models based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring models, and using said selected one of the candidate scoring models to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query.

8. The article of manufacture according to claim 7, wherein said selected one of the candidate scoring functions is selected based on the input query.

9. The article of manufacture according to claim 7, wherein:

the conducting a search includes extracting defined information about the input query; and

the selected one of the candidate scoring functions is selected based on said defined information.

10. The article of manufacture according to claim 7, wherein the selected one of the candidate scoring functions is selected based on said each candidate answer.

11. The article of manufacture according to claim 10, wherein:

the producing a plurality of scores for each of the candidate answers includes extracting defined information about each of the candidate answers; and

the selected one of the candidate scoring functions is selected based on said extracted defined information.

12. A system for generating answers to questions, comprising:

a computer device comprising;

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory device storing a plurality of candidate scoring models; and

at least one processor coupled to the memory device and operative for;

receiving a plurality of input queries;

for each query, using a candidate answer generation module to conduct a search, based on said each query, in one or more data sources to identify candidate answers to said each query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring functions based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring functions, and using selected one of the candidate scoring functions to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query; and

wherein;

for the candidate answers to each of a first group of the input queries, the selecting one of the candidate scoring functions includes selecting one of the candidate scoring functions based on the input query; and

for the candidate answers to each of a second group of the input queries, the selecting one of the candidate ranking functions includes selecting one of the candidate ranking functions based on said each candidate answer.

13. The system according to claim 12, wherein:

defined information is extracted about the input query; and

the one of the candidate scoring functions selected based on the input query is selected based on said defined information;

defined information is extracted about each of the candidate answers; and

the one of the candidate scoring functions selected based on the candidate answers is selected based on said extracted defined information about the candidate answers.