Method and system for quantitatively evaluating the confidence in information received from a user based on cognitive behavior

• US 9,846,844 B2
• Filed: 11/25/2015
• Issued: 12/19/2017
• Est. Priority Date: 11/25/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hecht, Jae-Lyn, Kozloski, James R., Lu, Fang, Salehi, Azadeh
• Primary Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US14/951,636
• Publication Number:
  US 20170147934A1
• Time in Patent Office: 755 Days
• Field of Search: 706 12, 706 45, 706 62
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A computer-implemented method of evaluating information confidence based on a cognitive trait of a user, is provided, the method including receiving information from a user as an answer to a question in an active learning question and answer system; monitoring a user for a cognitive behavior indicator when the user is providing the information; determining a cognitive trait based on the cognitive behavior indicator; determining a quantified level of information confidence for the information based on the cognitive trait; and annotating the information with the cognitive trait and the quantified level of information confidence. A system and a computer program product for evaluating information confidence based on a cognitive behavior of a user are also provided.

Claims

1. A computer-implemented method of evaluating information confidence based on cognitive behavior indicators of a user, the method comprising:

determining, by a processor, a system generated answer to a question in an active learning question and answer system, wherein the system generated answer includes a first answer confidence score for the system generated answer;

querying a user for an answer to the question in the active learning question and answer system based on the first answer confidence score;

receiving, by the processor, information from the user as an answer to the question in the active learning question and answer system;

monitoring, by the processor, the user for one or more cognitive behavior indicators when the user is providing the information, wherein the one or more cognitive behavior indicators comprise one or more biometric measurements of the user at a time the information is provided by the user and measures of an influence of an external stimuli on the user at the time the information is provided by the user;

determining, by the processor, a second answer confidence score for the information received from the user based on the one or more biometric measurements of the user and the measures of the influence of an external stimuli on the user;

annotating, by the processor, the information with the cognitive behavior indicators, a user profile associated with the user, and the second answer confidence score; and

updating the first answer confidence score based at least in part on the second answer confidence score.

2. The computer-implemented method of claim 1, wherein the cognitive behavior indicator comprises a physiological measurement.

3. The computer-implemented method of claim 1, wherein the cognitive behavior indicator comprises one or more of user eye tracking, user typing speed, user heart rate, user breathing rate, user blink frequency, user skin conductance, electroencephalogram information, user body temperature, user perspiration quantity, user blood pressure and user amount of time elapsed during information input.

4. The computer-implemented method of claim 1, further comprising annotating the information, the cognitive trait and the second answer confidence score for the information received from the user with information received from cognitive behavior traits of other users.

5. The computer-implemented method of claim 1, further comprising excluding information from the user if the quantified level of the information confidence is below a designated threshold.

6. The computer-implemented method of claim 1, further comprising ranking results of the information and cognitive traits of multiple users based on the quantified level of information confidence.

7. The computer-implemented method of claim 1, wherein the cognitive behavior indicator is measured by a user-wearable device.

8. The computer-implemented method of claim 1, further comprising annotating the information with user characteristics stored in a user profile.

9. A system for evaluating information confidence based on cognitive behavior indicators of a user, the system comprising:

a sensor to detect a cognitive behavior indicator of a user;

a memory;

a processor communicatively coupled to the memory and the sensor, where the processor is configured to perform;

determine a system generated answer to a question in an active learning question and answer system, wherein the system generated answer includes a first answer confidence score for the system generated answer;

querying a user for an answer to the question in the active learning question and answer system based on the first answer confidence score;

receiving information from the user as an answer to a question in an active learning question and answer system;

monitoring the user for one or more cognitive behavior indicator detected by the sensor when the user is providing the information, wherein the one or more cognitive behavior indicators comprise one or more biometric measurements of the user at a time the information is provided by the user and measures of an influence of an external stimuli on the user at the time the information is provided by the user;

determining a second answer confidence score for the information received from the user based on the one or more biometric measurements fo the user and the measures of the influence of an external stimuli on the user; and

annotating the information with the cognitive behavior indicators, a user profile associated with the user, and the second answer confidence score; and

update the first answer confidence score based at least in part on the second answer confidence score.

10. The system of claim 9, wherein the cognitive behavior indicator comprises a physiological measurement.

11. The system of claim 9, wherein the cognitive behavior indicator comprises one or more of user eye tracking, user typing speed, user heart rate, user breathing rate, user blink frequency, user skin conductance, electroencephalogram information, user body temperature, user perspiration quantity, user blood pressure and user amount of time elapsed during information input.

12. The system of claim 9, wherein the processor is further configured to perform annotating the information, the cognitive trait and the second answer confidence score for the information received from the user with information received from cognitive behavior traits of other users.

13. A computer program product for evaluating information confidence based on cognitive behavior indicators of a user, the computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising;

determining a system generated answer to a question in an active learning question and answer system, wherein the system generated answer includes a first answer confidence score for the system generated answer;

querying a user for an answer to the question in the active learning question and answer system based on the first answer confidence score;

receiving information from the user as an answer to the question in the active learning question and answer system;

monitoring the user for one or more cognitive behavior indicators detected by a sensor when the user is providing the information, wherein the one or more cognitive behavior indicators comprise one or more biometric measurements of the user at a time the information is provided by the user and measures of an influence of an external stimuli on the user at the time the information is provided by the user;

determining a second answer confidence score for the information received from the user based on the one or more biometric measurements of the user and the measures of external stimuli on the user;

annotating the information with the cognitive behavior indicators, a user profile associated with the user, and the second answer confidence score; and

updating the first answer confidence score based at least in part on the second answer confidence score.

14. The computer program product of claim 13, wherein the cognitive behavior indicator comprises a physiological measurement.

15. The computer program product of claim 13, wherein the cognitive behavior indicator comprises one or more of user eye tracking, user typing speed, user heart rate, user breathing rate, user blink frequency, user skin conductance, electroencephalogram information, user body temperature, user perspiration quantity, user blood pressure and user amount of time elapsed during information.

16. The computer program product of claim 13, wherein the computer readable program code is further configured to perform annotating the information, the cognitive trait and the second answer confidence score for the information received from the user with information received from cognitive behavior traits of other users.

17. The computer program product of claim 13, wherein the computer readable program code is further configured to perform excluding information from the user if the quantified level of the information confidence is below a designated threshold.