System and method for cognitive intervention on human interactions

• US 10,373,515 B2
• Filed: 01/04/2017
• Issued: 08/06/2019
• Est. Priority Date: 01/04/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Corelli, Helena Morgado, Fernandes, Rodrigo Braga, Guimaraes, Rodrigo Laiola, Matos, Rafael Sorana de, Nogima, Julio, Silva, Rogerio Cesar Barbosa dos Santos, Vital, Thiago Luiz de Barcelos
• Primary Examiner(s): D'Agostino, Paul A
• Application Number:
  US15/397,996
• Publication Number:
  US 20180190143A1
• Time in Patent Office: 944 Days
• Field of Search: None
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

A determination regarding whether to intervene in a dialog to provide system-initiated assistive information involves monitoring a dialog between at least two participants and capturing data from a dialog environment containing at least one of the participants. The captured data represent the content of the dialog and physiological data for one or more participants. Assistive information relevant to the dialog and participants is identified, and the captured data are used to determine an intervention index of delivering the assistive information to one or more participants during the dialog. This intervention index is then used to determine whether or not to intervene in the dialog to deliver the assistive information to one or more participants.

Claims

1. A method for intervening in a dialog to provide system-initiated assistive information, the method comprising:

monitoring a dialog between at least two participants using a separate cognitive entity as a passive monitoring participant in the dialog;

capturing data from a dialog environment containing at least one of the participants, the captured data comprising at least one of content of the dialog and physiological data for one or more participants;

identifying assistive information relevant to the dialog and participants;

using the captured data to determine an intervention index for delivering the assistive information to one or more participants during the dialog; and

intervening in the dialog to deliver the assistive information to one or more participants based on the intervention index;

wherein the cognitive entity identifies the assistive information, determines the intervention index and intervenes in the dialog in real-time based on the monitored content.

2. The method of claim 1, wherein the method further comprises using data capture devices disposed in the dialog environment to monitor the dialog and to capture data.

3. The method of claim 2, wherein the data capture devices comprise still cameras, video cameras, microphones, telephone equipment, telephone conferencing equipment, video conference equipment, telephones, cellular telephones, smartphones, tablet computers, electronic mail applications, text message applications, thermostats, thermometers, motion sensors, physiological sensors, activity sensors, biomedical sensors or combinations thereof.

4. The method of claim 1, wherein identifying assistive information comprises using the captured data to identify assistive information.

5. The method of claim 4, wherein using the captured data to identify assistive information comprises using the captured data in at least one of static searches and dynamic searches over fixed databases and continuous data streams.

6. The method of claim 1, wherein the method further comprises using the captured data to identify a physical or emotional condition of at least one participant.

7. The method of claim 1, wherein using the captured information to determine the intervention index comprises:

determining a value of the assistive information to the dialog and participants in the dialog; and

determining an intervention cost on the dialog caused by intervening in the dialog to communicate the assistive information.

8. The method of claim 7, wherein:

using the captured information to determine the intervention index further comprises comparing the value of the assistive information to the intervention cost on the dialog; and

intervening in the dialog further comprises intervening in the dialog if the value of the assistive information is sufficient to justify the intervention cost on the dialog.

9. The method of claim 1, wherein:

the method further comprises determining intervention parameters communicating the assistive information to the one or more participants; and

intervening in the dialog further comprising intervening in the dialog in accordance with the intervention parameters.

10. The method of claim 9, wherein determining the intervention parameters comprises at least one of:

identifying target participants in the dialog to receive the assistive information;

identifying an appropriate time during the dialog for intervening in the dialog that minimizes interference with the participants; and

identifying intervention mechanisms associated with the target participants that can be used to deliver the assistive information to the target participants.

11. The method of claim 10, wherein the intervention mechanisms comprise speakers, video monitors, lights, haptic systems, kinesthetic systems, graphical user interfaces, smart phones, tablet computers, telephones, cellular telephones or combinations thereof.

12. The method of claim 1, wherein the method further comprises:

determining post-intervention impacts on the dialog resulting from intervening in the dialog and delivering the assistive information; and

using the post-intervention impacts to determine intervention index of delivering additional assistive information to one or more participants at a later time during the dialog.

13. The method of claim 12, wherein determining post-intervention impacts comprises analyzing data captured from the dialog and participants after intervening in the dialog to determine the post-intervention impacts of intervening in the dialog to deliver the assistive information.

14. The method of claim 12, wherein using the post-intervention impacts comprises:

associating the post-intervention impacts with at least one of the dialog, a portion of the dialog, a type of dialog, the assistive information, a category of assistive information, a delivery mechanism used to communicate the assistive information and one or more participants; and

storing the post-intervention impacts and associations for use in determining the intervention index of delivering additional assistive information to one or more participants at the later time during the dialog.

15. A method for intervening in a dialog to provide system-initiated assistive information, the method comprising:

monitoring a dialog between at least two participants using a separate cognitive entity as a passive monitoring participant in the dialog;

capturing data from a dialog environment containing at least one of the participants, the captured data comprising at least one of content of the dialog and physiological data for one or more participants;

identifying assistive information relevant to the dialog and participants; and

using the captured data to determine an intervention index for delivering the assistive information to one or more participants during the dialog by;

determining a value of the assistive information to the dialog and participants in the dialog; and

determining an intervention cost on the dialog caused by intervening in the dialog to communicate the assistive information;

wherein the cognitive entity identifies the assistive information, determines the intervention index and intervenes in the dialog in real-time based on the monitored content.

16. The method of claim 15, wherein:

using the captured information to determine the intervention index further comprises comparing the value of the assistive information to the intervention cost on the dialog; and

the method further comprises intervening in the dialog if the value of the assistive information is sufficient to justify the intervention cost on the dialog.

17. The method of claim 15, wherein determining the value of the assistive information comprises:

assigning a low value to assistive information comprising a system-generated suggestion;

assigning a high value to assistive information comprising a resolution to a conflict occurring among the participants during the dialog; and

assigning a medium value between the low value and the high value to assistive information comprising an answer to a participant generated question.

18. The method of claim 15, wherein determining the intervention cost on the dialog comprises:

assigning a low intervention cost level when intervening in the dialog elicits a positive reaction from one or more participants;

assigning a high intervention cost level when intervening in the dialog elicits a negative reaction from one or more participants; and

assigning a medium intervention cost level between the low intervention cost level and the high intervention cost level when intervening in the dialog elicits a neutral reaction from one or more participants.

19. A computer-readable storage medium containing a computer-readable code that when read by a computer causes the computer to perform a method for intervening in a dialog to provide system-initiated assistive information, the method comprising using a separate cognitive entity as a passive monitoring participant in the dialog:

monitoring a dialog between at least two participants;

capturing data from a dialog environment containing at least one of the participants, the captured data comprising at least one of content of the dialog and physiological data for one or more participants;

identifying assistive information relevant to the dialog and participants;

using the captured data to determine an intervention index of delivering the assistive information to one or more participants during the dialog; and

intervening in the dialog to deliver the assistive information to one or more participants based on the intervention index;

wherein the cognitive entity identifies the assistive information, determines the intervention index and intervenes in the dialog in real-time based on the monitored content.

20. The computer-readable medium of claim 19, wherein:

using the captured information to determine the intervention index comprises;

determining a value of the assistive information to the dialog and participants in the dialog;

determining an intervention cost on the dialog caused by intervening in the dialog to communicate the assistive information; and

comparing the value of the assistive information to the intervention cost on the dialog; and

intervening in the dialog further comprises intervening in the dialog if the value of the assistive information is sufficient to justify the intervention cost on the dialog.