×

Evaluating conversation data based on risk factors

  • US 10,545,648 B2
  • Filed: 01/08/2019
  • Issued: 01/28/2020
  • Est. Priority Date: 09/09/2014
  • Status: Active Grant
First Claim
Patent Images

1. One or more non-transitory computer-readable media storing computer-readable instructions that, when executed, instruct one or more processors to perform acts comprising:

  • causing a conversation user interface to be displayed on a display device to enable a conversation between a user and a virtual assistant;

    receiving, via the conversation user interface, user input;

    processing the user input with one or more natural language processing techniques to identify an intent unit for the user input, the intent unit being associated with a language model for the natural language processing techniques;

    determining, based at least in part on a presence of one or more risk indicators for the user input, a measure of confidence that the intent unit is correctly identified for the user input;

    associating the measure of confidence with the user input;

    determining a health status of the intent unit based at least in part on the measure of confidence, the health status indicating a level of risk associated with the intent unit;

    presenting the health status of the intent unit via an output device associated with an administrator;

    receiving a selection of the intent unit to obtain feedback regarding the intent unit;

    presenting a feedback interface that enables a voter to provide feedback regarding matching of the user input to the intent unit;

    receiving feedback for the voter regarding an accuracy of matching the user input to the intent unit; and

    evaluating the intent unit based at least in part on the feedback;

    determining that the feedback indicates that the matching of the user input to the intent unit is not accurate;

    increasing a weighting to be applied to the one or more risk indicators based at least in part on the determining that the feedback indicates that the matching of the user input to the intent unit is not accurate; and

    applying the weighting to the one or more risk indicators;

    wherein the determining the measure of confidence includes determining the measure of confidence based at least in part on the presence of the one or more weighted risk indicators.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×