NARRATION SYSTEM FOR INTERACTIVE DASHBOARDS

• US 20200134037A1
• Filed: 10/26/2018
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Abandoned Application

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Mankovskii, Serge, Velez-Rojas, Maria, Greenspan, Steven
• Application Number:
  US16/171,778
• Publication Number:
  US 20200134037A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/56 Natural language generation

Abstract

Provided is a process, including: receiving a command corresponding to user input to an interactive dashboard application from a user; producing in response to the command, instances of data visualizations depicting data to be visualized; generating, with a trained captioning model, one or more narrative captions determined to be descriptive of the produced instances of data visualizations, wherein the one or more narrative captions include a natural language description of a phenomenon exhibited, at least in part, by the data to be visualized and visually depicted in at least one of the produced instances of data visualizations; and causing the one or more narrative captions to be presented to the user.

Claims

1. A method, comprising:

receiving, with one or more processors, a command corresponding to user input to an interactive dashboard application from a user, wherein;

the interactive dashboard application is configured to present a plurality of instances of data visualizations in a dashboard user interface,

the dashboard user interface comprises user-interface input elements, and

the interactive dashboard application is configured to adjust, responsive to the input elements, which data visualizations are shown, attributes of the data visualizations, or which data is depicted in the interactive dashboard application;

producing, with one or more processors, in response to the command, instances of data visualizations depicting data to be visualized;

generating, with one or more processors, with a trained captioning model, one or more narrative captions determined to be descriptive of the produced instances of data visualizations, wherein the one or more narrative captions include a natural language description of a phenomenon exhibited, at least in part, by the data to be visualized and visually depicted in at least one of the produced instances of data visualizations; and

causing, with one or more processors, the one or more narrative captions to be presented to the user.

2. The method of claim 1, wherein:

the command is input via one or more of the user-interface input elements;

causing the one or more narrative captions to be presented to the user comprises causing a plurality of narrative captions to be presented to the user on a user computing device via a web browser executing on the user computing device;

the produced instances of data visualizations include graphs that depict at least some of the data to be visualized;

the one or more narrative captions include at least two of the following;

an indication that a limit or target mark is predicted to be attained by a displayed metric within a threshold duration of time;

an indication that a comparison of multiple metrics indicates a designated amount of difference or correlation between the multiple metrics;

a description of a comparisons between multiple values of a single displayed metric;

a description of convergence or divergence of two displayed metrics;

a description of change in displayed metrics over time;

a ranking of multiple displayed metrics or of a single displayed metric over time;

a result of a comparison between a displayed metric and a visually depicted threshold of a data visualization;

a result of a what-if analysis of displayed metrics;

a suggested responsive action to adjust a system characterized by a displayed metric;

or

an indication of correspondence to a statistical distribution.

3. The method of claim 1, wherein:

at least some of the one or more narrative captions are input and associated with corresponding data visualizations during design of the corresponding data visualizations via a dashboard design application that is the same application or a different application from the interactive dashboard application.

4. The method of claim 1, wherein:

the command selects a subset of data from a data source supplying the data to be visualized and requests an update to an existing dashboard user interface to depict the selected subset of data.

5. The method of claim 4, wherein:

at least some of the one or more narrative captions describes a change in a depicted metric resulting from the requested update.

6. The method of claim 5, wherein generating the one or more narrative captions comprises:

comparing values of, or visual attributes of a depiction of, the depicted metric before the update to values of, or visual attributes of the depiction of, the depicted metric after the update.

7. The method of claim 4, wherein generating one or more narrative captions comprises:

selecting the one or more narrative captions from among a plurality of narrative captions determined to apply to at least some of the produced instances of data visualizations, wherein the selection is based on;

a determination that the one or more narrative captions are caused to apply to at least some of the produced instances by the command, and

a determination that unselected narrative captions are not caused to apply to at least some of the produced instances by the command.

8. The method of claim 1, wherein:

the command specifies an envelope in a space of potential instances of data visualizations; and

generating the one or more narrative captions comprises;

simulating a plurality of the potential instances of data visualizations in the space;

determining candidate narrative captions corresponding to each of the potential instances of data visualizations; and

determining the one or more narrative captions based on at least some of the candidate narrative captions.

9. The method of claim 8, wherein:

simulating a plurality of the potential instances of data visualizations in the space comprises a brute-force search of every permutation of values of dimensions of at least part of the space specified by a sampling regime.

10. The method of claim 8, wherein:

simulating a plurality of the potential instances of data visualizations in the space comprises stochastically sampling the space.

11. The method of claim 8, wherein:

simulating a plurality of the potential instances of data visualizations in the space comprises traversing a plurality of paths through a decision tree having leaf nodes specifying points in the space and non-leaf nodes corresponding to responsive actions taken based on predicted values of displayed metrics.

12. The method of claim 8, wherein:

simulating a plurality of the potential instances of data visualizations in the space comprises sampling the space based on probability distributions of values of at least some dimensions of the space.

13. The method of claim 8, wherein simulating a plurality of the potential instances of data visualizations in the space comprises:

inputting a current or projected set of metrics among the data to be visualized into a predictive model configured to output predicted values of at least some of the data to visualized; and

configuring the potential instances of data visualizations based on predicted values output by the predictive model.

14. The method of claim 1, wherein generating one or more narrative captions comprises:

inputting a plurality of intermediate narrative captions determined to be descriptive of produced instances of the data visualizations into a natural-language-processing text-summarization model that generates text of the one or more narrative captions.

15. The method of claim 14, wherein the text summarization model is an abstractive-text-summarization recurrent neural network configured to generate phrases absent from the intermediate narrative captions.

16. The method of claim 14, wherein:

generating one or more narrative captions comprises classifying the intermediate narrative captions according to an ontology of narrative captions; and

causing the one or more narrative captions to be presented comprises causing the intermediate narrative captions to be presented in visual association with an indication of results of the classifying.

17. The method of claim 8, comprising:

selecting a subset of the plurality of the potential instances of data visualizations based on entropy of corresponding candidate narrative captions; and

causing candidate narrative captions of the selected subset to be presented to the user in a guided tour of potential scenarios of a system characterized by the data to be visualized.

18. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:

receiving, with one or more processors, a command corresponding to user input to an interactive dashboard application from a user, wherein;

the interactive dashboard application is configures to present a plurality of instances of data visualizations in a dashboard user interface,

the dashboard user interface comprises user-interface input elements, and

the interactive dashboard application is configured to adjust, responsive to the input elements, which data visualizations are shown, attributes of the data visualizations, or which data is depicted in the interactive dashboard application;

producing, with one or more processors, in response to the command, instances of data visualizations depicting data to be visualized;

generating, with one or more processors, with a trained captioning model, one or more narrative captions determined to be descriptive of the produced instances of data visualizations, wherein the one or more narrative captions include a natural language description of a phenomenon exhibited, at least in part, by the data to be visualized and visually depicted in at least one of the produced instances of data visualizations; and

causing, with one or more processors, the one or more narrative captions to be presented to the user.

19. The medium of claim 18, wherein:

the command selects a subset of data from a data source supplying the data to be visualized and requests an update to an existing dashboard user interface to depict the selected subset of data.

20. The medium of claim 18, wherein:

the command specifies an envelope in a space of potential instances of data visualizations; and

generating the one or more narrative captions comprises;

simulating a plurality of the potential instances of data visualizations in the space;

determining candidate narrative captions corresponding to each of the potential instances of data visualizations; and

determining the one or more narrative captions based on at least some of the candidate narrative captions.