Method and apparatus for annotating a graphical output

• US 10,839,580 B2
• Filed: 03/27/2019
• Issued: 11/17/2020
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: Arria Data2text Limited
• Original Assignee: Arria Data2text Limited
• Inventors: Reiter, Ehud Baruch
• Primary Examiner(s): Stork, Kyle R
• Application Number:
  US16/367,095
• Publication Number:
  US 20200058145A1
• Time in Patent Office: 601 Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof

Abstract

Various methods are provided for generating and annotating a graph. One example method may include determining one or more key patterns in a primary data channel, wherein the primary data channel is derived from raw input data in response to a constraint being satisfied. A method may further include determining one or more significant patterns in one or more related data channels. A method may further include generating a natural language annotation for at least one of the one or more key patterns or the one or more significant patterns. A method may further include generating a graph that is configured to be displayed in a user interface, the graph having at least a portion of the one or more key patterns, the one or more significant patterns and the natural language annotation.

Claims

1. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

generate at least one message when a pattern from one or more patterns in a data channel or a pattern from one or more patterns in another data channel satisfies one or more message requirements, wherein the one or more patterns in the data channel and the one or more patterns in the another data channel are derived from raw input data;

select one or more words to express a concept or a relation in the at least one message;

apply a grammar to the selected one or more words to generate one or more phrases;

generate a graphical output for display in a user interface based on the data channel, the another data channel, context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel, and the one or more phrases; and

display, via the user interface, the generated graphical output and a narrative that linguistically describes the graphical output.

2. The apparatus of claim 1, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

3. The apparatus of claim 1, wherein the narrative is configured to be displayed separately in the user interface from the one or more phrases.

4. The apparatus of claim 1, wherein the one or more patterns is at least one of a trend, spike, or step in the data channel.

5. The apparatus of claim 1, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

assign an importance level to the one or more patterns; and

annotate the one or more patterns as one or more key patterns when the importance level of the one or more patterns exceeds a predefined importance level.

6. The apparatus of claim 5, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

generate a graph displaying a time period, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed; and

display, via the user interface, the graph with the generated graphical output and the narrative that linguistically describes the graphical output.

7. A computer program product comprising at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus causes the apparatus to:

generate at least one message when a pattern from one or more patterns in a data channel or a pattern from one or more patterns in another data channel satisfies one or more message requirements, wherein the one or more patterns in the data channel and the one or more patterns in the another data channel are derived from raw input data;

select one or more words to express a concept or a relation in the at least one message;

apply a grammar to the selected one or more words to generate one or more phrases;

generate a graphical output for display in a user interface based on the data channel, the another data channel, context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel, and the one or more phrases; and

display, via the user interface, the generated graphical output and a narrative that linguistically describes the graphical output.

8. The computer program product of claim 7, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

9. The computer program product of claim 7, wherein the narrative is configured to be displayed separately in the user interface from the one or more phrases.

10. The computer program product of claim 7, wherein the one or more patterns is at least one of a trend, spike, or step in the data channel.

11. The computer program product of claim 7, wherein the at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus further causes the apparatus to:

assign an importance level to the one or more patterns; and

annotate the one or more patterns as one or more key patterns when the importance level of the one or more patterns exceeds a predefined importance level.

12. The computer program product of claim 11, wherein the at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus further causes the apparatus at least to:

generate a graph displaying a time period, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed; and

display, via the user interface, the graph with the generated graphical output and the narrative that linguistically describes the graphical output.

13. A computer-implemented method for transforming raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the method comprising:

generating, using a processor, at least one message when a pattern from one or more patterns in a data channel or a pattern from one or more patterns in another data channel satisfies one or more message requirements, wherein the one or more patterns in the data channel and the one or more patterns in the another data channel are derived from raw input data;

selecting, using the processor, one or more words to express a concept or a relation in the at least one message;

applying, using the processor, a grammar to the selected one or more words to generate one or more phrases;

generating, using the processor, a graphical output for display in a user interface based on the data channel, the another data channel, context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel, and the one or more phrases; and

displaying, via the user interface and using the processor, the generated graphical output and a narrative that linguistically describes the graphical output.

14. The method of claim 13, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

15. The method of claim 13, wherein the narrative is configured to be displayed separately in the user interface from the one or more phrases.

16. The method of claim 13, wherein the one or more patterns is at least one of a trend, spike, or step in the data channel.

17. The method of claim 13, further comprising:

assigning, using the processor, an importance level to the one or more patterns; and

annotating, using the processor, the one or more patterns as one or more key patterns when the importance level of the one or more patterns exceeds a predefined importance level.

18. The method according to claim 17, further comprising:

generating, using the processor, a graph displaying a time period, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed; and

displaying, via the user interface and using the processor, the graph with the generated graphical output and the narrative that linguistically describes the graphical output.