Method and apparatus for annotating a graphical output

• US 9,405,448 B2
• Filed: 02/27/2015
• Issued: 08/02/2016
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: Arria Data2text Limited
• Original Assignee: Arria Data2text Limited
• Inventors: Reiter, Ehud Baruch
• Primary Examiner(s): STORK, KYLE R
• Application Number:
  US14/634,035
• Publication Number:
  US 20150242384A1
• Time in Patent Office: 522 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof

Abstract

Various methods are provided for generating and annotating a graph. One example method may include determining one or more key patterns in a primary data channel, wherein the primary data channel is derived from raw input data in response to a constraint being satisfied. A method may further include determining one or more significant patterns in one or more related data channels. A method may further include generating a natural language annotation for at least one of the one or more key patterns or the one or more significant patterns. A method may further include generating a graph that is configured to be displayed in a user interface, the graph having at least a portion of the one or more key patterns, the one or more significant patterns and the natural language annotation.

Claims

1. A method for transforming raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the method comprising:

detecting one or more patterns in a data channel derived from raw input data;

identifying one or more patterns in another data channel also derived from the raw input data;

determining from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generating, using a natural language generation system that is configured to execute on a processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel wherein the one or more phrases are generated by;

generating at least one message in an instance in which a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements;

selecting one or more words to express at least one of a concept or relation in the at least one message;

applying a grammar to the selected one or more words; and

generating a graphical output for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the generated graphical output of the data channel and the another data channel;

generating a narrative that linguistically describes the graphical output, wherein the narrative is configured to be displayed separately in the user interface from the one or more phrases; and

displaying, via a user interface, the generated graphical output, the one or more phrases and the narrative, wherein in response to a user interaction with the one or more phrases, additional text related to the one or more phrases is identified in the narrative via the user interface.

2. A method according to claim 1, further comprising:

detecting one or more patterns in the data channel by;

identifying one or more patterns wherein a pattern is at least one of a trend, spike or step in the data channel;

assigning an importance level to the one or more patterns; and

identifying one or more key patterns of the one or more patterns, wherein a key pattern is a pattern that exceeds a predefined importance level.

3. A method according to claim 2, further comprising:

determining a time period to be displayed in a graph, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed.

4. A method according to claim 3, further comprising:

identifying one or more patterns in the another data channel by;

assigning an importance level to one or more unexpected patterns; and

identifying one or more significant patterns of the one or more unexpected patterns, wherein a significant pattern is an unexpected pattern.

5. A method according to claim 4, further comprising:

determining that the one or more patterns identified in an another data channel violate a predetermined constraint; and

determining that the one or more patterns are one or more unexpected patterns.

6. A method according to claim 4, wherein the one or more unexpected patterns are identified within the time period of the graph.

7. A method according to claim 4, further comprising:

generating the graph based on the graphical output, wherein the graph comprises at least a portion of the data channel that contains the one or more key events, the another data channel that contains the one or more significant events and the one or more contextual channels, the at least a portion of the one or more key events, the one or more significant events and the one or more contextual channels being annotated by the one or more phrases.

8. A method according to claim 1, wherein the data channel is selected from one or more data channels derived from the raw input data, the data channel is selected based on an indication received from at least one of a user, an alarm or another computing device.

9. A method according to claim 7, wherein the one or more key patterns and the one or more significant patterns are highlighted in response to the user interaction with the one or more phrases in the user interface.

10. A method according to claim 1, wherein the context information provides at least background or circumstance information having influenced the one or more patterns in the data channel or the one or more patterns in the another data channel.

17. A method according to claim 1, further comprising:

receiving an indication of a determined scale from a user via a user interface; and

generating the graphical output based on the data channel, the another data channel and the one or more phrases in accordance with the determined scale.

18. A method according to claim 1, further comprising:

receiving a selection of one or more of the primary data channel, the another data channel, or the one or more contextual channels from a user via a user interface; and

generating the graphical output based on the selection of the one or more of the primary data channel, the another data channel, or the one or more contextual channels and the natural language annotation for the selection.

11. An apparatus that is configured to transform raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least;

determine one or more key patterns in a primary data channel, wherein the primary data channel is derived from raw input data in response to a constraint being satisfied;

determine one or more significant patterns in one or more related data channels;

determining from one or more contextual channels context information for at least one of the one or more key patterns in the primary data channel or the one or more significant patterns in the one or more related data channels;

generate a natural language annotation for at least one of the one or more key patterns or the one or more significant patterns wherein the natural language annotation is generated based on at least one message that is generated as a result of at least one of the one or more key patterns or the one or more significant patterns, a selection of one or more words to express at least one of a concept or relation in the at least one message, and an application of a grammar;

generate a graph to be displayed in a user interface, the graph visually presenting at least a portion of the primary data channel having one or more key patterns, at least a portion of the another data channel having the one or more significant patterns, the one or more contextual channels having context information for at least one of the one or more key patterns in the primary data channel or the one or more significant patterns in the one or more related data channels, and the natural language annotation;

generate a narrative that linguistically describes the graphical output, wherein the narrative is configured to be displayed separately in the user interface from the natural language annotation; and

display the generated graph, the natural language annotation, and the narrative, wherein in response to a user interaction with the natural language annotation, additional text related to the natural language annotation is identified in the narrative via the user interface.

12. An apparatus according to claim 11, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

identify one or more patterns in the primary data channel;

assign an importance level to the one or more patterns; and

identify one or more key patterns of the one or more patterns in an instance in which the importance level of a pattern of the one or more patterns exceeds a threshold defined by a domain model.

13. An apparatus according to claim 12, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine a time period to be represented by the graphical output, wherein the time period is configured such that at least a portion of the one or more key patterns occur within the time period.

14. An apparatus according to claim 12, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine one or more patterns in one or more related data channels that corresponds to an occurrence of the one or more key patterns; and

determine one or more significant patterns of the one or more patterns, wherein a significant pattern is a pattern that represents an anomaly.

15. An apparatus according to claim 11, wherein the context information provides at least background or circumstance information having influenced the one or more key patterns in the primary data channel or the one or more significant patterns in the one or more related data channels.

16. A computer program product that is configured to transform raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the computer program product comprising:

at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus causes the apparatus at least to;

identify one or more key patterns in a primary data channel, wherein the one or more key patterns have an importance level that exceeds a predefined importance level;

identify context information in a contextual channel applied at the time of the one or more key patterns in a primary data channel;

generate a natural language annotation of at least a portion of the one or more key patterns, wherein the natural language annotation is generated based on at least one message that is generated as a result of at least one of the one or more key patterns, a selection of one or more words to express at least one of a concept or relation in the at least one message, and an application of a grammar;

generate a graph for display in a user interface, the graph having at least a portion of the primary data channel having one or more key patterns, the contextual channel having context information applied at the time of the one or more key patterns in a primary data channel, and the natural language annotation;

generate a narrative that linguistically describes the graphical output, wherein the narrative is configured to be displayed separately in the user interface from the natural language annotation; and

display the generated graph, the natural language annotation, and the narrative, wherein in response to a user interaction with the natural language annotation, additional text related to the natural language annotation is identified in the narrative via the user interface.

19. A computer program product according to claim 16, wherein the context information provides at least background or circumstance information having influenced the one or more key patterns in a primary data channel.