Method and apparatus for annotating a graphical output

• US 10,282,878 B2
• Filed: 06/21/2016
• Issued: 05/07/2019
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: Arria Data2text Limited
• Original Assignee: Arria Data2text Limited
• Inventors: Reiter, Ehud Baruch
• Primary Examiner(s): Stork, Kyle R
• Application Number:
  US15/188,423
• Publication Number:
  US 20170018107A1
• Time in Patent Office: 1,050 Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof

Abstract

Various methods are provided for generating and annotating a graph. One example method may include determining one or more key patterns in a primary data channel, wherein the primary data channel is derived from raw input data in response to a constraint being satisfied. A method may further include determining one or more significant patterns in one or more related data channels. A method may further include generating a natural language annotation for at least one of the one or more key patterns or the one or more significant patterns. A method may further include generating a graph that is configured to be displayed in a user interface, the graph having at least a portion of the one or more key patterns, the one or more significant patterns and the natural language annotation.

Claims

1. A computer-implemented method for transforming raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the method comprising:

detecting, by a processor, one or more patterns in a data channel derived from raw input data;

identifying, by the processor, one or more patterns in another data channel also derived from the raw input data;

determining, by the processor, from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generating, using a natural language generation system that is configured to execute on the processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by;

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generating, using the processor, a graphical output for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

2. A method according to claim 1, further comprising:

detecting, by the processor, one or more patterns in the data channel by;

identifying, by the processor, one or more patterns wherein a pattern is at least one of a trend, spike or step in the data channel;

assigning, by the processor, an importance level to the one or more patterns; and

identifying, by the processor, one or more key patterns of the one or more patterns, wherein a key pattern is a pattern that exceeds a predefined importance level.

3. A method according to claim 2, further comprising:

determining, by the processor, a time period to be displayed in a graph, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed.

4. A method according to claim 3, further comprising:

identifying, by the processor, one or more patterns in the another data channel by;

assigning, by the processor, an importance level to one or more unexpected patterns; and

identifying, by the processor, one or more significant patterns of the one or more unexpected patterns, wherein a significant pattern is an unexpected pattern.

5. A method according to claim 4, further comprising:

determining, by the processor, that the one or more patterns identified in an another data channel violate a predetermined constraint; and

determining, by the processor, that the one or more patterns are one or more unexpected patterns.

6. A method according to claim 4, wherein the one or more unexpected patterns are identified within the time period of the graph.

7. A method according to claim 4, further comprising:

generating, by the processor, the graph based on the graphical output, wherein the graph comprises at least a portion of the data channel that contains the one or more key events, the another data channel that contains the one or more significant events and the one or more contextual channels, the at least a portion of the one or more key events, the one or more significant events and the one or more contextual channels being annotated by the one or more phrases;

generating, by the processor, a narrative; and

causing, by the processor, the graph to be displayed in a user interface.

8. A method according to claim 7, wherein additional information related to the one or more phrases is displayed in response to a user interaction with the user interface.

9. A method according to claim 7, wherein the one or more key patterns and the one or more significant patterns are highlighted in response to a user interaction with the one or more phrases in the user interface.

10. A method according to claim 1, wherein the context information provides at least background or circumstance information having influenced the one or more patterns in the data channel or the one or more patterns in the another data channel.

11. A method according to claim 1, wherein the data channel is selected from one or more data channels derived from the raw input data, the data channel is selected based on an indication received from at least one of a user, an alarm or another computing device.

12. A method according to claim 1, further comprising:

receiving, by the processor, an indication of a determined scale from a user via a user interface; and

generating, by the processor, the graphical output based on the data channel, the another data channel and the one or more phrases in accordance with the determined scale.

13. A method according to claim 1, further comprising:

receiving, by the processor, a selection of one or more of the primary data channel, the another data channel, or a contextual channel from a user via a user interface; and

generating, by the processor, the graphical output for display in the user interface, based on the selection of the one or more of the primary data channel, the another data channel, or the contextual channel and the natural language annotation for the selection.

14. An apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least;

detect one or more patterns in a data channel derived from raw input data;

identify one or more patterns in another data channel also derived from the raw input data;

determine from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generate, using a natural language generation system that is configured to execute on a processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by;

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generate a graphical output, for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

15. An apparatus according to claim 14, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

identify one or more patterns in the primary data channel;

assign an importance level to the one or more patterns; and

identify one or more key patterns of the one or more patterns in an instance in which the importance level of a pattern of the one or more patterns exceeds a threshold defined by a domain model.

16. An apparatus according to claim 15, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine a time period to be represented by the graphical output, wherein the time period is configured such that at least a portion of the one or more key patterns occur within the time period.

17. An apparatus according to claim 15, wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine one or more patterns in one or more related data channels that corresponds to an occurrence of the one or more key patterns; and

determine one or more significant patterns of the one or more patterns, wherein a significant pattern is a pattern that represents an anomaly.

18. An apparatus according to claim 14, wherein context information provides at least background or circumstance information having influenced the one or more key patterns in the primary data channel or the one or more significant patterns in the one or more related data channels.

19. An apparatus according to claim 14, wherein additional information related to the natural language annotation is displayed in the user interface in response to a user interaction with the user interface.

20. A computer program product comprising:

at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus causes the apparatus at least to;

detect one or more patterns in a data channel derived from raw input data;

identify one or more patterns in another data channel also derived from the raw input data;

determine from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generate, using a natural language generation system that is configured to execute on a processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by;

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generate a graphical output, for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.