Data trend analysis

• US 9,355,172 B2
• Filed: 03/14/2013
• Issued: 05/31/2016
• Est. Priority Date: 01/10/2013
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Howes, Joshua Z., Solderitsch, James, LaSalle, Ryan M., Rozmiarek, David W., Ellett, Eric A.
• Primary Examiner(s): Patel, Nirav B
• Application Number:
  US13/826,965
• Publication Number:
  US 20140283048A1
• Time in Patent Office: 1,174 Days
• Field of Search: 707/746, 707/730, 707/737, 707/728, 707/769, 707/741, 707/603, 707/750, 707/759, 707/736, 726 22- 25
• US Class Current: 1/1
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

According to an example, a method for data trend analysis may include retrieving data from data sources, associating the data with a time, and identifying co-occurrences of terms and concepts within the data. In response to determining that co-occurrences of term and concept pairs reach a predefined threshold, the method may include adding the term and concept pairs to an ontology. The method may include logging occurrences of terms in the ontology within the data with respect to associated data times, identifying a plurality of time periods, and for one of the plurality of time periods and for the logged terms, determining a first score indicative of a weighted term frequency metric for a logged term within the data during the one time period, and determining a second score indicative of a commonality of a presence of the logged term within the data among the plurality of time periods.

Claims

1. A method for cyber security threat related data trend analysis, the method comprising:

retrieving, over a computer network, cyber security threat related data from at least one data source;

associating the cyber security threat related data with a time;

identifying, by an ontology creation hardware module, co-occurrences of terms and concepts within the cyber security threat related data;

in response to determining that co-occurrences of term and concept pairs reach a predefined threshold, adding the term and concept pairs to an ontology;

logging occurrences of terms in the ontology within the cyber security threat related data with respect to associated data times;

identifying a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determining, by a hardware processor, a first score indicative of a weighted term frequency metric for a logged term within the cyber security threat related data during the one time period, and

determining, by the hardware processor, a second score indicative of a commonality of a presence of the logged term within the cyber security threat related data among the plurality of time periods;

using the first and second scores to determine a cyber security threat trend;

monitoring the at least one data source or another source related to the at least one data source based on the determined cyber security threat trend;

generating, based on the monitoring, a report indicative of the cyber security threat trend related to the cyber security threat related data;

determining a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods; and

filtering the logged terms based on user preferences by multiplying the third score by a user-preference factor.

2. The method of claim 1, wherein determining the first score further comprises:

determining a logarithm of a count of occurrences of the logged term within the cyber security threat related data during the one time period.

3. The method of claim 1, wherein determining the second score further comprises:

determining a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods; and

determining a logarithm of an inverse of the quotient value.

4. The method of claim 1, wherein identifying co-occurrences of the terms and the concepts within the cyber security threat related data further comprises:

using Latent Semantic Analysis (LSA) to identify co-occurrences of the terms and the concepts within the cyber security threat related data.

5. The method of claim 1, wherein retrieving the cyber security threat related data from the at least one data source further comprises:

retrieving the cyber security threat related data from at least one of structured, unstructured, and semi-structured data sources;

in response to determining that the at least one data source is the structured data source, parsing the cyber security threat related data for the structured data source; and

in response to determining that the at least one data source is the unstructured or the semi-structured data source, identifying and extracting the terms from the unstructured or the semi-structured data sources.

6. The method of claim 1, further comprising:

filtering the logged terms based on community feedback by multiplying the third score by a community feedback factor.

7. The method of claim 6, further comprising:

prioritizing the filtered logged terms based on an ascending or a descending order related to the third score.

8. The method of claim 1, wherein the cyber security threat related data is related to cyber security threat information and cyber security threat related information, the method further comprises:

retrieving the cyber security threat related data related to the cyber security threat information from a structured data source of the least one data source;

retrieving the cyber security threat related data related to the cyber security threat related information from semi-structured and un-structured data sources of the least one data source;

extracting additional cyber security threat information from the retrieved cyber security threat related information; and

identifying co-occurrences of the terms and the concepts within the cyber security threat related data by identifying co-occurrences of threat-related terms and threat-related concepts within the cyber security threat information and the additional cyber security threat information.

9. A method for forecasting cyber security threat risks, the method comprising:

retrieving, over a computer network, cyber security threat information from structured data sources;

retrieving, over the computer network, cyber security threat related information from semi-structured and un-structured data sources;

extracting additional cyber security threat information from the retrieved cyber security threat related information;

identifying, by an ontology creation hardware module, co-occurrences of threat-related terms and threat-related concepts within the cyber security threat information and the additional cyber security threat information;

in response to determining that co-occurrences of threat-related term and threat-related concept pairs reach a predefined threshold, adding the term and concept pairs to an ontology;

logging occurrences of terms in the ontology within the cyber security threat information or the cyber security threat related information or both with respect to time;

identifying a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determining, by a hardware processor, a first score indicative of a weighted term frequency metric for a logged term during the one time period, and

determining, by the hardware processor, a second score indicative of a commonality of a presence of the logged term among the plurality of time periods;

using the first and second scores to determine a cyber security threat trend;

monitoring the structured data sources or the semi-structured and un-structured data sources based on the determined cyber security threat trend;

generating, based on the monitoring, a report indicative of the cyber security threat trend;

determining a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods; and

filtering the logged terms based on user preferences by multiplying the third score by a user-preference factor.

10. The method of claim 9, wherein determining the second score further comprises:

determining a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods; and

determining a logarithm of an inverse of the quotient value.

11. The method of claim 9, wherein identifying co-occurrences of the threat-related terms and the threat-related concepts further comprises:

using Latent Semantic Analysis (LSA) to identify co-occurrences of the threat-related terms and the threat-related concepts.

12. The method of claim 9, further comprising:

filtering the logged terms based on community feedback by multiplying the third score by a community feedback factor.

13. The method of claim 12, further comprising:

prioritizing the filtered logged terms based on an ascending or a descending order related to the third score.

14. A cyber security threat related data trend analysis system comprising:

at least one hardware processor; and

a memory storing machine readable instructions that when executed by the at least one hardware processor cause the at least one hardware processor to;

retrieve, over a computer network, cyber security threat related data from at least one data source;

associate the cyber security threat related data with a time;

identify, by an ontology creation hardware module, co-occurrences of terms and concepts within the cyber security threat related data;

in response to determining that co-occurrences of term and concept pairs reach a predefined threshold, add the term and concept pairs to an ontology;

log occurrences of terms in the ontology within the cyber security threat related data with respect to associated data times;

identify a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determine a first score indicative of a weighted term frequency metric for a logged term within the cyber security threat related data during the one time period, and

determine a second score indicative of a commonality of a presence of the logged term within the cyber security threat related data among the plurality of time periods;

use the first and second scores to determine a cyber security threat trend;

monitor the at least one data source or another source related to the at least one data source based on the determined cyber security threat trend;

receive user preferences;

determine a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods;

filter the logged terms based on the user preferences by multiplying the third score by a user-preference factor;

prioritize the filtered logged terms; and

generate, based on the monitoring, a report indicative of the cyber security threat trend related to the cyber security threat related data based on the filtered and prioritized logged terms.

15. The data trend analysis system of claim 14, wherein the machine readable instructions to determine the second score further comprise:

determining a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods; and

determining a logarithm of an inverse of the quotient value.

16. The data trend analysis system of claim 14, wherein the machine readable instructions to retrieve the cyber security threat related data from the at least one data source further comprise:

retrieving the cyber security threat related data from at least one of structured, unstructured, and semi-structured data sources;

in response to determining that the at least one data source is the structured data source, parsing the cyber security threat related data for the structured data source; and

in response to determining that the at least one data source is the unstructured or the semi-structured data source, identifying and extracting the terms from the unstructured or the semi-structured data sources.

17. The data trend analysis system of claim 14, wherein the machine readable instructions to determine the first score further comprise:

determining a logarithm of a count of occurrences of the logged term within the cyber security threat related data during the one time period.

18. The data trend analysis system of claim 14, wherein the machine readable instructions to identify co-occurrences of the terms and the concepts within the cyber security threat related data further comprise:

using Latent Semantic Analysis (LSA) to identify co-occurrences of the terms and the concepts within the cyber security threat related data.

19. A non-transitory computer readable medium having stored thereon a computer executable program to provide cyber security threat related data trend analysis, the computer executable program when executed causes at least one hardware processor to:

retrieve, over a computer network, cyber security threat related data from at least one data source;

associate the cyber security threat related data with a time;

identify, by an ontology creation hardware, co-occurrences of terms and concepts within the cyber security threat related data;

in response to determining that co-occurrences of term and concept pairs reach a predefined threshold, add the term and concept pairs to an ontology;

log occurrences of terms in the ontology within the cyber security threat related data with respect to associated data times;

identify a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determine a first score indicative of a weighted term frequency metric for a logged term within the cyber security threat related data during the one time period, and

determine a second score indicative of a commonality of a presence of the logged term within the cyber security threat related data among the plurality of time periods;

use the first and second scores to determine a cyber security threat trend;

monitor the at least one data source or another source related to the at least one data source based on the determined cyber security threat trend to

predict a future event related to the at least one data source or the another source related to the at least one data source, and/or

estimate uncertain past events related to the at least one data source or the another source related to the at least one data source; and

generate, based on the monitoring, a report indicative of the cyber security threat trend related to the cyber security threat related data.

20. The non-transitory computer readable medium of claim 19, the computer executable program when executed further causes the computer system to:

determine a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods.