Data trend analysis

• US 9,531,743 B2
• Filed: 05/05/2016
• Issued: 12/27/2016
• Est. Priority Date: 01/10/2013
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Lasalle, Ryan M., Howes, Joshua Z., Solderitsch, James, Rozmiarek, David W., Ellett, Eric A.
• Primary Examiner(s): Patel, Nirav B
• Application Number:
  US15/147,471
• Publication Number:
  US 20160248793A1
• Time in Patent Office: 236 Days
• Field of Search: 726 22- 25, 707/746, 707/730, 707/737, 707/728, 707/769, 707/741, 707/603, 707/750, 707/759, 707/736
• US Class Current: 1/1
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

According to an example, a method for data trend analysis may include retrieving data from data sources, associating the data with a time, and identifying co-occurrences of terms and concepts within the data. In response to determining that co-occurrences of term and concept pairs reach a predefined threshold, the method may include adding the term and concept pairs to an ontology. The method may include logging occurrences of terms in the ontology within the data with respect to associated data times, identifying a plurality of time periods, and for one of the plurality of time periods and for the logged terms, determining a first score indicative of a weighted term frequency metric for a logged term within the data during the one time period, and determining a second score indicative of a commonality of a presence of the logged term within the data among the plurality of time periods.

Claims

1. A method for cyber security threat related data trend analysis, the method comprising:

retrieving, over a computer network, cyber security threat related data from at least one data source;

associating the cyber security threat related data with a time;

identifying, by a hardware processor, co-occurrences of terms and concepts within the cyber security threat related data;

in response to determining that co-occurrences of term and concept pairs reach a predefined threshold, adding the term and concept pairs to an ontology;

logging occurrences of terms in the ontology within the cyber security threat related data with respect to associated data times;

identifying a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determining, by the hardware processor, a first score indicative of a weighted term frequency metric for a logged term within the cyber security threat related data during the one time period by

determining a logarithm of a count of occurrences of the logged term within the cyber security threat related data during the one time period, and

determining, by the hardware processor, a second score indicative of a commonality of a presence of the logged term within the cyber security threat related data among the plurality of time periods;

using the first and second scores to determine a cyber security threat trend;

monitoring the at least one data source or another source related to the at least one data source based on the determined cyber security threat trend; and

generating, based on the monitoring, a report indicative of the cyber security threat trend related to the cyber security threat related data.

2. The method of claim 1, wherein determining the second score further comprises:

determining a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods; and

determining a logarithm of an inverse of the quotient value.

3. The method of claim 1, wherein identifying co-occurrences of the terms and the concepts within the cyber security threat related data further comprises:

using Latent Semantic Analysis (LSA) to identify co-occurrences of the terms and the concepts within the cyber security threat related data.

4. The method of claim 1, wherein retrieving the cyber security threat related data from the at least one data source further comprises:

retrieving the cyber security threat related data from at least one of structured, unstructured, and semi-structured data sources;

in response to determining that the at least one data source is the structured data source, parsing the cyber security threat related data for the structured data source; and

in response to determining that the at least one data source is the unstructured or the semi-structured data source, identifying and extracting the terms from the unstructured or the semi-structured data sources.

5. The method of claim 1, further comprising:

determining a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods; and

filtering the logged terms based on community feedback by multiplying the third score by a community feedback factor.

6. The method of claim 5, further comprising:

prioritizing the filtered logged terms based on an ascending or a descending order related to the third score.

7. The method of claim 1, wherein the cyber security threat related data is related to cyber security threat information and cyber security threat related information, the method further comprises:

retrieving the cyber security threat related data related to the cyber security threat information from a structured data source of the least one data source;

retrieving the cyber security threat related data related to the cyber security threat related information from semi-structured and un-structured data sources of the least one data source;

extracting additional cyber security threat information from the retrieved cyber security threat related information; and

identifying co-occurrences of the terms and the concepts within the cyber security threat related data by identifying co-occurrences of threat-related terms and threat-related concepts within the cyber security threat information and the additional cyber security threat information.

8. A cyber security threat related data trend analysis system comprising:

at least one hardware processor; and

a memory storing machine readable instructions that when executed by the at least one hardware processor cause the at least one hardware processor to;

retrieve, over a computer network, cyber security threat information from structured data sources;

retrieve, over the computer network, cyber security threat related information from semi-structured and un-structured data sources;

extract additional cyber security threat information from the retrieved cyber security threat related information;

identify co-occurrences of threat-related terms and threat-related concepts within the cyber security threat information and the additional cyber security threat information;

in response to determining that co-occurrences of threat-related term and threat-related concept pairs reach a predefined threshold, add the term and concept pairs to an ontology;

log occurrences of terms in the ontology within the cyber security threat information or the cyber security threat related information or both with respect to time;

identify a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determine a first score indicative of a weighted term frequency metric for a logged term during the one time period, and

determine a second score indicative of a commonality of a presence of the logged term among the plurality of time periods;

use the first and second scores to determine a cyber security threat trend;

monitor the structured data sources or the semi-structured and un-structured data sources based on the determined cyber security threat trend;

generate, based on the monitoring, a report indicative of the cyber security threat trend;

determine a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods; and

filter the logged terms based on community feedback by multiplying the third score by a community feedback factor.

9. The data trend analysis system of claim 8, wherein the machine readable instructions to determine the second score further comprise machine readable instructions that when executed by the at least one hardware processor further cause the at least one hardware processor to:

determine a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods; and

determine a logarithm of an inverse of the quotient value.

10. The data trend analysis system of claim 8, wherein the machine readable instructions to identify co-occurrences of the threat-related terms and the threat-related concepts further comprise machine readable instructions that when executed by the at least one hardware processor further cause the at least one hardware processor to:

using Latent Semantic Analysis (LSA) to identify co-occurrences of the threat-related terms and the threat-related concepts.

11. The data trend analysis system of claim 8, further comprising machine readable instructions that when executed by the at least one hardware processor further cause the at least one hardware processor to:

prioritize the filtered logged terms based on an ascending or a descending order related to the third score.

12. A non-transitory computer readable medium having stored thereon a computer executable program to provide cyber security threat related data trend analysis, the computer executable program when executed causes at least one hardware processor to:

retrieve, over a computer network, cyber security threat related data from at least one data source;

associate the cyber security threat related data with a time;

identify co-occurrences of terms and concepts within the cyber security threat related data;

in response to determining that co-occurrences of term and concept pairs reach a predefined threshold, add the term and concept pairs to an ontology;

log occurrences of terms in the ontology within the cyber security threat related data with respect to associated data times;

identify a plurality of time periods;

for one of the plurality of time periods and for the logged terms;

determine a first score indicative of a weighted term frequency metric for a logged term within the cyber security threat related data during the one time period, and

determine a second score indicative of a commonality of a presence of the logged term within the cyber security threat related data among the plurality of time periods;

use the first and second scores to determine a cyber security threat trend;

monitor the at least one data source or another source related to the at least one data source based on the determined cyber security threat trend;

generate, based on the monitoring, a report indicative of the cyber security threat trend related to the cyber security threat related data;

determine a third score indicative of the weighted term frequency metric for the logged term during the one time period and the commonality of the presence of the logged term among the plurality of time periods;

filter the logged terms based on the third score; and

prioritize the filtered logged terms based on an ascending or a descending order.

13. The non-transitory computer readable medium of claim 12, the computer executable program when executed further causes the at least one hardware processor to:

determine the first score by determining a logarithm of a count of occurrences of the logged term within the cyber security threat related data during the one time period.

14. The non-transitory computer readable medium of claim 12, the computer executable program when executed further causes the at least one hardware processor to:

determine the second score by determining a quotient value by dividing a count of occurrences of the logged term across the plurality of time periods by a count of occurrences of the logged terms across the plurality of time periods, and determining a logarithm of an inverse of the quotient value.

15. The non-transitory computer readable medium of claim 12, wherein the computer executable program to cause the at least one hardware processor to identify co-occurrences of the terms and the concepts within the cyber security threat related data further causes the at least one hardware processor to:

use Latent Semantic Analysis (LSA) to identify co-occurrences of the terms and the concepts within the cyber security threat related data.

16. The non-transitory computer readable medium of claim 12, the computer executable program when executed further causes the at least one hardware processor to:

retrieve the cyber security threat related data from at least one of structured, unstructured, and semi-structured data sources;

in response to determining that the at least one data source is the structured data source, parse the cyber security threat related data for the structured data source; and

in response to determining that the at least one data source is the unstructured or the semi-structured data source, identify and extract the terms from the unstructured or the semi-structured data sources.

17. The non-transitory computer readable medium of claim 12, wherein the computer executable program to cause the at least one hardware processor to filter the logged terms based on the third score further causes the at least one hardware processor to:

filter the logged terms based on community feedback by multiplying the third score by a community feedback factor.

18. The non-transitory computer readable medium of claim 17, wherein the computer executable program to cause the at least one hardware processor to prioritize the filtered logged terms based on the ascending or the descending order further causes the at least one hardware processor to:

prioritize the filtered logged terms based on the ascending or the descending order related to the third score.

19. The non-transitory computer readable medium of claim 12, wherein the at least one data source includes at least one of structured, unstructured, and semi-structured data sources.

20. The non-transitory computer readable medium of claim 12, wherein the cyber security threat related data is related to cyber security threat information and cyber security threat related information, the computer executable program when executed further causes the at least one hardware processor to:

retrieve the cyber security threat related data related to the cyber security threat information from a structured data source of the least one data source;

retrieve the cyber security threat related data related to the cyber security threat related information from semi-structured and un-structured data sources of the least one data source;

extract additional cyber security threat information from the retrieved cyber security threat related information; and

identify co-occurrences of the terms and the concepts within the cyber security threat related data by identifying co-occurrences of threat-related terms and threat-related concepts within the cyber security threat information and the additional cyber security threat information.