Unstructured security threat information analysis

• US 9,716,721 B2
• Filed: 08/29/2014
• Issued: 07/25/2017
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Ramnani, Roshni Ramesh, Sengupta, Shubhashis, Mohamedrasheed, Annervaz Karukapadath, Hovor, Elvis, Modi, Shimon
• Primary Examiner(s): Pham, Hung Q
• Application Number:
  US14/473,743
• Publication Number:
  US 20160065599A1
• Time in Patent Office: 1,061 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/53 using third party service providers

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for creating structured data using data received from unstructured textual data sources. One of the methods includes receiving unstructured textual data, identifying one or more keywords in the unstructured textual data, determining one or more patterns included in the unstructured textual data using the identified keywords, identifying one or more intelligence types that correspond with the unstructured textual data using the determined patterns, and associating, for each of the identified intelligence types, a data subset from the unstructured textual data with the respective intelligence type.

Claims

1. A computer-implemented method comprising:

receiving, by an analysis system that includes one or more computers, a plurality of unstructured textual datasets that each include information about a respective potential security threat;

determining that a first subset of the plurality of unstructured textual datasets and a second, different subset of the plurality of unstructured textual datasets both comprise information about a particular threat, the second, different subset being a different subset than the first subset;

discarding the first subset in response to determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat;

for each respective subset in the plurality of unstructured textual datasets that has not been discarded;

identifying, by the analysis system, one or more keywords in the respective subset;

determining, by the analysis system, one or more patterns included in the respective subset using the identified one or more keywords;

identifying, by the analysis system, one or more intelligence types that correspond with the respective subset using the one or more patterns; and

associating, by the analysis system for each respective intelligence type of the identified one or more intelligence types, the respective subset from the plurality of unstructured textual datasets with the respective intelligence type;

determining a rule for a third party that indicates that the third party should receive data associated with a particular intelligence type of the one or more intelligence types;

determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party.

2. The method of claim 1, wherein associating, for each respective intelligence type of the identified one or more intelligence types, the respective subset from the plurality of unstructured textual datasets with the respective intelligence type comprises storing, for each respective intelligence type of the identified one or more intelligence types, at least one new record, in a database, specific to the respective intelligence type that each comprises information from the respective subset.

3. The method of claim 1, wherein:

receiving the plurality of unstructured textual datasets comprises receiving a security advisory that identifies at least one of a particular hardware device or a particular software application;

determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat comprises determining that the first subset and the second, different subset both comprise information about the particular hardware device or the particular software application; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing, to the third party, the second subset that comprises information about the particular hardware device or the particular software application.

4. The method of claim 1, wherein receiving the plurality of unstructured textual datasets comprises receiving at least some of the plurality of unstructured textual datasets from a government source or a security source.

5. The method of claim 1, wherein identifying the one or more intelligence types that correspond with the respective subset using the one or more patterns comprises:

determining one or more rules using the one or more patterns; and

identifying the one or more intelligence types that correspond with the respective subset using the one or more rules.

6. The method of claim 1, wherein providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party is responsive to determining the rule for the third party that indicates that the third party should receive data associated with the particular intelligence type of the one or more intelligence types and determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type.

7. The method of claim 6, wherein providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing instructions to the third party for presentation of information included in the second subset.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving a plurality of unstructured textual datasets that each include information about a respective potential security threat;

determining that a first subset of the plurality of unstructured textual datasets and a second, different subset of the plurality of unstructured textual datasets both comprise information about a particular threat, the second, different subset being a different subset than the first subset;

discarding the first subset in response to determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat;

for each respective subset in the plurality of unstructured textual datasets that has not been discarded;

identifying one or more keywords in the respective subset;

determining one or more patterns included in the respective subset using the identified one or more keywords; and

identifying one or more intelligence types that correspond with the respective subset using the one or more patterns; and

associating, for each respective intelligence type of the identified one or more intelligence types, the respective subset from the plurality of unstructured textual datasets with the respective intelligence type;

determining a rule for a third party that indicates that the third party should receive data associated with a particular intelligence type of the one or more intelligence types;

determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party.

9. The system of claim 8, wherein associating, for each respective intelligence type of the identified one or more intelligence types, the respective subset from the plurality of unstructured textual datasets with the respective intelligence type comprises storing, for respective intelligence type of the identified one or more intelligence types, at least one new record, in a database, specific to the respective intelligence type that each comprises information from the respective subset.

10. The system of claim 8, wherein providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party is responsive to determining the rule for the third party that indicates that the third party should receive data associated with the particular intelligence type of the one or more intelligence types and determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type.

11. The system of claim 10, wherein providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing instructions to the third party for presentation of information included in the second subset.

12. The system of claim 8, wherein:

receiving the plurality of unstructured textual datasets comprises receiving a security advisory that identifies at least one of a particular hardware device or a particular software application;

determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat comprises determining that the first subset and the second, different subset both comprise information about the particular hardware device or the particular software application; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing, to the third party, the second subset that comprises information about the particular hardware device or the particular software application.

13. The system of claim 8, wherein receiving the plurality of unstructured textual datasets comprises receiving at least some of the plurality of unstructured textual datasets from a government source or a security source.

14. The system of claim 8, wherein identifying the one or more intelligence types that correspond with the respective subset using the one or more patterns comprises:

determining one or more rules using the one or more patterns; and

identifying the one or more intelligence types that correspond with the respective subset using the one or more rules.

15. A computer storage medium encoded with instructions that, when executed by a user device, cause the user device to perform operations comprising:

receiving a plurality of unstructured textual datasets that each include information about a respective potential security threat;

determining that a first subset of the plurality of unstructured textual datasets and a second, different subset of the plurality of unstructured textual datasets both comprise information about a particular threat, the second, different subset being a different subset than the first subset;

discarding the first subset in response to determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat;

for each respective subset in the plurality of unstructured textual datasets that has not been discarded;

identifying one or more keywords in the respective subset;

determining one or more patterns included in the respective subset using the identified one or more keywords;

identifying one or more intelligence types that correspond with the respective subset using the one or more patterns; and

associating, for each respective intelligence type of the identified one or more intelligence types, the respective subset from the plurality of unstructured textual datasets with the respective intelligence type;

determining a rule for a third party that indicates that the third party should receive data associated with a particular intelligence type of the one or more intelligence types;

determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party.

16. The computer storage medium of claim 15, wherein associating, for each respective intelligence type of the identified one or more intelligence types, the respective subset from the unstructured textual datasets with the respective intelligence type comprises storing, for each respective intelligence type of the identified one or more intelligence types, at least one new record, in a database, specific to the respective intelligence type that each comprises information from the respective subset.

17. The computer storage medium of claim 15, wherein:

receiving the plurality of unstructured textual datasets comprises receiving a security advisory that identifies at least one of a particular hardware device or a particular software application;

determining that the first subset of the plurality of unstructured textual datasets and the second, different subset of the plurality of unstructured textual datasets both comprise information about the particular threat comprises determining that the first subset and the second, different subset both comprise information about the particular hardware device or the particular software application; and

providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing, to the third party, the second subset that comprises information about the particular hardware device or the particular software application.

18. The computer storage medium of claim 15, wherein receiving the plurality of unstructured textual datasets comprises receiving at least some of the plurality of unstructured textual datasets from a government source or a security source.

19. The computer storage medium of claim 15, wherein identifying the one or more intelligence types that correspond with the respective subset using the one or more patterns comprises:

determining one or more rules using the one or more patterns; and

identifying the one or more intelligence types that correspond with the respective subset using the one or more rules.

20. The computer storage medium of claim 15, wherein providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party is responsive to determining the rule for the third party that indicates that the third party should receive data associated with the particular intelligence type of the one or more intelligence types and determining that the second subset of the plurality of unstructured textual datasets is associated with the particular intelligence type.

21. The computer storage medium of claim 20, wherein providing providing the second subset of the plurality of unstructured textual datasets that is associated with the particular intelligence type to the third party comprises providing instructions to the third party for presentation of information included in the second subset.