MALICIOUS ACTIVITY DETECTION ON A COMPUTER NETWORK AND NETWORK METADATA NORMALISATION

• US 20180248902A1
• Filed: 08/30/2016
• Published: 08/30/2018
• Est. Priority Date: 08/28/2015
• Status: Abandoned Application

• Current Assignee: Ankur Modi, Mircea Danila-Dumitrescu
• Original Assignee: Ankur Modi, Mircea Danila-Dumitrescu
• Inventors: DANILA-DUMITRESCU, Mircea, MODI, Ankur
• Application Number:
  US15/756,065
• Publication Number:
  US 20180248902A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

The invention relates to a network security and data normalisation system for a computer network, IT system or infrastructure, or similar. According to an aspect, there is provided a method for identifying abnormal user interactions within one or more monitored computer networks, comprising the steps of: receiving metadata from one or more devices within the one or more monitored computer networks; identifying from the metadata events corresponding to a plurality of user interactions with the monitored computer networks; storing user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks; updating a probabilistic model of expected user interactions from said stored user interaction event data; and testing each of said plurality of user interactions with the monitored computer networks against said probabilistic model to identify abnormal user interactions.

Claims

1. A method for identifying abnormal user interactions within one or more monitored computer networks, comprising the steps of:

receiving metadata from one or more devices within the one or more monitored computer networks;

identifying from the metadata events corresponding to a plurality of user interactions with the monitored computer networks;

extracting relevant parameters from the metadata and mapping said relevant parameters to a common data schema, thereby creating normalised user interaction data;

storing the normalised user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks;

testing the normalised user interaction event data against a probabilistic model of expected user interactions to identify abnormal user interactions; and

updating said probabilistic model from said stored user interaction event data.

2. The method of claim 1, wherein the probabilistic model comprises one or more predetermined models developed from previously identified malicious user interaction scenarios and is operable to identify malicious user interactions.

3. The method of claim 1 or 2, wherein said user interaction event data comprises any or a combination of:

data related to a user involved in an event;

data related to an action performed in an event; and

/or

data related to a device and/or application involved in an event.

4. The method of any preceding claim, wherein said common data schema comprises:

data identifying an action performed in an event; and

data identifying a user involved in an event and/or data identifying a device and/or application involved in an event.

5. The method of claim 4, wherein said common data schema further comprises any or a combination of:

data related to the or a user involved in an event;

data related to the or an action performed in an event; and

/or

data related to the or a device and/or application involved in an event.

6. The method of any preceding claim, wherein the mapping comprises looking up a metadata schema and allocating the extracted relevant parameters to the common data schema on the basis of the metadata schema

7. The method of any preceding claim, further comprising the step of storing contextual data, wherein said contextual data is related to a user interaction event and/or any of:

a user, an action, or an object involved in said event

8. The method of claim 7, wherein identifying from the metadata events corresponding to a plurality of user interactions further comprises identifying additional parameters by reference to contextual data.

9. The method according to claim 7 or 8, wherein the contextual data comprises data related to any one or more of:

identity data, job roles, psychological profiles, risk ratings, working or usage patterns, action permissibility, and/or times and dates of events.

10. The method of any of claims 7 to 9, further comprising the step of testing the normalised user interaction event data against heuristics related to contextual data to identify abnormal and/or malicious user interactions.

11. The method of any of claims 7 to 10, wherein a trained artificial neural network is used to test the normalised user interaction event data against one or more predetermined models developed from previously identified malicious user interaction scenarios and the heuristics related to contextual data.

12. The method of any of claims 7 to 11, wherein the normalised user interaction event data and contextual data are stored in a graph database.

13. The method of claim 12, further comprising the step of storing metadata and/or the relevant parameters therefrom in an index database.

14. The method of any preceding claim, wherein testing the normalised user interaction event data against said probabilistic model comprises performing continuous time analysis.

15. The method of any preceding claim, further comprising the step of testing two or more sets of normalised user interaction event data against said probabilistic model to identify abnormal user interactions.

16. The method of claim 15, further comprising the step of determining whether said two or more of the sets of normalised user interaction event data are part of an identifiable sequence of user interactions indicative of user behaviour in performing an activity.

17. The method of claim 15 or claim 16, wherein the time difference between two or more of the sets of normalised user interaction event data is tested.

18. The method of claim 17, wherein the time difference is tested against the time difference of related historic user interactions.

19. The method of any preceding claim, further comprising the step of analysing the normalised user interaction data using a further one or more probabilistic model, the results of the probabilistic models being analysed by a higher level probabilistic model to identify higher level abnormal user interactions.

20. The method of any preceding claim, wherein receiving metadata comprises aggregating metadata at a single entry point.

21. The method of any preceding claim, wherein metadata is received at the device via one or more of a third party server instance, a client server within one or more computer networks, or a direct link with the one or more devices.

22. The method of any preceding claim, wherein each of the sets of normalised user interaction event data are tested for abnormality substantially immediately following said normalised user interaction event data being stored.

23. The method of claim 22, wherein normalised user interaction event data is tested for abnormality according to a predetermined schedule in parallel with other tests.

24. The method of claim 23, wherein testing for abnormality according to a predetermined schedule comprises analysing all available normalised user interaction event data corresponding to a plurality of user interactions with the monitored computer networks, wherein said plurality of user interactions occurred within a predetermined time period.

25. The method of any preceding claim, further comprising the step of calculating a score for the normalised user interaction event data based on one or more tests.

26. The method of claim 25, further comprising the step of classifying the normalised user interaction event data based on a comparison of calculated scores for the normalised user interaction event data in combination with one or more predetermined or dynamically calculated thresholds.

27. The method of claim 26, further comprising the step of prioritising any identified abnormal and/or malicious user interactions using calculated scores and the potential impact of the identified abnormal and/or malicious user interactions.

28. The method of any of claims 25 to 27, wherein the scores are calculated in additional dependence on one or more correlations between identified abnormal and/or malicious user interactions and one or more user interactions involving the user, action, and/or object involved in the identified abnormal and/or malicious user interactions.

29. The method of any of claims 2 to 28, further comprising the step of reporting identified abnormal and/or malicious user interactions.

30. The method of any of claims 2 to 28, further comprising the step of implementing precautionary measures in response to one or more identified abnormal and/or malicious user interactions, said precautionary measures comprising one or more of:

issuing an alert, issuing a block on a user or device or a session involving said user or device, saving data, and/or performing a custom programmable action.

31. The method of any of claims 2 to 28, further comprising the step of receiving feedback related to the accuracy of the identification of the abnormal and/or malicious user interactions and updating the probabilistic model of expected user interactions and the one or more predetermined models developed from previously identified malicious user interaction scenarios in dependence on said feedback.

32. The method of any preceding claim, wherein metadata is extracted from one or more monitored computer networks via one or more of:

an application programming interface, a stream from a file server, manual export, application proxy systems, active directory log-in systems, and/or physical data storage.

33. The method of any preceding claim, further comprising the step of generating human-readable information relating to user interaction events.

34. The method of claim 33, further comprising the step of presenting said information as part of a timeline.

40. Apparatus for carrying out the method of any of claims 1 to 34.

41. A computer program product comprising software code for carrying out the method of any of claims 1 to 34.

35. Apparatus for identifying abnormal and/or malicious user interactions within one or more monitored computer networks, comprising:

a metadata-ingesting module configured to receive and aggregate metadata from one or more devices within the one or more monitored computer networks;

a data pipeline module configured to identify from the metadata events corresponding to a plurality of user interactions with the monitored computer networks;

a data store configured to store user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks; and

an analysis module comprising a probabilistic model of expected user interactions and an artificial neural network trained using one or more predetermined models developed from previously identified malicious user interaction scenarios, wherein the probabilistic model is updated from said stored user interaction event data;

wherein the analysis module is used to test the user interaction events to identify abnormal and/or malicious user interactions.

36. Apparatus according to claim 35, further comprising a user interface accessible via a web portal and/or mobile application.

37. Apparatus according to claim 36, wherein the user interface may be used to:

view metrics, graphs and reports related to identified abnormal and/or malicious user interactions, query the data store, and/or provide feedback regarding identified abnormal and/or malicious user interactions.

38. Apparatus according to any or claims 35 to 37, further comprising a transfer module configured to aggregate and send at least a portion of the metadata from the one or more devices within the one or more monitored computer networks, wherein the transfer module is within the one or more monitored computer networks.

39. Apparatus according to any of claims 35 to 38, wherein the data pipeline module is further configured to normalise the plurality of user interactions using a common data schema.

42. A method for identifying abnormal user interactions within one or more monitored computer networks, comprising the steps of:

receiving metadata from one or more devices within the one or more monitored computer networks;

identifying from the metadata events corresponding to a plurality of user interactions with the monitored computer networks;

storing user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks;

updating a probabilistic model of expected user interactions from said stored user interaction event data; and

testing each of said plurality of user interactions with the monitored computer networks against said probabilistic model to identify abnormal user interactions.

43. The method of claim 42, further comprising testing each of the plurality of user interactions with the monitored computer networks against one or more predetermined models developed from previously identified malicious user interaction scenarios to identify malicious user interactions.

44. The method of claim 42 or 43, wherein said user interaction event data comprises any or a combination of:

data related to a user involved in an event;

data related to an action performed in an event; and

/or

data related to a device and/or application involved in an event.

45. The method of any of claims 42 to 44, wherein identifying from the metadata events corresponding to a plurality of user interactions with the monitored computer networks comprises extracting relevant parameters from computer and/or network device metadata and mapping said relevant parameters to a common data schema.

46. The method of claim 45, further comprising storing contextual data, wherein said contextual data is related to a user interaction event and/or any of:

a user, an action, or an object involved in said event.

47. The method of claim 46, wherein identifying from the metadata events corresponding to a plurality of user interactions further comprises identifying additional parameters by reference to contextual data.

48. The method according to claim 46 or 47, wherein the contextual data comprises data related to any one or more of:

identity data, job roles, psychological profiles, risk ratings, working or usage patterns, action permissibilities, and/or times and dates of events.

49. The method of any of claims 46 to 48, further comprising testing each of the plurality of user interactions with the monitored computer networks against heuristics related to contextual data to identify abnormal and/or malicious user interactions.

50. The method of any of claims 46 to 49, wherein a trained artificial neural network is used to test each of the plurality of user interactions with the monitored computer networks against the one or more predetermined models developed from previously identified malicious user interaction scenarios and the heuristics related to contextual data.

51. The method of any of claims 46 to 50, wherein user interaction event data and contextual data are stored in a graph database.

52. The method of claim 51, further comprising storing metadata and/or the relevant parameters therefrom in an index database.

53. The method of any of claims 42 to 52, wherein testing each of said plurality of user interactions with the monitored computer networks against said probabilistic model comprises performing continuous time analysis.

54. The method of any of claims 42 to 53, further comprising testing two or more of said plurality of user interactions in combination against said probabilistic model to identify abnormal user interactions.

55. The method of claim 54, further comprising the step of determining whether said two or more of the plurality of user interactions are part of an identifiable sequence of user interactions indicative of user behaviour in performing an activity.

56. The method of claim 54 or claim 55, wherein the time difference between two or more of said plurality of user interactions is tested.

57. The method of claim 56, wherein the time difference is tested against the time difference of related historic user interactions.

58. The method of any of claims 42 to 57, wherein receiving metadata comprises aggregating metadata at a single entry point.

59. The method of any of claims 42 to 58, wherein metadata is received at the device via one or more of a third party server instance, a client server within one or more computer networks, or a direct link with the one or more devices.

60. The method of any of claims 42 to 59, wherein each of the plurality of user interactions with the monitored computer networks are tested for abnormality substantially immediately following said user interaction event data being stored.

61. The method of claim 60, wherein each of the plurality of user interactions with the monitored computer networks are tested for abnormality according to a predetermined schedule in parallel with other tests.

62. The method of claim 61, wherein testing for abnormality according to a predetermined schedule comprises analysing all available user interaction data corresponding to a plurality of user interactions with the monitored computer networks, wherein said plurality of user interactions occurred within a predetermined time period.

63. The method of any of claims 42 to 62, further comprising calculating a score for each of the plurality of user interactions and/or a plurality of user interactions with the monitored computer networks based on one or more tests.

64. The method of claim 63, further comprising classifying each of the plurality of user interactions with the monitored computer networks based on a comparison of calculated scores for each of the plurality of user interactions and/or a plurality of user interactions in combination with one or more predetermined or dynamically calculated thresholds.

65. The method of claim 64, further comprising prioritising any identified abnormal and/or malicious user interactions using calculated scores and the potential impact of the identified abnormal and/or malicious user interactions.

66. The method of any of claims 63 to 65, wherein the scores are calculated in additional dependence on one or more correlations between identified abnormal and/or malicious user interactions and one or more user interactions involving the user, action, and/or object involved in the identified abnormal and/or malicious user interactions.

67. The method of any of claims 43 to 66, further comprising reporting identified abnormal and/or malicious user interactions.

68. The method of any of claims 43 to 67, further comprising implementing precautionary measures in response to one or more identified abnormal and/or malicious user interactions, said precautionary measures comprising one or more of:

issuing an alert, issuing a block on a user or device or a session involving said user or device, saving data, and/or performing a custom programmable action.

69. The method of any of claims 43 to 68, further comprising receiving feedback related to the accuracy of the identification of the abnormal and/or malicious user interactions and updating the probabilistic model of expected user interactions and the one or more predetermined models developed from previously identified malicious user interaction scenarios in dependence on said feedback.

70. The method of any of claims 42 to 69, wherein metadata is extracted from one or more monitored computer networks via one or more of:

an application programming interface, a stream from a file server, manual export, application proxy systems, active directory log-in systems, and/or physical data storage.

71. The method of any of claims 42 to 70, further comprising generating human-readable information relating to user interaction events.

72. The method of claim 71, further comprising presenting said information as part of a timeline.

73. A method for normalising metadata having a plurality of content schemata from one or more devices, within one or more monitored computer networks, comprising the steps of:

receiving metadata from the one or more devices within the one or more monitored computer networks;

extracting relevant parameters from the metadata and mapping said relevant parameters to a common data schema in order to identify events corresponding to a plurality of user interactions with the monitored computer networks; and

storing user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks.

74. The method of claim 73, wherein said common data schema comprises:

data identifying an action performed in an event; and

data identifying a user involved in an event and/or data identifying a device and/or application involved in an event.

75. The method of claim 73 or 74, wherein said common data schema further comprises any or a combination of:

data related to the or a user involved in an event;

data related to the or an action performed in an event; and

/or

data related to the or a device and/or application involved in an event.

76. The method of any of claims 73 to 75, wherein the mapping comprises looking up a metadata schema and allocating the extracted relevant parameters to the common data schema on the basis of the metadata schema.

77. The method of any of claims 73 to 76, further comprising identifying additional parameters related to the metadata.

78. The method of claim 77, wherein the additional parameters are identified from a look-up table.

79. The method of claim 77 or 78, further comprising storing the additional parameters as part of the user interaction event data.

80. The method of any of claims 73 to 79, further comprising analysing the metadata.

81. The method of claim 80, wherein analysing comprises testing a first event against a second related event to identify a chain of related events.

82. The method of any of claims 73 to 81, further comprising reporting.

83. The method of claim 82, wherein reporting comprises compiling a sequence of one or more related events and providing data relating to those events.

84. The method claim 83, wherein the one or more related events relate to a particular time period.

85. The method of claim 83 or 84, further comprising providing said data as part of a timeline.

86. The method of any of claims 83 to 85, wherein the one or more related events relate to the same user, device, object, and/or chain.

87. The method of any of claims 82 to 86, wherein reporting comprises providing data relating to one or more events in the form of human-readable statements.

88. The method of any of claims 73 to 87, wherein receiving metadata comprises aggregating metadata at a single entry point.

89. The method of any of claims 73 to 88, wherein metadata is received via one or more of a third party server instance, a client server within one or more computer networks, or a direct link with the one or more devices.

90. The method of any of claims 73 to 89, wherein metadata is extracted from one or more monitored computer networks via one or more of:

an application programming interface, a stream from a file server, manual export, application proxy systems, active directory log-in systems, and/or physical data storage.

91. The method of any of claims 73 to 90, wherein user interaction event data are stored in a graph database.

92. The method of any of claims 73 to 91, wherein user interaction event data are stored in an index database.

97. Apparatus for carrying out the method of any of claims 73 to 92.

98. A computer program product comprising software code for carrying out the method of any of claims 73 to 92.

93. Apparatus for normalising metadata having a plurality of content schemata from one or more devices, within one or more monitored computer networks, comprising:

a metadata-ingesting module configured to receive and aggregate metadata from one or more devices within the one or more monitored computer networks;

a data pipeline module configured to extract relevant parameters from the metadata and map said relevant parameters to a common data schema in order to identify from the metadata events corresponding to a plurality of user interactions with the monitored computer networks; and

a data store configured to store user interaction event data from the identified said events corresponding to a plurality of user interactions with the monitored computer networks.

94. Apparatus according to claim 93, further comprising a user interface accessible via a web portal and/or mobile application.

95. Apparatus according to claim 94, wherein the user interface may be used to:

view metrics, graphs and reports related to identified events, and/or query the data store.

96. Apparatus according any of claims 93 to 95, further comprising a transfer module configured to aggregate and send at least a portion of the metadata from the one or more devices within the one or more monitored computer networks, wherein the transfer module is within the one or more monitored computer networks.

99. A method substantially as herein described and/or as illustrated with reference to the accompanying figures.

100. Apparatus substantially as herein described and/or as illustrated with reference to the accompanying figures.