Assessing a computing resource for compliance with a computing resource policy regime specification

• US 8,990,392 B1
• Filed: 05/09/2014
• Issued: 03/24/2015
• Est. Priority Date: 04/11/2012
• Status: Active Grant

• Current Assignee: Artemis Internet Inc.
• Original Assignee: NCC Group, Inc. (NCC Group Plc)
• Inventors: Stamos, Alexander Charles
• Primary Examiner(s): Barry, Lance L
• Application Number:
  US14/274,671
• Time in Patent Office: 319 Days
• US Class Current: 709/224
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/50 Testing arrangements
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 61/302 Administrative registration, e.g. for domain names at internet corporation for assigned names and numbers [ICANN]
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A determination is made, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated. The assessment is associated with a plurality of tests. A pre-scan associated with the computing resource is performed. The pre-scan is configured to assess functionality provided by the computing resource. The assessment is dispatched. Dispatching the assessment includes selecting, as a test, based at least in part on the pre-scan, a subset of the tests associated with the assessment.

Claims

1. A system, comprising:

a processor configured to;

determine, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiate a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatch the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment, and wherein dispatching the assessment includes performing a deduplicating action on the test set; and

deliver results of the assessment; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 wherein the assessment is dispatched based at least in part on a determination being made by a timer that the assessment should be dispatched.

3. The system of claim 1 wherein delivering the results includes weighting the results.

4. The system of claim 3 wherein dispatching the assessment includes initiating a plurality of third party scans and wherein weighting the results includes weighting the third party scan results.

5. The system of claim 1 wherein dispatching the assessment includes requesting a plurality of scans from a plurality of third party external scanners.

6. The system of claim 1 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

7. The system of claim 6 wherein the auditor interface includes results from a plurality of computing resources.

8. The system of claim 7 wherein the auditor interface includes results from a plurality of customers of the system.

9. The system of claim 1 wherein the processor is further configured to receive permission from an administrator associated with the computing resource to make available at least a portion of the results to a third party.

10. The system of claim 1 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

11. The system of claim 1 wherein the processor is further configured to queue, for operator review, a violation of the computing resource policy regime specification detected during the assessment.

12. The system of claim 1 wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification.

13. The system of claim 12 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

14. The system of claim 13 wherein the processor is further configured to receive, from the administrator, an indication that more time is required by the administrator to remedy the violation.

15. The system of claim 12 wherein the processor is further configured to receive, from the administrator, a request to retest the computing resource with respect to the violation.

16. The system of claim 12 wherein the processor is further configured to receive, from the administrator, an indication that the administrator believes the violation to be a false positive.

17. The system of claim 16 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

18. The system of claim 1 wherein delivering the results includes storing the results in a results repository.

19. The system of claim 1 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

20. A method, comprising:

determining, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiating a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatching the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment and wherein dispatching the assessment includes performing a deduplicating action on the test set; and

delivering results of the assessment.

21. The method of claim 20 wherein the assessment is dispatched based at least in part on a determination being made by a timer that the assessment should be dispatched.

22. The method of claim 20 wherein delivering the results includes weighting the results.

23. The method of claim 22 wherein dispatching the assessment includes initiating a plurality of third party scans and wherein weighting the results includes weighting the third party scan results.

24. The method of claim 20 wherein dispatching the assessment includes requesting a plurality of scans from a plurality of third party external scanners.

25. The method of claim 20 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

26. The method of claim 25 wherein the auditor interface includes results from a plurality of computing resources.

27. The method of claim 26 wherein the auditor interface includes results from a plurality of customers of the system.

28. The method of claim 20 further comprising receiving permission from an administrator associated with the computing resource to make available at least a portion of the results to a third party.

29. The method of claim 20 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

30. The method of claim 20 further comprising queuing, for operator review, a violation of the computing resource policy regime specification detected during the assessment.

31. The method of claim 20 wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification.

32. The method of claim 31 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

33. The method of claim 32 further comprising receiving, from the administrator, an indication that more time is required by the administrator to remedy the violation.

34. The method of claim 31 further comprising receiving, from the administrator, a request to retest the computing resource with respect to the violation.

35. The method of claim 31 further comprising receiving, from the administrator, an indication that the administrator believes the violation to be a false positive.

36. The method of claim 35 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

37. The method of claim 20 wherein delivering the results includes storing the results in a results repository.

38. The method of claim 20 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

39. A system, comprising:

a processor configured to;

determine, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiate a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatch the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment, and wherein dispatching the assessment includes requesting a plurality of scans from a plurality of third party external scanners; and

deliver results of the assessment; and

a memory coupled to the processor and configured to provide the processor with instructions.

40. The system of claim 39 wherein the assessment is dispatched based at least in part on a determination being made by a timer that the assessment should be dispatched.

41. The system of claim 39 wherein dispatching the assessment includes performing a deduplicating action on the test set.

42. The system of claim 39 wherein delivering the results includes weighting the results.

43. The system of claim 39 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

44. The system of claim 43 wherein the auditor interface includes results from a plurality of computing resources.

45. The system of claim 44 wherein the auditor interface includes results from a plurality of customers of the system.

46. The system of claim 39 wherein the processor is further configured to receive permission from an administrator associated with the computing resource to make available at least a portion of the results to a third party.

47. The system of claim 39 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

48. The system of claim 39 wherein the processor is further configured to queue, for operator review, a violation of the computing resource policy regime specification detected during the assessment.

49. The system of claim 39 wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification.

50. The system of claim 49 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

51. The system of claim 50 wherein the processor is further configured to receive, from the administrator, an indication that more time is required by the administrator to remedy the violation.

52. The system of claim 49 wherein the processor is further configured to receive, from the administrator, a request to retest the computing resource with respect to the violation.

53. The system of claim 49 wherein the processor is further configured to receive, from the administrator, an indication that the administrator believes the violation to be a false positive.

54. The system of claim 53 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

55. The system of claim 39 wherein delivering the results includes storing the results in a results repository.

56. The system of claim 39 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

57. A method, comprising:

determining, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiating a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatching the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment, and wherein dispatching the assessment includes requesting a plurality of scans from a plurality of third party external scanners; and

delivering results of the assessment.

58. The method of claim 57 wherein the assessment is dispatched based at least in part on a determination being made by a timer that the assessment should be dispatched.

59. The method of claim 57 wherein dispatching the assessment includes performing a deduplicating action on the test set.

60. The method of claim 57 wherein delivering the results includes weighting the results.

61. The method of claim 57 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

62. The method of claim 61 wherein the auditor interface includes results from a plurality of computing resources.

63. The method of claim 62 wherein the auditor interface includes results from a plurality of customers of the system.

64. The method of claim 57 further comprising receiving permission from an administrator associated with the computing resource to make available at least a portion of the results to a third party.

65. The method of claim 57 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

66. The method of claim 57 further comprising queuing, for operator review, a violation of the computing resource policy regime specification detected during the assessment.

67. The method of claim 57 wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification.

68. The method of claim 67 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

69. The method of claim 68 further comprising receiving, from the administrator, an indication that more time is required by the administrator to remedy the violation.

70. The method of claim 67 further comprising receiving, from the administrator, a request to retest the computing resource with respect to the violation.

71. The method of claim 67 further comprising receiving, from the administrator, an indication that the administrator believes the violation to be a false positive.

72. The method of claim 71 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

73. The method of claim 57 wherein delivering the results includes storing the results in a results repository.

74. The method of claim 57 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

75. A system, comprising:

a processor configured to;

determine, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiate a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatch the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment; and

deliver results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification, and wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken; and

a memory coupled to the processor and configured to provide the processor with instructions.

76. The system of claim 75 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

77. The system of claim 76 wherein the auditor interface includes results from a plurality of computing resources.

78. The system of claim 77 wherein the auditor interface includes results from a plurality of customers of the system.

79. The system of claim 75 wherein the processor is further configured to receive permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

80. The system of claim 75 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

81. The system of claim 75 wherein the processor is further configured to queue, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

82. The system of claim 75 wherein the processor is further configured to receive, from the administrator, an indication that more time is required by the administrator to remedy the violation.

83. The system of claim 75 wherein the processor is further configured to receive, from the administrator, a request to retest the computing resource with respect to the violation.

84. The system of claim 75 wherein the processor is further configured to receive, from the administrator, an indication that the administrator believes the violation to be a false positive.

85. The system of claim 84 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

86. The system of claim 75 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

87. A method, comprising:

determining, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiating a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatching the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment; and

delivering results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification, and wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

88. The method of claim 87 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

89. The method of claim 88 wherein the auditor interface includes results from a plurality of computing resources.

90. The method of claim 89 wherein the auditor interface includes results from a plurality of customers of the system.

91. The method of claim 87 further comprising receiving permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

92. The method of claim 87 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

93. The method of claim 87 further comprising queuing, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

94. The method of claim 87 further comprising receiving, from the administrator, an indication that more time is required by the administrator to remedy the violation.

95. The method of claim 87 further comprising receiving, from the administrator, a request to retest the computing resource with respect to the violation.

96. The method of claim 87 further comprising receiving, from the administrator, an indication that the administrator believes the violation to be a false positive.

97. The method of claim 96 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

98. The method of claim 87 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

99. A system, comprising:

a processor configured to;

determine, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiate a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatch the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment;

deliver results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification; and

receive, from the administrator, a request to retest the computing resource with respect to the violation; and

a memory coupled to the processor and configured to provide the processor with instructions.

100. The system of claim 99 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

101. The system of claim 100 wherein the auditor interface includes results from a plurality of computing resources.

102. The system of claim 101 wherein the auditor interface includes results from a plurality of customers of the system.

103. The system of claim 99 wherein the processor is further configured to receive permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

104. The system of claim 99 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

105. The system of claim 99 wherein the processor is further configured to queue, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

106. The system of claim 99 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

107. The system of claim 106 wherein the processor is further configured to receive, from the administrator, an indication that more time is required by the administrator to remedy the violation.

108. The system of claim 99 wherein the processor is further configured to receive, from the administrator, an indication that the administrator believes the violation to be a false positive.

109. The system of claim 108 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

110. The system of claim 99 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

111. A method, comprising:

determining, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiating a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatching the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment;

delivering results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification; and

receiving, from the administrator, a request to retest the computing resource with respect to the violation.

112. The method of claim 111 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

113. The method of claim 112 wherein the auditor interface includes results from a plurality of computing resources.

114. The method of claim 113 wherein the auditor interface includes results from a plurality of customers of the system.

115. The method of claim 111 further comprising receiving permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

116. The method of claim 111 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

117. The method of claim 111 wherein further comprising queuing, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

118. The method of claim 111 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

119. The method of claim 118 further comprising receiving, from the administrator, an indication that more time is required by the administrator to remedy the violation.

120. The method of claim 111 further comprising receiving, from the administrator, an indication that the administrator believes the violation to be a false positive.

121. The method of claim 120 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

122. The method of claim 111 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

123. A system, comprising:

a processor configured to;

determine, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiate a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatch the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment; and

deliver results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification; and

receiving, from the administrator, an indication that the administrator believes the violation to be a false positive; and

a memory coupled to the processor and configured to provide the processor with instructions.

124. The system of claim 123 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

125. The system of claim 124 wherein the auditor interface includes results from a plurality of computing resources.

126. The system of claim 125 wherein the auditor interface includes results from a plurality of customers of the system.

127. The system of claim 123 wherein the processor is further configured to receive permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

128. The system of claim 123 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

129. The system of claim 123 wherein the processor is further configured to queue, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

130. The system of claim 123 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

131. The system of claim 130 wherein the processor is further configured to receive, from the administrator, an indication that more time is required by the administrator to remedy the violation.

132. The system of claim 123 wherein the processor is further configured to receive, from the administrator, a request to retest the computing resource with respect to the violation.

133. The system of claim 123 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

134. The system of claim 123 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.

135. A method, comprising:

determining, based at least in part on a computing resource policy regime specification associated with a computing resource, that an assessment of the computing resource should be initiated, wherein the assessment is associated with a plurality of tests;

initiating a pre-scan associated with the computing resource, wherein the pre-scan is configured to assess functionality provided by the computing resource;

dispatching the assessment, wherein dispatching the assessment includes selecting as a test set, based at least in part on the pre-scan, a subset of the tests associated with the assessment;

delivering results of the assessment, wherein delivering the results of the assessment includes alerting an administrator associated with the computing resource of a violation of the computing resource policy regime specification; and

receiving, from the administrator, an indication that the administrator believes the violation to be a false positive.

136. The method of claim 135 wherein delivering the results of the assessment includes delivering the results to an auditor in an auditor interface.

137. The method of claim 136 wherein the auditor interface includes results from a plurality of computing resources.

138. The method of claim 137 wherein the auditor interface includes results from a plurality of customers of the system.

139. The method of claim 135 wherein the processor is further configured to receive permission from the administrator associated with the computing resource to make available at least a portion of the results to a third party.

140. The method of claim 135 wherein delivering the results of the assessment includes making a publicly available scorecard that summarizes the results.

141. The method of claim 135 further comprising queuing, for operator review, the violation of the computing resource policy regime specification detected during the assessment.

142. The method of claim 135 wherein alerting the administrator includes indicating to the administrator an amount of time by which the violation must be remedied before a remedial action is taken.

143. The method of claim 142 further comprising receiving, from the administrator, an indication that more time is required by the administrator to remedy the violation.

144. The method of claim 141 further comprising receiving, from the administrator, a request to retest the computing resource with respect to the violation.

145. The method of claim 144 wherein, in response to receiving the indication that the administrator believes the violation to be a false positive, the violation finding is flagged for review by an operator.

146. The method of claim 135 wherein at least one test included in the test set is a custom test authored on behalf of an owner of the computing resource.