System and method for testing web applications with recursive discovery and analysis

• US 20050251863A1
• Filed: 02/11/2005
• Published: 11/10/2005
• Est. Priority Date: 02/11/2004
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Sima, Caleb
• Granted Patent: US 8,566,945 B2
• Time in Patent Office: Days
• US Class Current: 726/25
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A recursive web crawling and analysis tool that includes conducting an initial crawl of a target to identify testable or analyzable objects. The objects are then parsed to identify vulnerabilities, as well as additional objects that can be analyzed. An attack is then launched against the analyzable objects in an effort to break or verify the vulnerabilities. During this attack, additional analyzable objects may be discovered. If such additional objects are discovered, the web crawler is invoked on the additional objects as well, and the results of the crawl are fed back into the parser and attacker functions.

Claims

1. A method for testing web applications with recursive discovery and analysis comprising the steps of:

(a) receiving a target;

(b) crawling the target to discover analyzable objects;

(c) conduct attack sequences on the analyzable objects;

(d) discovering an additional analyzable object during the attack, and continuing at step (b) with the additional analyzable object.

2. The method of claim 1, wherein the target is a URL and the step of crawling the target further comprises parsing the HTML and objects associated with the URL.

3. The method of claim 1, wherein the target is a web application accessible at a URL, and the step of crawling the target involves examining the web application.

4. The method of claim 1, wherein the step of crawling comprises searching the target for links, sessions and files.

5. The method of claim 4, wherein the step of discovering an additional analyzable object comprises discovering an additional link, session or file.

6. The method of claim 5, further comprising the step of providing attack results.

7. A method for testing web applications with recursive discovery and analysis comprising the steps of:

(a) receiving a target;

(b) crawling the target to discover analyzable objects;

(c) conduct attack sequences on the analyzable objects;

(d) discovering additional analyzable objects during the attack, and

(e) continuing at step (b) with each additional analyzable object.

8. The method of claim 7, further comprising a depth limit, and the step of continuing at step (b) with each additional analyzable object is restricted by the depth limit.

9. A system for testing web applications with recursive discovery and analysis, the system comprising:

a sessions table for housing sessions;

a crawler that is operable to;

retrieve information from the sessions table and to examine web items associated with the session information; and

entering additional sessions identified into the sessions table;

a request queue that is operable to;

extract unsent session information from the sessions table and to exercise he web application using the extracted session information; and

receive response information from the web application and enter the response information into the sessions table in a manner to associate the response with the extracted session information;

a parser operative to;

extract session information from the sessions table and search the session information to identify vulnerabilities and sessions; and

enter identified vulnerabilities and sessions into the sessions table; and

an application audit that is operable to attack the web application by applying the session information within the sessions table.

10. The system of claim 9, wherein the operation of the crawler and the request queue can be substantially in parallel.

11. The system of claim 9, wherein the operation of the crawler, the request queue and the parser can be substantially in parallel.

12. The system of claim 9, wherein the operation of the crawler, the request queue, the parser and the application audit can be in parallel.

13. The system of claim 9, wherein the entries into the session table initially include a request, and once the request queue has sent the request, includes any response received from sending the request.

14. The system of claim 9, wherein if the application audit identifies additional sessions, the application audit enters these additional sessions into the session table to be operated on by the crawler.

15. The system of claim 9, wherein if the application audit identifies additional sessions, the application audit enters these additional sessions into the session table to be operated on by the crawler, the request queue and the parser.

16. The system of claim 9, wherein the system includes an interface over which the system can receive an initial entry into the sessions table.

17. The system of claim 9, further comprising a proxy, wherein the proxy can enter additional sessions into the session table.

18. A method for analyzing web based objects using recursive discovery and analysis, the method comprising the steps of:

(a) receiving a web-based address of a target;

(b) crawling the target starting at the web-based address to discover analyzable objects;

(c) parsing each analyzable object to identify a first additional object;

(d) conduct attack sequences on each analyzable objects;

(d) discovering a second additional analyzable object during the attack, and continuing at step (b) with the first and second additional analyzable object.