Method and apparatus for retrieving and analyzing data stored at network sites

• US 6,463,455 B1
• Filed: 12/30/1998
• Issued: 10/08/2002
• Est. Priority Date: 12/30/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Turner, Cameron R., Blumer, Thomas P.
• Primary Examiner(s): Barot, Bharat
• Application Number:
  US09/223,696
• Time in Patent Office: 1,378 Days
• Field of Search: 709/200-203, 709/206-207, 709/217-219, 709/223-226, 709/227-229, 707/1-5, 707/9-10, 707/100-104, 707/500-501, 707/200-205
• US Class Current: 709/200
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/953 Querying, e.g. by the use of web search engines
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A network analyzer is provided that searches all of the sites of a network, such as the Internet, contained in a site list and analyzes the analysis criteria. The site list may be specified by a user, randomly generated, or retrieved from a file. The analysis criteria may be default analysis criteria, retrieved from a file, or created by the user. After the site list and analysis criteria have been specified, a file, such as the HTML file, at each of the specified sites is downloaded and stored as a text file. The site is analyzed for number of occurrences and/or sizes of analysis criteria data. After all of the sites on the site list have been analyzed, the results are displayed. Results of an analysis may be compared to results of a previous analysis in order to track trends.

Claims

1. A method of analyzing data currently stored at network sites, comprising:

(a) maintaining a site list of said network sites to be analyzed remotely from said network sites; and

(b) for each of said network sites in said site list;

(i) obtaining an identity of a network site to be analyzed from said site list;

(ii) obtaining a query criterion;

(iii) retrieving a data file from the network site to be analyzed;

(iv) searching for the query criterion in the retrieved data file; and

(v) storing the results of said search as an analysis of the retrieved data file according to the query criterion.

2. The method of claim 1, wherein the site list comprises a plurality of sites.

3. The method of claim 2, wherein said network sites in said site list are randomly selected.

4. The method of claim 3, wherein a maximum number of randomly selected sites is determined by a user.

5. The method of claim 2, wherein said network sites in said site list are determined by a user.

6. The method of claim 5, further comprising saving said site list.

7. The method of claim 2, wherein the network sites contained in said site list are determined by an existing site list.

8. The method of claim 7, further comprising modifying the existing site list.

9. The method of claim 8, further comprising saving the modified site list.

10. The method of claim 1, further comprising:

(a) obtaining additional query criteria; and

(b) for each of said network sites in said site list;

(i) retrieving a data file from each of said network sites in said site list,

(ii) searching for said additional query criteria in said data file, and

(iii) storing the results of said search as an analysis of said data file according to said additional query criteria.

11. The method of claim 10, wherein the additional query criteria are determined by a user.

12. The method of claim 11, further comprising saving the additional query criteria.

13. The method of claim 10, wherein the additional query criteria are determined by an existing list of query criteria.

14. The method of claim 13, further comprising modifying the existing list of query criteria.

15. The method of claim 14, further comprising saving the modified list of query criteria.

16. The method of claim 10, wherein the additional query criteria are determined by a default list.

17. The method of claim 16, wherein the default list is modifiable by a user.

18. The method of claim 17, further comprising saving the modified default list.

19. The method of claim 1, wherein searching for the query criterion comprises counting occurrences of the query criterion in the data file.

20. The method of claim 1, wherein searching for the query criterion comprises determining the size of the data contained within the data file specified by the query criterion.

21. The method of claim 1, further comprising reporting the analysis of the retrieved data file according to the query criterion to a user.

22. The method of claim 1, further comprising saving the analysis of the retrieved data file according to the query criterion.

23. The method of claim 1, further comprising comparing the analysis of the retrieved data file according to the query criterion to an existing analysis of the query criterion.

24. The method of claim 23, further comprising reporting the differences between the analysis of the retrieved data file according to the query criterion and the existing analysis of the retrieved data file according to the query criterion.

25. The method of claim 1, wherein searching for the query criterion in the data file, comprises:

(a) converting the data file to a text file; and

(b) searching for the query criterion in the text file.

26. The method of claim 25, wherein the text file includes HTML tags and wherein only occurrences of the query criterion located within an HTML tag are analyzed.

27. The method of claim 1, further comprising:

(a) locating all links in the retrieved data file;

(b) retrieving a linked data file identified by each of the links in the retrieved data file;

(c) analyzing the query criterion in each of the linked data files; and

(d) storing the analysis of the query criterion found in the linked data files as part of the analysis of the retrieved data file.

28. The method of claim 27, wherein retrieving a linked data file identified by the links in the data comprises:

(a) determining whether each link is a local or remote link; and

(b) retrieving a linked data file from each of the remote links.

29. A computer-readable medium having computer-executable instructions for performing the method recited in any one of claims 1-28.

30. A computer-controlled apparatus capable of performing the method recited in any one of claims 1-28.