Page information collection program, page information collection method, and page information collection apparatus

• US 20060041827A1
• Filed: 02/28/2005
• Published: 02/23/2006
• Est. Priority Date: 08/17/2004
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yamaoka, Yuji, Nakayama, Yuko, Morikawa, Ikuya, Kojima, Hisashi
• Granted Patent: US 7,757,164 B2
• Time in Patent Office: Days
• US Class Current: 715/205
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 40/134 Hyperlinking

Abstract

A page information collection program for efficiently collecting pages required to verify a web site. When a page acquisition request is input, a page acquisition section acquires the contents of a page and creates page information including the contents of the page and communication information used to acquire the page. Next, a classification section stores the page information put into a group in accordance with an assignment determination condition. If the target page information does not satisfy the assignment determination condition of any group, a grouping section creates an assignment determination condition satisfied by the target page information and a corresponding group, and stores the page information put into the created group. A page acquisition request section outputs a page acquisition request based on the link information in the page information put into the group created by the grouping section, to the page acquisition section.

Claims

1. A page information collection program for collecting a set of pages associated by link information from a server on a network, the page information collection program causing a computer to execute the processing of:

acquiring the contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and the communication information used for page acquisition, by means of a page acquisition means;

taking the page information created by the page acquisition means as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage means, by means of a classification means;

creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage means, by means of a grouping means; and

extracting the link information from the target page information put into the group created by the grouping means and outputting a request for acquiring the page based on the extracted link information to the page acquisition means, by means of a page acquisition request means.

2. The page information collection program according to claim 1, causing the computer to execute further the processing of:

creating the new assignment determination condition under a predefined assignment-determination-condition creation rule, by means of the grouping means.

3. The page information collection program according to claim 2, causing the computer to execute further the processing of:

changing the contents of the assignment-determination-condition creation rule under the control of the user, by means of an assignment-determination-condition creation rule definition block.

4. The page information collection program according to claim 1, causing the computer to execute further the processing of:

changing the contents of the assignment determination condition under the control of the user, by means of an assignment-determination-condition changing means.

5. The page information collection program according to claim 1, wherein the assignment determination condition defines a requirement concerning the conformity of a response status code given when the contents of the page are acquired.

6. The page information collection program according to claim 1, wherein the assignment determination condition defines a requirement concerning the conformity of link information included in the contents of the page.

7. The page information collection program according to claim 6, wherein the requirement concerning the conformity of link information includes a requirement of assignment to the group, concerning a query parameter having a predetermined name, being the conformity in both the query parameter name and the value, and another requirement of assignment to the group, concerning the other query parameters, being the conformity in the query parameter name.

8. The page information collection program according to claim 1, causing the computer to execute further the processing of:

referencing reacquisition path information indicating the sequence of link information to be followed when acquiring the page information which was acquired before, and reacquiring the page information by following the link information along the path according to the reacquisition path information, by means of the page acquisition means.

9. The page information collection program according to claim 8, causing the computer to execute further the processing of:

classifying the page information reacquired by the page acquisition means, checking whether the page information is put into the same group as the page information acquired before by the same request, and outputting the result, by means of the classification means.

10. The page information collection program according to claim 9, causing the computer to execute further the processing of:

reacquiring the page information by following one or more paths from the base page to the page acquired before in response to a shortest-path detection request and determining a path including the smallest number of page acquisitions among the paths by which the page information in the same group was acquired before by the same request, as the reacquisition path, by means of the page acquisition means.

11. A page information collection method for collecting a set of pages associated by link information, from a server on a network, by means of a computer, the page information collection method comprising the steps of:

acquiring the contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and the communication information used for page acquisition, by means of a page acquisition means;

taking the page information created by the page acquisition means as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage means, by means of a classification means;

creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage means, by means of a grouping means; and

extracting the link information from the target page information put into the group created by the grouping means and outputting a request for acquiring the page based on the extracted link information to the page acquisition means, by means of a page acquisition request means.

12. A page information collection apparatus for collecting a set of pages associated by link information, from a server on a network, the page information collection apparatus comprising:

a page acquisition means for acquiring the contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and the communication information used for page acquisition;

a classification means for taking the page information created by the page acquisition means as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage means;

a grouping means for creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage means; and

a page acquisition request means for extracting the link information from the target page information put into the group created by the grouping means and outputting a request for acquiring the page based on the extracted link information to the page acquisition means.

13. A computer-readable storage medium having recorded thereon a page information collection program for collecting a set of pages associated by link information, from a server on a network, the page information collection program causing a computer to execute the processing of:

acquiring the contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and the communication information used for page acquisition, by means of a page acquisition means;

taking the page information created by the page acquisition means as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage means, by means of a classification means;

creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage means, by means of a grouping means; and

extracting the link information from the target page information put into the group created by the grouping means and outputting a request for acquiring the page based on the extracted link information to the page acquisition means, by means of a page acquisition request means.