Page information collection program, page information collection method, and page information collection apparatus

• US 7,757,164 B2
• Filed: 02/28/2005
• Issued: 07/13/2010
• Est. Priority Date: 08/17/2004
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yamaoka, Yuji, Nakayama, Yuko, Morikawa, Ikuya, Kojima, Hisashi
• Primary Examiner(s): Paula; Cesar B.
• Assistant Examiner(s): Patel; Manglesh M
• Application Number:
  US11/066,541
• Publication Number:
  US 20060041827A1
• Time in Patent Office: 1,961 Days
• Field of Search: 715/200, 715/234, 715/277, 715206-209, 715253-255, 709201-203, 709/217, 709/219
• US Class Current: 715/234
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 40/134 Hyperlinking

Abstract

A page information collection program for efficiently collecting pages required to verify a web site. When a page acquisition request is input, a page acquisition section acquires the contents of a page and creates page information including the contents of the page and communication information used to acquire the page. Next, a classification section stores the page information put into a group in accordance with an assignment determination condition. If the target page information does not satisfy the assignment determination condition of any group, a grouping section creates an assignment determination condition satisfied by the target page information and a corresponding group, and stores the page information put into the created group. A page acquisition request section outputs a page acquisition request based on the link information in the page information put into the group created by the grouping section, to the page acquisition section.

Claims

1. A computer-readable storage medium having recorded thereon a page information collection program for collecting a set of pages associated by link information from a server on a network, the page information collection program causing a computer to execute the processing of:

acquiring contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and a response status code used for page acquisition, the creating the page information being automatically performed without user interaction;

taking the page information created as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage block, the comparing the assignment determination condition being automatically performed without user interaction;

creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage block the creating the assignment determination condition and the creating the group being automatically performed without user interaction; and

extracting the link information only from the target page information put first into the group created and outputting the page acquisition request for acquiring the page based on the extracted link information, the extracting the link information and the outputting the page acquisition request being automatically performed without user interaction, wherein the link information is not extracted from the target page information put into already existing groups;

wherein the assignment determination condition includes a URL and an option to select a query conformity field;

wherein the assignment determination condition defines a requirement concerning the conformity of the page acquisition request and the response status code given when the contents of the page are acquired, the assignment determination condition is specific to each group, and contents of the assignment determination condition are individually changeable; and

wherein in the comparing the assignment determination condition and the target page information, page information acquired by different page acquisition requests and having different response status codes is determined to belong to different groups.

2. The computer-readable storage medium according to claim 1, causing the computer to execute further the processing of:

creating the new assignment determination condition under a predefined assignment-determination-condition creation rule.

3. The computer-readable storage medium according to claim 2, causing the computer to execute further the processing of:

changing the contents of the assignment-determination-condition creation rule under the control of the user.

4. The computer-readable storage medium according to claim 1, causing the computer to execute further the processing of:

changing the contents of the assignment determination condition under the control of the user.

5. The computer-readable storage medium according to claim 1, wherein the assignment determination condition defines a requirement concerning the conformity of link information included in the contents of the page.

6. The computer-readable storage medium according to claim 5, wherein the requirement concerning the conformity of link information includes a requirement of assignment to the group, concerning a query parameter having a predetermined name, being the conformity in both the query parameter name and the value, and another requirement of assignment to the group, concerning the other query parameters, being the conformity in the query parameter name.

7. The computer-readable storage medium according to claim 1, causing the computer to execute further the processing of:

referencing reacquisition path information indicating the sequence of link information to be followed when acquiring the page information which was acquired before, and reacquiring the page information by following the link information along the path according to the reacquisition path information.

8. The computer-readable storage medium according to claim 7, causing the computer to execute further the processing of:

classifying the page information reacquired, checking whether the page information is put into the same group as the page information acquired before by the same request, and outputting the result.

9. The computer-readable storage medium according to claim 8, causing the computer to execute further the processing of:

reacquiring the page information by following one or more paths from the base page to the page acquired before in response to a shortest-path detection request and determining a path including the smallest number of page acquisitions among the paths by which the page information in the same group was acquired before by the same request, as the reacquisition path.

10. A page information collection method for collecting a set of pages associated by link information, from a server on a network, the page information collection method comprising:

acquiring contents of a page through the network in response to a page acquisition request and creating page information including the contents of the page and a response status code used for page acquisition, the creating the page information being automatically performed without user interaction;

taking the page information created as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage block, the comparing the assignment determination condition being automatically performed without user interaction;

creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage block, the creating the assignment determination condition and the creating the group being automatically performed without user interaction; and

extracting the link information only from the target page information put first into the group created and outputting the page acquisition request for acquiring the page based on the extracted link information, the extracting the link information and the outputting the page acquisition request being automatically performed without user interaction, wherein the link information is not extracted from the target page information put into already existing groups;

wherein the assignment determination condition includes a URL and an option to select a query conformity field;

wherein the assignment determination condition defines a requirement concerning the conformity of the page acquisition request and the response status code given when the contents of the page are acquired, the assignment determination condition is specific to each group, and contents of the assignment determination condition are individually changeable; and

wherein in the comparing the assignment determination condition and the target page information, page information acquired by different page acquisition requests and having different response status codes is determined to belong to different groups.

11. A page information collection apparatus for collecting a set of pages associated by link information, from a server on a network, the page information collection apparatus comprising:

a page acquisition means for acquiring the contents of a page through the network in response to a page acquisition request and creating page information including contents of the page and a response status code used for page acquisition, the creating the page information being automatically performed without user interaction;

a classification means for taking the page information created by the page acquisition means as target page information, comparing an assignment determination condition defining the requirements of page information to be included in each group and the target page information, to find a group having the assignment determination condition satisfied by the target page information, and storing the target page information put into the group in a storage means, the comparing the assignment determination condition being automatically performed without user interaction;

a grouping means for creating an assignment determination condition satisfied by the target page information if the target page information does not satisfy the assignment determination condition of any group, creating a group corresponding to the created assignment determination condition, and storing the target page information put into the created group in the storage means, the creating assignment determination condition and the creating the group being automatically performed without user interaction; and

a page acquisition request means for extracting the link information only from the target page information put first into the group created by the grouping means and outputting the page acquisition request for acquiring the page based on the extracted link information to the page acquisition means, the extracting the link information and the outputting the page acquisition request being automatically performed without user interaction, wherein the link information is not extracted from the target page information put into already existing groups;

wherein the assignment determination condition includes a URL and an option to select a query conformity field;

wherein the assignment determination condition defines a requirement concerning the conformity of the page acquisition request and the response status code given when the contents of the page are acquired, the assignment determination condition is specific to each group, and contents of the assignment determination condition are individually changeable; and

wherein in the comparing the assignment determination condition and the target page information, page information acquired by different page acquisition requests and having different response status codes is determined to belong to different groups.