Method and apparatus for automatic information filtering using URL hierarchical structure and automatic word weight learning

• US 6,976,070 B1
• Filed: 02/14/2000
• Issued: 12/13/2005
• Est. Priority Date: 02/16/1999
• Status: Expired due to Fees

• Current Assignee: KDD Corporation (KDDI Corporation)
• Original Assignee: KDD Corporation (KDDI Corporation)
• Inventors: Inoue, Naomi, Hashimoto, Kazuo, Hoashi, Keiichirou
• Primary Examiner(s): Caldwell, Andrew
• Assistant Examiner(s): Nguyen, Hai V.
• Application Number:
  US09/503,757
• Time in Patent Office: 2,129 Days
• Field of Search: 709/217, 709/219, 709/203, 709/224, 709/229, 709/225, 713/201, 713/200, 707/5, 707/9, 707/100, 707/104.1, 715/716
• US Class Current: 709/224
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

Disclosed method and apparatus for automatic information filtering are capable of improving both precision and recall and accurately judging inappropriateness of the content even for a page that contains very few or no text information and only displays images, by utilizing an upper level URL of a URL given in a hierarchical structure. Also, disclosed method and apparatus for automatic information filtering are capable of setting weights of words easily and accurately and judging inappropriateness of the information by utilizing these weights, by using an automatic learning based on a linear discrimination function that can discriminate the inappropriate information and the appropriate information on a vector space.

Claims

1. A method of automatic information filtering for identifying inappropriate information among various information provided through Internet and blocking presentation of identified inappropriate information, comprising the steps of:

entering an HTML (HyperText Markup Language) information provided through the Internet;

judging whether a URL (Uniform Resource Locator) of said HTML information entered from the Internet is a top page URL or not, the top page URL being a URL ending with a prescribed character string defining according to a URL hierarchical structure by which each URL is constructed;

extracting words appearing in information indicated by the top page URL and carrying out an automatic filtering to judge whether said information indicated by the top page URL is inappropriate or not according to the words extracted from said information indicated by the top page URL, when said URL of said HTML information is the top page URL;

registering an upper level URL derived from the top page URL into an inappropriate upper level URL list and blocking presentation of said information indicated by the top page URL, when said information indicated by the top page URL is judged as inappropriate by the automatic filtering, the upper level URL being derived from the top page URL by keeping a character string constituting the top page URL only up to a rightmost slash;

comparing said URL of said HTML information with each URL registered in the inappropriate upper level URL list and judging whether there is any matching URL in the inappropriate upper level URL list when said URL of said HTML information is not the top page URL, and blocking presentation of information indicated by said URL of said HTML information when there is a matching URL in the inappropriate upper level URL list, the matching URL being one upper level URL whose character string is contained in said URL of said HTML information;

extracting words appearing in said information indicated by said URL of said HTML information, and carrying out the automatic filtering to judge whether said information indicated by said URL of said HTML information is inappropriate or not according to the words extracted from said information indicated by said URL of said HTML information, when there is no matching URL in the inappropriate upper level URL list; and

blocking presentation of said information indicated by said URL of said HTML information when said information indicated by said URL of said HTML information is judged as inappropriate by the automatic filtering.

2. The method of claim 1, further comprising the steps of:

registering in advance URLs that provide inappropriate information in an inappropriate URL list; and

carrying out a third part rating based filtering for comparing said URL of said HTML information with each URL registered in the inappropriate URL list and judging whether there is any matching URL in the inappropriate URL list, and blocking presentation of said information indicated by said URL of said HTML information when there is a matching URL in the inappropriate URL list.

3. An automatic information filtering apparatus for identifying inappropriate information among various information provided through Internet and blocking presentation of identified inappropriate information, comprising:

an input unit for entering an HTML (HyperText Markup Language) information provided through the Internet;

a top page URL judging unit for judging whether a URL (Uniform Resource Locator) of said HTML information entered from the Internet is a top page URL or not, the top page URL being a URL ending with a prescribed character string defined according to a URL hierarchical structure by which each URL is constructed;

a first automatic filtering unit for extracting words appearing in information indicated by the top page URL and carrying out an automatic filtering to judge whether said information indicated by the top page URL is inappropriate or not according to the words extracted from said information indicated by the top page URL, when said URL of said HTML information is the top page URL;

an inappropriate upper level URL list registration unit for registering an upper level URL derived from the top page URL into an inappropriate upper level URL list and blocking presentation of said information indicated by the top page URL is judged as inappropriate by the automatic filtering, the upper level URL being derived from the top page URL by keeping a character string constituting the top page URL only up to a rightmost slash;

an inappropriate URL judging unit for comparing said URL of said HTML information with each URL registered in the inappropriate upper level URL list and judging whether there is any matching URL in the inappropriate upper level URL list when said URL of said HTML information is not the top page URL, and blocking presentation of information indicated by said URL of said HTML information when there is a matching URL in the inappropriate upper level URL list, the matching URL being one upper level URL whose character string is contained in said URL of said HTML information;

a second automatic filtering unit for extracting words appearing in said information indicated by said URL of said HTML information, and carrying out the automatic filtering to judge whether said information indicated by said URL of said HTML information is inappropriate or not according to the words extracted from said information indicated by said URL of said HTML information, when there is no matching URL in the inappropriate upper level URL list; and

an information presentation blocking unit for blocking presentation of said information indicated by said URL of said HTML information is judged as inappropriate by the automatic filtering.

4. The apparatus of claim 3, further comprising:

an inappropriate URL list registration unit for registering in advance URLs that provide inappropriate information in an inappropriate URL list; and

a third party rating based filtering unit for carrying out a third part rating based filtering for comparing said URL of said HTML information with each URL registered in the inappropriate URL list and judging whether there is any matching URL in the inappropriate URL list, and blocking presentation of said information indicated by said URL of said HTML information when there is a matching URL in the inappropriate URL list.

5. A computer usable medium having computer readable program codes embodied therein for causing a computer to function as an automatic information filtering apparatus for identifying inappropriate information among various information provided through Internet and blocking presentation of identified inappropriate information, the computer readable program codes include:

a first computer readable program code for causing said computer to enter an HTML (HyperText Markup Language) information provided through the Internet;

a second computer readable program code for causing said comptuer to judge whether a URL (Uniform Resource Locator) of said HTML information entered from the Internet is a top page URL or not, the top page URL being a URL ending with a prescribed character string defined according to the URL hierarchical structure by which each URL is constructed;

a third computer readable program code for causing said computer to extract words appearing in information indicated by the top page URL and carry out an automatic filtering to judge whether said information indicated by the top page URL is inappropriate or not according to the words extracted from said information indicated by the top page URL, when said URL of said HTML information is the top page URL;

a fourth computer readable program code for causing said computer to register an upper level URL derived from the top page URL into an inappropriate upper level URL list and block presentation of said information indicated by the top page URL, when said information indicated by the top page URL is judged as inappropriate by the automatic filtering, the upper level URL being derived from the top page URL by keeping a character string constituting the top page URL only up to a rightmost slash;

a fifth computer readable program code for causing said computer to compare said URL of said HTML information with each URL registered in the inappropriate upper level URL list and judge whether there is any matching URL in the inappropriate upper level URL list when said URL of said HTML information is not the top page URL, and block presentation of information indicated by said URL of said HTML information when there is a matching URL in the inappropriate upper level URL list, the matching URL being one upper level URL whose character string is contained in said URL of said HTML information;

a sixth computer readable program code for causing said computer to extract words appearing in said information indicated by said URL of said HTML information, and carry out the automatic filtering to judge whether said information indicated by said URL of said HTML information is inappropriate or not according to the words extracted from said information indicated by said URL of said HTML information, when there is no matching URL in the inappropriate upper level URL list; and

a seventh computer readable program code for causing said computer to block presentation of said information indicated by said URL of said HTML information when said information indicated by said URL of said HTML information is judged as inappropriate by the automatic filtering.

6. The computer usable medium of claim 5, wherein the computer readable program codes further include:

an eighth computer readable program code for causing said computer to register in advance URLs that provide inappropriate information in an inappropriate URL list; and

a ninth computer readable program code for causing said computer to carry out a third part rating based filtering for comparing said URL of said HTML information with each URL registered in the inappropriate URL list and judging whether there is any matching URL in the inappropriate URL list, and blocking presentation of said information indicated by said URL of said HTML information when there is a matching URL in the appropriate URL list.