Method and apparatus for filtering URL

• US 9,331,981 B2
• Filed: 06/17/2014
• Issued: 05/03/2016
• Est. Priority Date: 12/31/2011
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Jiang, Wu, Xue, Zhihui, Li, Shiguang, Wan, Shiguang
• Primary Examiner(s): Gee, Jason K.
• Assistant Examiner(s): Zhu, Zhimei
• Application Number:
  US14/307,014
• Publication Number:
  US 20140298445A1
• Time in Patent Office: 686 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A method and an apparatus for filtering a uniform resource locator (URL). According to the method, a first category corresponding to a URL connection request can be found in a pre-stored category information table; when the first category conforms to a predetermined URL passing through policy, the URL connection request is allowed to pass through; the URL connection request is forwarded to a corresponding server; a second category corresponding to a URL is determined according to web page content returned by the server; if the second category conforms to the predetermined URL passing through policy, the web page content is sent to a client; if the second category does not conform to the predetermined URL passing through policy, the web page content is blocked. A category to which a URL belongs can be determined in real time, and implementing a function of accurate category filtration.

Claims

1. A method for filtering a uniform resource locator (URL), comprising:

receiving a first URL connection request initiated by a client;

finding, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the first URL connection request;

determining that the first category does not conform to a predetermined URL passing through policy, wherein the URL passing through policy comprises unblocked web page categories;

based upon the determining, sending the first URL connection request to a first server corresponding to the URL carried in the first URL connection request when the first URL connection request carries a predetermined label;

receiving web page content returned by the first server;

determining a second category corresponding to the URL according to the web page content; and

sending the web page content to the client when the determined second category conforms to the predetermined URL passing through policy.

2. The method according to claim 1, further comprising:

receiving a second URL connection request initiated by the client;

finding, among correspondence between web page categories and URLs in the pre-stored category information table, a first category corresponding to a URL carried in the second URL connection request;

determining that the first category corresponding to the URL carried in the second URL connection request conforms to the predetermined URL passing through policy;

sending the second URL connection request to a second server corresponding to the second URL connection request;

receiving web page content returned by the second server;

determining a second category corresponding to the URL carried in the second URL connection request according to the web page content; and

blocking the web page content when the second category corresponding to the URL carried in the second URL connection request does not conform to the predetermined URL passing through policy, wherein determining the second category corresponding to the URL carried in the second URL connection request according to the web page content comprises;

decoding the web page content to extract an identifying keyword of the web page content;

searching for a web page category corresponding to the extracted identifying keyword according to a correspondence between identifying keywords and web page categories stored in a local lexicon list; and

determining the second category corresponding to the URL carried in the second URL connection request is the web page category corresponding to the extracted identifying keyword.

3. The method according to claim 2, wherein the second category corresponding to the URL carried in the second URL connection request does not conform to the predetermined URL passing through policy, and wherein the method further comprises updating the web page category corresponding to the URL carried in the second URL connection request in the pre-stored category information table into the second category.

4. The method according to claim 1, wherein finding the first category corresponding to the URL carried in the first URL connection request comprises:

obtaining the pre-stored category information table from a remote categorizing server and locally buffering the pre-stored category information table; and

finding the first category corresponding to the URL carried in the first URL connection request according to a correspondence between URLs and web page categories in the pre-stored category information table.

5. The method according to claim 1, wherein when the second category conforms to the predetermined URL passing through policy, the method further comprises updating the web page category corresponding to the URL carried in the first URL connection request in the pre-stored category information table into the second category.

6. The method according to claim 1, further comprises:

receiving a third URL connection request initiated by a client;

finding, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the third URL connection request;

determining that the first category corresponding to a URL carried in the third URL connection request does not conform to a predetermined URL passing through policy;

based upon the determining, blocking the third URL connection request when the third URL connection request does not carries a predetermined label.

7. An apparatus for filtering a uniform resource locator (URL), comprising:

a memory;

a processor coupled to the memory and configured to;

receive a first URL connection request initiated by a client;

find, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the first URL connection request;

determine that the first category does not conform to a predetermined URL passing through policy, wherein the URL passing through policy comprises unblocked web page categories;

based upon the determining, send the first URL connection request to a first server corresponding to the URL carried in the first URL connection request when the first URL connection request carries a predetermined label;

receive web page content returned by the first server;

determine a second category corresponding to the URL according to the web page content; and

send the web page content to the client when the determined second category conforms to the predetermined URL passing through policy.

8. The apparatus according to claim 7, wherein the processor is further configured to:

receive a second URL connection request initiated by the client;

find, among correspondence between web page categories and URLs in the pre-stored category information table, a first category corresponding to a URL carried in the second URL connection request;

determine that the first category corresponding to the URL carried in the second URL connection request conforms to the predetermined URL passing through policy;

send the second URL connection request to a second server corresponding to the second URL connection request;

receive web page content returned by the second server;

determine a second category corresponding to the URL carried in the second URL connection request according to the web page content; and

block the web page content when the second category corresponding to the URL carried in the second URL connection request does not conform to the predetermined URL passing through policy,

wherein determining a second category corresponding to the URL carried in the second URL connection request according to the web page content comprises;

decoding the web page content to extract an identifying keyword of the web page content;

searching for a web page category corresponding to the extracted identifying keyword according to a correspondence between identifying keywords and web page categories stored in a local lexicon list; and

determining the second category corresponding to the URL carried in the second URL connection request is the web page category corresponding to the extracted identifying keyword.

9. The apparatus according to claim 8, wherein the processor is further configured to update the web page category corresponding to the URL carried in the second URL connection request in the locally buffered pre-stored category information table into the second category when the second category corresponding to the URL carried in the second URL connection request does not conform to the predetermined URL passing through policy.

10. The apparatus according to claim 7, wherein the processor is further configured to:

obtain the pre-stored category information table from a remote categorizing server and locally buffer the pre-stored category information table; and

find the first category corresponding to the URL carried in the first URL connection request according to a correspondence between URLs and web page categories in the pre-stored category information table.

11. The apparatus according to claim 10, wherein the processor is further configured to update the web page category corresponding to the URL carried in the first URL connection request in the locally buffered category information table into the second category when the second category conforms to the predetermined URL passing through policy.

12. The apparatus according to claim 7, wherein

the processor is configured to:

receive a third URL connection request initiated by a client;

find, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the third URL connection request;

determine that the first category corresponding to a URL carried in the third URL connection request does not conform to a predetermined URL passing through policy;

based upon the determining, block the third URL connection request when the third URL connection request does not carry a predetermined label.

13. An apparatus for filtering a uniform resource locator (URL), comprising:

a processor and a memory coupled to the processor;

wherein the processor is configured to;

receive a URL connection request initiated by a client;

find, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the URL connection request;

determine whether the first category conforms to a predetermined URL passing through policy, wherein the URL passing through policy comprises unblocked web page categories;

block the web page content when the first category does not conform to the predetermined URL passing through policy;

filter blocked URL connection requests to obtain URL connection requests carrying a predetermined label;

send each of the URL connection request obtained by filtering to a server corresponding to the URL connection request; and

receive web page content returned by the server,

determine a second category corresponding to the URL according to the returned web page content; and

determine whether the second category conforms to the predetermined URL passing through policy,

send the web page content to the client when the second category conforms to the predetermined URL passing through policy, and

block the web page content when the second category does not conform to the predetermined URL passing through policy.