Method and Apparatus for Filtering URL

• US 20140298445A1
• Filed: 06/17/2014
• Published: 10/02/2014
• Est. Priority Date: 12/31/2011
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Jiang, Wu, Li, Shiguang, Xue, Zhihui, Wan, Shiguang
• Granted Patent: US 9,331,981 B2
• Time in Patent Office: Days
• US Class Current: 726/13
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A method and an apparatus for filtering a uniform resource locator (URL). According to the method, a first category corresponding to a URL connection request can be found in a pre-stored category information table; when the first category conforms to a predetermined URL passing through policy, the URL connection request is allowed to pass through; the URL connection request is forwarded to a corresponding server; a second category corresponding to a URL is determined according to web page content returned by the server; if the second category conforms to the predetermined URL passing through policy, the web page content is sent to a client; if the second category does not conform to the predetermined URL passing through policy, the web page content is blocked. A category to which a URL belongs can be determined in real time, and implementing a function of accurate category filtration.

Claims

1. A method for filtering a uniform resource locator (URL), comprising:

receiving a URL connection request initiated by a client;

finding, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the URL connection request;

determining whether the first category conforms to a predetermined URL passing through policy, wherein the URL passing through policy comprises unblocked web page categories;

sending the URL connection request to a server corresponding to the URL connection request when the first category conforms to the predetermined URL passing through policy;

receiving web page content returned by the server;

determining a second category corresponding to the URL according to the web page content;

determining whether the second category conforms to the predetermined URL passing through policy;

sending the web page content to the client when the second category conforms to the predetermined URL passing through policy; and

blocking the web page content when the second category does not conform to the predetermined URL passing through policy.

2. The method according to claim 1, wherein determining the second category corresponding to the URL according to the web page content comprises:

decoding the web page content to extract an identifying keyword of the web page content;

searching for a web page category corresponding to the extracted identifying keyword according to a correspondence between identifying keywords and web page categories stored in a local lexicon list;

determining the second category corresponding to the URL is the web page category corresponding to the extracted identifying keyword.

3. The method according to claim 1, wherein finding the first category corresponding to a URL carried in the URL connection request comprises:

obtaining the pre-stored category information table from a remote categorizing server and locally buffering the pre-stored category information table; and

finding the first category corresponding to the URL carried in the URL connection request according to a correspondence between URLs and web page categories in the category information table.

4. The method according to claim 1, wherein the second category does not conform to the predetermined URL passing through policy, and wherein the method further comprises updating the web page category corresponding to the URL carried in the URL connection request in the pre-stored category information table into the second category.

5. The method according to claim 1, wherein the first category does not conform to the predetermined URL passing through policy, and wherein the method further comprises:

blocking the URL connection request;

filtering blocked URL connection requests to obtain a URL connection request carrying a predetermined label;

sending the URL connection request obtained by filtering to a server corresponding to the URL connection request, and receiving web page content returned by the server;

determining the second category corresponding to the URL according to the returned web page content;

determining whether the second category conforms to the predetermined URL passing through policy;

sending the web page content to the client when the second category conforms to the predetermined URL passing through policy;

blocking the web page content when the second category does not conform to the predetermined URL passing through policy.

6. The method according to claim 1, wherein the first category does not conform to the predetermined URL passing through policy, and wherein the method further comprises:

determining whether the URL connection request carries a predetermined label;

sending the URL connection request to a server corresponding to the URL connection request when the URL connection request carries a predetermined label;

receiving web page content returned by the server;

determining the second category corresponding to the URL according to the web page content;

determining whether the second category conforms to the predetermined URL passing through policy;

sending the web page content to the client when the second category conforms to the predetermined URL passing through policy;

blocking the web page content when the second category does not conform to the predetermined URL passing through policy; and

blocking the URL connection request when the URL connection request does not carry a predetermined label.

7. The method according to claim 5, wherein the second category conforms to the predetermined URL passing through policy, and wherein the method further comprises updating the web page category corresponding to the URL carried in the URL connection request in the pre-stored category information table into the second category.

8. The method according to claim 6, wherein the second category conforms to the predetermined URL passing through policy, and wherein the method further comprises updating the web page category corresponding to the URL carried in the URL connection request in the pre-stored category information table into the second category.

9. The method according to claim 2, wherein the first category does not conform to the predetermined URL passing through policy, and wherein the method further comprises:

blocking the URL connection request;

filtering blocked URL connection requests to obtain a URL connection request carrying a predetermined label;

sending the URL connection request obtained by filtering to a server corresponding to the URL connection request;

receiving web page content returned by the server;

determining the second category corresponding to the URL according to the returned web page content;

determining whether the second category conforms to the predetermined URL passing through policy;

sending the web page content to the client when the second category conforms to the predetermined URL passing through policy; and

blocking the web page content when the second category does not conform to the predetermined URL passing through policy.

10. The method according to claim 2, wherein the first category does not conform to the predetermined URL passing through policy, and wherein the method further comprises:

determining whether the URL connection request carries a predetermined label;

sending the URL connection request to a server corresponding to the URL connection request when the URL connection request carries a predetermined label;

receiving web page content returned by the server;

determining the second category corresponding to the URL according to the web page content;

determining whether the second category conforms to the predetermined URL passing through policy;

sending the web page content to the client when the second category conforms to the predetermined URL passing through policy;

blocking the web page content if the second category does not conform to the predetermined URL passing through policy; and

blocking the URL connection request if the URL connection request does not carry a predetermined label.

11. An apparatus for filtering a uniform resource locator (URL), comprising:

a request receiving module configured to receive a URL connection request initiated by a client;

a first category obtaining module configured to find, among correspondence between web page categories and URLs in a pre-stored category information table, a first category corresponding to a URL carried in the URL connection request;

a passing through determining module configured to determine whether the first category conforms to a predetermined URL passing through policy, wherein the URL passing through policy comprises unblocked web page categories;

a request sending module configured to;

send the URL connection request to a server corresponding to the URL connection request when the first category conforms to a predetermined URL passing through policy; and

receive web page content returned by the server;

a category determining module configured to;

determine a second category corresponding to the URL according to the web page content; and

determine whether the second category conforms to the predetermined URL passing through policy;

a content returning module configured to send the web page content to the client when the second category conforms to the predetermined URL passing through policy; and

a blocking module configured to block the web page content when the second category does not conform to the predetermined URL passing through policy.

12. The apparatus according to claim 11, wherein the category determining module comprises:

a web page decoding submodule configured to decode the web page content to extract an identifying keyword of the web page content;

a second category determining submodule configured to;

search for a web page category corresponding to the extracted identifying keyword according to a correspondence between identifying keywords and web page categories stored in a local lexicon list; and

determine the second category corresponding to the URL is the web page category corresponding to the extracted identifying keyword; and

a category determining submodule configured to determine whether the second category identified by the second category determining submodule conforms to the URL passing through policy.

13. The apparatus according to claim 11, wherein the first category obtaining module further comprises:

an information table obtaining submodule configured to obtain the prestored category information table from a remote categorizing server and locally buffer the prestored category information table; and

a first category determining submodule configured to find the first category corresponding to the URL carried in the URL connection request according to a correspondence between URLs and web page categories in the category information table.

14. The apparatus according to claim 11, further comprising:

a category updating module configured to update the web page category corresponding to the URL carried in the URL connection request in the locally buffered category information table into the second category when the category determining module determines that the second category does not conform to the predetermined URL passing through policy.

15. The apparatus according to claim 11, wherein the blocking module is further configured to block the URL connection request when the passing through determining module determines that the first category does not conform to the predetermined URL passing through policy, wherein the apparatus further comprises a label filtering module configured to filter blocked URL connection requests to obtain URL connection requests carrying a predetermined label each, wherein the request sending module is configured to:

send the URL connection request to a server corresponding to the URL connection request for each of the URL connection requests obtained by filtering; and

receive web page content returned by the server,

wherein the category determining module is configured to;

determine a second category corresponding to the URL according to the returned web page content; and

determine whether the second category conforms to the predetermined URL passing through policy;

wherein the content returning module sends the web page content to the client the second category conforms to the predetermined URL passing through policy, and

wherein the blocking module blocks the web page content when the second category does not conform to the predetermined URL passing through policy.

16. The apparatus according to claim 11, further comprising a label determining module configured to:

determine whether the URL connection request carries a predetermined label when the determination result of the passing through determining module is no; and

trigger the request sending module to send the URL connection request to a server corresponding to the URL connection request and receive web page content returned by the server when the URL connection request carries a predetermined label,

wherein the category determining module is configured to;

determine a second category corresponding to the URL according to the web page content; and

determine whether the second category conforms to the predetermined URL passing through policy,

wherein the content returning module sends the web page content to the client when the second category conforms to the predetermined URL passing through policy,

wherein the blocking module is triggered to block the web page content when the second category does not conform to the predetermined URL passing through policy, and

wherein the blocking module is triggered to block the URL connection request when the URL connection request does not carry a predetermined label.

17. The apparatus according to claim 11, further comprising a category updating module configured to update the web page category corresponding to the URL carried in the URL connection request in the locally buffered category information table into the second category when the category determining module determines that the second category conforms to the predetermined URL passing through policy.

18. The apparatus according to claim 12, wherein the blocking module is further configured to block the URL connection request when the passing through determining module determines that the first category does not conform to the predetermined URL passing through policy, wherein the apparatus further comprises a label filtering module configured to filter blocked URL connection requests to obtain URL connection requests carrying a predetermined label, wherein the request sending module is configured to:

send the URL connection request to a server corresponding to the URL connection request for each of the URL connection requests obtained by filtering; and

receive web page content returned by the server,

wherein the category determining module is configured to;

determine a second category corresponding to the URL according to the returned web page content; and

determine whether the second category conforms to the predetermined URL passing through policy,

wherein the content returning module sends the web page content to the client when the second category conforms to the predetermined URL passing through policy, and

wherein the blocking module blocks the web page content when the second category does not conform to the predetermined URL passing through policy.

19. The apparatus according to claim 12, further comprising a label determining module configured to:

determine whether the URL connection request carries a predetermined label when the determination result of the passing through determining module is no; and

trigger the request sending module to send the URL connection request to a server corresponding to the URL connection request and receive web page content returned by the server when the URL connection request carries a predetermined label,

wherein the category determining module is configured to;

determine a second category corresponding to the URL according to the web page content; and

determine whether the second category conforms to the predetermined URL passing through policy,

wherein the content returning module sends the web page content to the client when the second category conforms to the predetermined URL passing through policy,

wherein the blocking module is triggered to block the web page content when the second category does not conform to the predetermined URL passing through policy, and

wherein the blocking module is triggered to block the URL connection request when the URL connection request does not carry a predetermined label.