System and method for enabling a web site robot trap

• US 6,925,465 B2
• Filed: 09/12/2001
• Issued: 08/02/2005
• Est. Priority Date: 09/12/2000
• Status: Expired due to Term

• Current Assignee: Trend Micro Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Lamberton, Marc, Levy-Abegnoli, Eric, Thubert, Pascal
• Primary Examiner(s): Alam, Shahid
• Assistant Examiner(s): Truong, Cam Y T
• Application Number:
  US09/950,729
• Publication Number:
  US 20020046223A1
• Time in Patent Office: 1,420 Days
• Field of Search: 707/9, 707/3, 707/10, 707/6, 713/200, 713/201
• US Class Current: 1/1
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99939 Privileged access

Abstract

The invention allows a system to prevent robots from browsing a Web site beyond a welcome page. When an initial request from an undefined originator is received Web site responds to it with a welcome page including at least one trap. Then, on receiving further requests from the undefined originator Web site can check whether a trap is used or not. If not used the undefined originator is assumed to be a human being and is authorized to go on. If a trap is however used the undefined originator is assumed to be a robot in which case site access is further denied. The invention prevents Web site contents from being investigated by robots while not requiring users to have to log on.

Claims

1. A method for preventing robots from browsing a Web site beyond a welcome page, said method in said Web site comprising the steps of:

on receiving an initial request from an undefined originator to establish a session;

responding to said initial request with a said welcome page including at least one trap;

dynamically monitoring action by the undefined originator on said at least one trap during said session;

on receiving further requests from said undefined originator during said session;

checking whether one of said at least one trap is used or not;

if not used;

assuming that said undefined originator is a human being and keep processing said further requests; and

if used;

assuming that said undefined originator is a robot and stopping processing of said further requests.

2. The method according to claim 1 further including the steps of:

logging a source IP address of said undefined originator and starting a timer.

3. The method according to claim 2 further including, whenever receiving a new request, the steps of:

checking whether a source IP address of said new request is matching said logged source IP address; and

if matching;

checking whether said timer has expired or not; and

if expired;

resetting said logged IP address; and

proceeding normally with said new request;

if not expired;

performing one of dropping and redirecting said TCP connection; and

if not matching;

proceeding normally with said new request; and

repeating said steps with every new received request.

4. The method according to claim 1 wherein said step of stopping processing said further requests includes performing one of dropping and redirecting a TCP connection established with said undefined originator.

5. The method according to claim 1 wherein said trap is an HTML anchor invisible when displayed by a Web browser.

6. The method according to claim 5 wherein said HTML anchor includes an invisible image.

7. The method according to claim 1 wherein every page out of said Web site includes at least one trap.

8. A system for a Web site to implement at least one trap in a Web page for preventing robots from browsing a Web site beyond a welcome page, comprising:

a processor comprising a response component, a trap checking component and processing component;

response component for, on receiving an initial request from an undefined originator to establish a session, responding to said initial request with a welcome page including at least one trap;

trap checking component for, dynamically monitoring action by the undefined originator on said at least one trap during said session, and on receiving further requests from said undefined originator in the same session, checking whether one of said at least one trap is used or not;

processing component for, if said at least one trap is not used, processing said further requests; and

if said at least one trap is used, stopping processing of said further requests.

9. The system of claim 8 further comprising logging means for logging a source IP address of said undefined originator and starting a timer.

10. The system of claim 9 wherein said stopping processing by said processing component comprises:

performing one of dropping and redirecting a TCP connection established with said undefined originator.

11. The system according to claim 8 further including means for checking whether a source IP address of said new request is matching said logged source IP address;

and

if matching;

checking whether said timer has expired or not; and

if expired;

resetting said logged IP address; and

proceeding normally with said new request;

if not expired;

performing one of dropping and redirecting said TCP connection; and

if not matching;

proceeding normally with said new request; and

repeating said steps with every new received request.

12. A computer like readable medium comprising instructions for carrying out a method for preventing robots from browsing a Web site beyond a welcome page, said method in said Web site comprising the steps of:

on receiving an initial request from an undefined originator to establish a session;

responding to said initial request with a welcome page including at least one trap;

monitoring action by the undefined originator on said at least one trap during said session;

on receiving further requests from said undefined originator in the same session;

checking whether one of said at least one trap is used or not;

if not used;

assuming that said undefined originator is a human being and keep processing said further requests; and

if used;

assuming that said undefined originator is a robot and stopping processing of said further requests.