Method and system for searching for web content

• US 7,089,233 B2
• Filed: 09/06/2001
• Issued: 08/08/2006
• Est. Priority Date: 09/06/2001
• Status: Expired due to Term

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Osias, Michael James
• Primary Examiner(s): Robinson, Greta
• Assistant Examiner(s): Veillard, Jacques
• Application Number:
  US09/947,866
• Publication Number:
  US 20030046275A1
• Time in Patent Office: 1,797 Days
• Field of Search: 707 2- 5, 707/7, 707/10, 707/102, 707/100, 707/104.1, 709201-203, 709228-229, 709/225, 709217-219, 715/501.1, 715/513, 715/517, 715/744, 715/500, 715/506, 715/522, 715/738, 705/10, 705/102
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/953 Querying, e.g. by the use of web search engines
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

The present invention provides a method and system for searching for web content. Specifically, the present invention provides a system and method for retrieving web content from designated web pages and hyperlinks, indexing the retrieved web content in a local database, and searching the local database for desired web content. Retrieved content is indexed in the local database so that future access of the web content can be more efficient.

Claims

1. A method for searching for web content, comprising the steps of:

a user designating a web address corresponding to a first web page to be searched;

the user designating a hyperlink within the first web page corresponding to a second web page;

retrieving web content for the user from the first web page and from the second web page; and

indexing the retrieved content in a database, wherein the indexing is statistical, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user.

2. The method of claim 1, further comprising the steps of:

searching the database to identify indexed content having terms that match a search term; and

retrieving the identified content from the database.

3. The method of claim 1, further comprising the step of identifying web addresses corresponding to identified content, prior to the retrieving step.

4. The method of claim 1, wherein the searching step is performed according to a predetermined search schedule.

5. The method of claim 1, wherein the retrieving step is performed according to a predetermined retrieval schedule.

6. The method of claim 1, wherein the statistical indexing is based upon an indexing term whereby the web content is indexed in the database according to a frequency that the web content includes terms that match the indexing term.

7. A method for searching fir web content, comprising the steps of:

a user designating a web address corresponding to a first web page to be searched;

the user designating a hyperlink within the first web page corresponding to a second web page;

retrieving web content from the first web page and from the second web page, after the designating steps;

statistically indexing the retrieved web content into a database, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user;

searching the database to identify indexed content having terms that match a search term;

identifying web addresses corresponding to identified web content, after the searching step; and

retrieving the identified content from the database.

8. The method of claim 7, wherein the searching step is performed according to a predetermined search schedule.

9. The method of claim 7, wherein the retrieving step is performed according to a predetermined retrieval schedule.

10. The method of claim 7, wherein the statistical indexing is based upon an indexing term whereby the web content is indexed in the database according to a frequency that the web content includes terms that match the indexing term.

11. A method for searching for web content, comprising the steps of:

a user designating a web address corresponding to a first web page to be searched;

the user designating a hierarchy of hyperlinks in the first web page corresponding to additional web pages;

retrieving web content from the first and additional web pages according to a predetermined retrieval schedule;

statistically indexing the retrieved content in a database, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user;

searching the database to identify indexed content having terms that match a search term; and

retrieving identified content from the database.

12. The method of claim 11, wherein the retrieving step is performed according to a retrieval schedule, and wherein the searching step is performed according to a searching schedule.

13. A method for searching for web content, comprising the steps of:

a user configuring a web crawler by designating a web address to be searched, a hyperlink within a web page corresponding to the web address, and a retrieval schedule;

initiating the web crawler to retrieve web content from the web pages at the designated web address and from a web page corresponding to the hyperlink according to the retrieval schedule;

indexing the retrieved content in a database, wherein the indexing is statistical, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user;

searching the database to identify content having terms that match a search term; and

retrieving the identified content from the database.

14. The method of claim 13, wherein the statistical indexing is based upon an indexing term whereby the web content is indexed in the database according to a frequency that the web content includes terms that match the indexing term.

15. A system for searching for web content, comprising:

a configuration system for a user to designate a specific web address corresponding to a first web page to be searched and for the user to designate a specific hyperlink in the first web page corresponding to a second web page;

a retrieval system for retrieving web content from the first and the second web pages; and

an indexing system for indexing the retrieved web content in a database, wherein the indexing is statistical, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user.

16. The system of claim 15, further comprising a search system for identifying and retrieving indexed web content in the database having terms that match a search term.

17. The system of claim 15, wherein the statistical indexing is based upon an indexing term whereby the web content is indexed in the database according to a frequency that the web content includes terms that match the indexing term.

18. A system for searching for web content, comprising:

a configuration system for a user to designate a specific web address corresponding to a first web page to be searched and for the user to designate a hierarchy of hyperlinks in the first web page corresponding to additional web pages;

a retrieval system for retrieving web content from the first and the second web pages according to a designated retrieval schedule; and

an indexing system for statistically indexing the retrieved web content in a database, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user; and

a search system for identifying and retrieving indexed web content having terms that match a search term from the database.

19. A program product stored on a recordable medium for searching for web content, comprising:

a configuration system, for a user to designate a specific web address corresponding to a first web page to be searched and for the user to designate a specific hyperlink in the first web page corresponding to a second web page;

a retrieval system, for retrieving web content from the first and the second web pages; and

an indexing system for indexing the retrieved web content in a database according to an indexing criterion wherein the indexing is statistical, wherein the statistical indexing is based upon a behavioral history of the user whereby the web content is indexed in the database according to a frequency of previous access by the user.

20. The program product of claim 19, further comprising a search system for identifying and retrieving indexed web content in the database having terms that match a search term.

21. The program product of claim 19, wherein the statistical indexing is based upon an indexing term whereby the web content is indexed in the database according to a frequency that the web content includes terms that match the indexing term.