Method for parsing, indexing and searching world-wide-web pages

• US 6,021,409 A
• Filed: 09/08/1998
• Issued: 02/01/2000
• Est. Priority Date: 08/09/1996
• Status: Expired due to Term

• Current Assignee: Eureka Database Solutions, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Burrows, Michael
• Primary Examiner(s): AMSBURY, WAYNE P
• Application Number:
  US09/149,090
• Time in Patent Office: 511 Days
• Field of Search: 707/102, 707/2, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A system for indexing stored information includes a processor and memory. The processor parses the information into indexable words. Each word represents either a portion of the information or an attribute of one or more portions of the information. The memory stores index entries. Each index entry includes a word entry representing a unique one of the words, and one or more location entries indicating a location of the unique word within the information.

Claims

1. A system for indexing stored information, comprising:

a processor configured to parse the stored information into indexable words, each indexable word representing a portion of the stored information [and (ii)] or an attribute of one or more portions of the stored information; and

a memory configured to store index entries, each index entry including a word entry representing a unique one of the indexable words and one or more location entries, each of the one or more location entries indicating a respective location of the unique one of the indexable words within the stored information.

2. The system as recited in claim 1, wherein the stored information is stored as a plurality of pages of information.

3. The system as recited in claim 2, wherein the plurality of pages are Web pages of the INTERNET.

4. The system as recited in claim 2, wherein each of the plurality of pages has a unique URL (universal record locator).

5. The system as recited in claim 4, wherein at least one of the plurality of pages includes a URL link to another of the plurality of pages.

6. The system as recited in claim 2, wherein each of the one or more location entries indicates the respective location of the unique one of the indexable words within the plurality of pages.

7. The system as recited in claim 1, wherein the stored information is stored in at least one storage device.

8. The system as recited in claim 7, wherein the at least one storage device is multiple storage devices connected to each other by a communications network.

9. The system as recited in claim 1, further comprising:

a communication interface configured to receive the stored information.

10. The system as recited in claim 9, wherein the processor is further configured to fetch batches of specified information, and the stored information received by the communication interface is the fetched batches of specified information.

11. The system as recited in claim 10, wherein the specified information is a plurality of pages of the specified information.

12. The system as recited in claim 11, wherein:

the plurality of pages are Web pages of the INTERNET, each of the Web pages having a unique URL (universal record locator) and at least one of the Web pages includes a URL link to another of the Web pages; and

the Web pages are fetched in accordance with the unique URLs and the URL links.

13. The system as recited in claim 1, wherein the processor is further configured to parse the stored information by sequentially partitioning the stored information into the indexable words.

14. The system as recited in claim 1, wherein:

the stored information is a stored plurality of pages of information; and

each indexable word represents a portion of one of the plurality of pages or an attribute of one or more portions of one of the plurality of pages.

15. The system as recited in claim 1, wherein the processor is further configured to fetch the stored information prior to parsing.

16. The system as recited in claim 1, wherein the processor is further configured to sequentially assign locations to the indexable words as they are parsed.

17. The system as recited in claim 1, wherein:

the stored information is stored on the Web of the INTERNET; and

each of the one or more location entries indicates the respective location where the unique one of the indexable words occurs in the Web.

18. The system as recited in claim 1, wherein the processor is further configured to parse a query into terms and operators relating to the terms, to read a location entry of an index entry having a word entry representing one of the terms of the query, and to direct a presentation of segments of the stored information in which the one term is located.

19. The system as recited in claim 18, wherein the stored information is stored as a plurality of pages of information and the segments of the stored information in which the one term is located are those of the plurality of pages in which the one term is located.

20. A method for indexing stored information, comprising the steps of:

parsing the stored information into indexable words, each indexable word representing a portion of the stored information or an attribute of one or more portions of the stored information; and

storing index entries, each index entry including a word entry representing a unique one of the indexable words and one or more location entries, each of the one or more location entries indicating a respective location of the unique one of the indexable words within the stored information.

21. The method as recited in claim 20, wherein the stored information is stored as a plurality of pages of information.

22. The method as recited in claim 21, wherein the plurality of pages are Web pages of the INTERNET.

23. The method as recited in claim 21, wherein each of the one or more location entries indicates the respective location of the unique one of the indexable words within the plurality of pages.

24. The method as recited in claim 20, further comprising the step of:

fetching a batch of the stored information prior to parsing.

25. The method as recited in claim 24, wherein the batch of stored information is a plurality of specified pages of the stored information.

26. The method as recited in claim 25, wherein:

plurality of specified pages are Web pages of the INTERNET, each of the Web pages having a unique URL (universal record locator) and at least one of the Web pages including a URL link to another of the Web pages; and

the Web pages are fetched in accordance with the unique URLs and URL links.

27. The method as recited in claim 20, wherein the stored information is parsed by sequentially partitioning the stored information into the indexable words.

28. The method as recited in claim 20, wherein:

the stored information is in the form of a plurality of pages;

the portion of the stored information is a portion of one of the plurality of pages; and

the attribute of one or more portions of the stored information is an attribute of one or more portions of one of the plurality of pages.

29. The method as recited in claim 20, further comprising the step of:

fetching the stored information prior to parsing.

30. The method as recited in claim 20, further comprising the step of:

sequentially assigning locations to the indexable words as they are parsed.

31. The method as recited in claim 20, wherein:

the stored information is stored on the Web of the INTERNET; and

each of the one or more location entries indicates the respective location where the unique one of the indexable words occurs in the Web.

32. The method as recited in claim 20, further comprising the steps of:

parsing a query into terms and operators relating to the terms;

reading a location entry of an index entry having a word entry representing one of the terms of the query; and

presenting a segment of the stored information in which the one term is located.

33. The method as recited in claim 32, wherein the stored information is in the form of a plurality of pages and the segment of the stored information in which the one term is located is a page within the plurality of pages in which the one term is located.

34. A system for indexing pages of information, the pages of information stored in one or more computers, the system comprising:

a communication interface for receiving a plurality of the pages;

a parser for sequentially partitioning the plurality of received pages into indexable words, each indexable word representing a portion of one received page or an attribute of one or more portions of one received page, the parser sequentially assigning locations to the indexable words as they are parsed;

a memory for storing index entries, each index entry including a word entry representing a unique one of the indexable words and one or more location entries, each of the one or more location entries indicating a location of the unique one of the indexable words within the plurality of received pages;

a query module for parsing a query into terms and operators relating the terms;

a search engine for reading a specified index entry, the specified index entry having a word entry corresponding to at least one of the terms of the query; and

a display module for presenting those of the plurality of pages which are located in accordance with the one or more location entries of the specified index entry.