Spatially directed crawling of documents

• US 20050091193A1
• Filed: 11/17/2004
• Published: 04/28/2005
• Est. Priority Date: 02/22/2000
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Metacarta, Inc.
• Inventors: Frank, John R., Rauch, Erik M., Donoghue, Karen
• Granted Patent: US 7,539,693 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

An interface program stored on a computer-readable medium for causing a computer system with a display device to perform the functions of: accepting search criteria from a user including a free text entry query and a domain identifier identifying a domain; in response to accepting the search criteria, retrieving a plurality of record identifiers each of which identifies a corresponding record which: (1) has associated therewith a location identifier that locates it at a specific location within the domain identified by the domain identifier; and (2) contains information that is responsive to the free text entry query; displaying a representation of the domain on the display device; and displaying on the display device a plurality of icons as representations of the records identified by the plurality of record identifiers, wherein for each of the record identifiers, a corresponding one of the plurality of icons is displayed within the representation of the domain that is being displayed on the display device, the corresponding icon for each of the plurality of record identifiers being positioned within the representation of the domain at a coordinate within the domain that corresponds to the location identifier for the corresponding record.

Claims

1-32. -32. (canceled)

33. A method for populating a spatial document database with hyperlinked documents containing spatial information, the method comprising:

providing a destination database containing potential sources of gatherable documents;

providing a history database of known sources where documents have been gathered;

providing a crawler computer process which can follow a hyperlink in a document to access a potential source of gatherable documents specified by the hyperlink;

bootstrapping the crawler;

iterating the crawler over the destination database, including the steps of;

moving a potential source of gatherable documents from the destination database to the history database;

inspecting the potential source for gatherable documents;

storing any such gatherable documents in the spatial document database; and

adding to the destination database all potential sources of gatherable documents which are referenced by a hyperlink in the gatherable documents.

34-36. -36. (canceled)

37. A method for populating a document repository, said method using a page queue for storing document addresses, said method comprising:

retrieving a document address from the page queue;

loading into the document repository a document that is identified by the retrieved document address;

parsing the loaded document for links to new documents;

storing addresses of the new documents into the page queue along with a spatial relevance level for each stored address;

iteratively repeating the steps of retrieving, loading, parsing and storing to populate the document repository, and wherein retrieving involves using the spatial relevance levels of the stored addresses in the page queue to determine which document addresses are retrieved from the page queue.

38. The method of claim 37, wherein iteratively repeating first retrieves document addresses with a relevance level greater than a predetermined threshold.

39. The method of claim 37, further comprising prioritizing the retrieval of document addresses based at least in part of the relevance levels of the document addresses.

40. The method of claim 37, wherein retrieving a document address further comprises changing a parameter associated with that document address to indicate it has been retrieved.

41. The method of claim 37, further comprising, for each loaded document, determining a spatial relevance of that loaded document.

42. The method of claim 41, wherein the spatial relevance of that loaded document is a measure of the degree to which that loaded document relates to spatial locations mentioned within its spatial information content.

43. The method of claim 37, further comprising bootstrapping the document repository before retrieving any document addresses from the page queue.

44. The method of claim 43, wherein bootstrapping involves loading into the document repository a set of documents known to have spatially relevant content.

45. The method of claim 44, wherein bootstrapping further involves scanning each document of the set of documents for links to other document and storing addresses of the other documents in the page queue along with a spatial relevance level for each stored address.

46. The method of claim 43, wherein bootstrapping involves using a metasearcher to identify a set of documents that have spatially relevant content.