Real time structured summary search engine

• US 6,397,209 B1
• Filed: 09/14/1999
• Issued: 05/28/2002
• Est. Priority Date: 08/30/1996
• Status: Expired due to Term

• Current Assignee: March Networks Corporation (Infinova (Canada) Ltd.)
• Original Assignee: March Networks Corporation (Infinova (Canada) Ltd.)
• Inventors: Reed, Jim, Streatch, Paul
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US09/242,909
• Time in Patent Office: 987 Days
• Field of Search: 707/100,3,4,5,101,102,10,2
• US Class Current: 1/1
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A method of organizing electronic documents for storage and subsequent retrieval, involves storing a summary structure describing the structure of summary records associated with each document. Each structured summary record has at least one descriptor field representative of a characteristic of the document. Predefined field entries identify a characteristic of the document. Predefined keyword criteria associated with the field entries are stored. Each document is analyzed to build a text index listing the occurrence of unique significant words in the document. The text index is compared with the keyword criteria to determine the appropriate field entry for the document. For example, one descriptor field might related to topic, which could have the field entries of “financial” or “sports”. The preponderance of certain keyword criteria, such as “money” or “shares” would identify the document with the financial topic.

Claims

1. A method of processing electronic documents for subsequent retrieval, comprising the steps of:

storing in memory a summary structure database describing the structure of summary records associated with each document, each structured summary record having at least one descriptor field with predefined allowed field entries identifying A characteristic of the document;

for the or each said descriptor field storing in memory respective groups of predefined criteria keywords associated with each of said allowed field entries;

analyzing each document to build a text index listing the occurrence of unique significant words in the document; and

matching said text index with said predefined criteria keywords to determine the appropriate corresponding field entry for the associated descriptor field in accordance with predetermined criteria.

2. A method as claimed in claim 1, wherein the appropriate field entry is selected from the allowed field entries by taking the allowed field entry whose associated group of predefined criteria keywords contains a word with the highest word count in the document.

3. A method as claimed in claim 2, wherein a series of candidate entries corresponding to the field entries of each descriptor field of the summary records are stored in memory, and said candidate entries are examined on an iterative basis to determine the entry having the predefined criteria keywords that correspond to the highest word count in the document.

4. A method as claimed in claim 3, further comprising building an index that maps the criteria keywords to the summary candidates so as to permit the sub-set of candidates applicable to a document to be rapidly determined.

5. A method as claimed in claim 1, wherein said summary structure includes additional fields having unlimited entries.

6. A method as claimed in claim 5, wherein one said additional field comprises a keyword field listing the words in said text index having the highest count.

7. A method as claimed in claim 5, wherein another said additional field comprises an excerpt field listing the sentences in said document containing the words in said text index with the highest count.

8. A method as claimed in claim 1, wherein structured summaries are compared to determine whether two documents have the same content.

9. A method as claimed in claim 1, wherein structured summaries are compared to determined whether two documents have similar content based on a predetermined match of field entries in the documents.

10. A method as claimed in claim 1, wherein said descriptor fields with predefined allowed entries have at least one sub-field arranged in a hierarchical structure.

11. A method as claimed in claim 1, wherein said documents comprises news articles.

12. A method as claimed in claim 11, wherein said news articles are extracted from television broadcasts.

13. A method as claimed in claim 1, wherein said structured summary records also include a ranking field containing a keyword count to permit search hits to be ranked in order of importance.

14. A method as claimed in any one of claim 1, wherein each summary record contains a pointer to the location of the associated document.

15. A system for processing electronic documents for subsequent retrieval, comprising:

a memory storing a summary structure describing the structure of summary records associated with each document, each structured summary record having at least one descriptor field with predefined allowed field entries identifying a characteristic of the document;

a memory storing, for the or each said descriptor field, groups of predetermined criteria keywords associated with each of said respective allowed field entries;

means for analyzing each document to build a text index listing the occurrence of unique significant words in the document; and

means for matching said text index with said criteria keywords to determine the appropriate field entry for the associated descriptor field in accordance with predetermined criteria.