Apparatus and method for searching and retrieving structured, semi-structured and unstructured content

• US 20040044659A1
• Filed: 05/14/2003
• Published: 03/04/2004
• Est. Priority Date: 05/14/2002
• Status: Abandoned Application

• Current Assignee: Verity Incorporated (Healthstream Incorporated)
• Original Assignee: Verity Incorporated (Healthstream Incorporated)
• Inventors: Toman, Troy, Lok, Patrick, Karsh, Bruce D., Subbaroyan, Ram, Lahiri, Rahul, Judd, Douglass Russell
• Application Number:
  US10/439,338
• Publication Number:
  US 20040044659A1
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/81 Indexing, e.g. XML tags; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/835 Query processing
• CPC Class Codes: G06F 16/8373 Query execution

Abstract

A search and retrieval permits a user to search free text within sections of schema independent documents. The documents, which may include structured, semi-structured, and unstructured documents, contain text organized into a plurality of sections, such as XML tags. The repository of documents is schema independent, such that the search system does not require pre-defined fields for the sections. To execute a search, the search system receives a query that specifies at least one section and at least one free text query construct for text within the section. In general, the free text query construct specifies at least one free text search condition. The search system identifies sections in the repository of documents as specified in the query, and evaluates the free text query construct for the text within sections to determine whether the free text search condition is met.

Claims

1. A method for searching for documents, comprising:

storing a repository of documents, said documents comprising text organized into a plurality of sections;

receiving a query with at least one specified section and at least one free text query construct for text within said specified section, said free text query construct specifying at least one free text search condition;

processing said query to identify said specified section in a group of documents; and

evaluating said free text query construct for said text within said group of documents to determine whether said free text search condition is met.

2. The method of claim 1, wherein:

storing comprises storing documents with corresponding nodes and text associated with said nodes;

receiving comprises receiving a query comprising a node construct for specifying at least one node and said free text query construct; and

processing comprises identifying nodes within said repository of documents that correspond to said node construct.

3. The method of claim 2, wherein:

receiving comprises receiving a location path to identify a node set.

4. The method of claim 1, further comprising returning a document section if said free text search condition is met.

5. The method of claim 1, wherein:

receiving comprises receiving semi-structured documents, said semi-structured documents comprising a plurality of fields with associated data, and at least a portion of said semi-structured documents comprising unstructured free text.

6. The method of claim 1, wherein:

receiving includes receiving information for a plurality of documents, said documents comprising a plurality of structured fields and free text associated with said structured fields, and said documents comprise a plurality of different schemas that define formats for sections of said documents; and

generating an index for said documents, said index for identifying words in said documents and said index comprising information to associate said free text to said structured fields for documents that comprise different schemas.

7. The method of claim 6, wherein:

generating includes generating an offset between said free text and corresponding structured fields.

8. The method of claim 6, wherein:

generating includes generating a start position and an end position to define free text associated with a structured field.

9. The method of claim 6, wherein:

generating includes generating a word count that specifies a number of words associated with a structured field.

10. The method of claim 6, wherein:

receiving includes storing documents with structured fields organized in one or more nodes arranged hierarchically; and

generating includes associating said free text to said structured fields via depth level information for a word corresponding to a level of said word in said hierarchy.

11. A computer readable media, comprising:

a repository of documents with text organized into a plurality of sections; and

executable instructions to

process a query with at least one specified section and at least one free text query construct for text within said specified section, said free text query construct specifying at least one free text search condition;

process said query to identify said specified section in a group of documents; and

evaluate said free text query construct for said text within said group of documents to determine whether said free text search condition is met.

12. The computer readable medium of claim 11,

wherein said repository includes:

documents with corresponding nodes and text associated with said nodes; and

wherein said executable instructions include instructions to;

receive a query comprising a node construct for specifying at least one node and said free text query construct; and

identify nodes within said repository of documents that correspond to said node construct.

13. The computer readable medium of claim 12, wherein said executable instructions include instructions to:

receive a location path to identify a node set.

14. The computer readable medium of claim 11, further comprising executable instructions to return a document section if said free text search condition is met.

15. The computer readable medium of claim 11, wherein said repository includes:

semi-structured documents comprising a plurality of fields with associated data, and at least a portion of said semi-structured documents comprising unstructured free text.

16. The computer readable medium of claim 11,

wherein said repository includes:

documents comprising a plurality of structured fields and free text associated with said structured fields, and said documents comprise a plurality of different schemas that define a format for sections of said documents; and

wherein said executable instructions include instructions to

generate an index for said documents, said index for identifying words in said documents and said index comprising information to associate said free text to said structured fields for documents that comprise different schemas.

17. The computer readable medium of claim 16, wherein said executable instructions include instructions to:

generate an offset between said free text and corresponding structured fields.

18. The computer readable medium of claim 16, wherein said executable instructions include instructions to:

generate a start position and an end position to define free text associated with a structured field.

19. The computer readable medium of claim 16, wherein said executable instructions include instructions to:

generate a word count that specifies a number of words associated with a structured field.

20. The computer readable medium of claim 16,

wherein said repository includes:

documents with structured fields organized in one or more nodes arranged hierarchically; and

wherein said executable instructions include instructions to

associate said free text to said structured fields via depth level information for a word corresponding to a level of said word in said hierarchy.