Object oriented information retrieval framework mechanism

• US 6,457,018 B1
• Filed: 05/19/1998
• Issued: 09/24/2002
• Est. Priority Date: 04/30/1996
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rubin, Bradley Scott
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US09/081,174
• Time in Patent Office: 1,589 Days
• Field of Search: 707/103, 707/5, 707/4, 707/100
• US Class Current: 707/742
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: Y10S 707/913 Multimedia
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A framework for use with object-oriented programming systems provides a reusable object oriented (OO) framework for use with object oriented programming systems that provides an information retrieval (IR) shell that permits a framework user to define an index class that includes word index objects and provides an extensible information retrieval system that evaluates a user query by comparing information contained in the user query with information contained in the word index objects that relates to stored documents. The information in word index objects is produced by preprocessing operations on documents such that the documents relevant to the user query will be identified, thereby providing a query result. The information retrieval system user can load documents into the computer system storage, index documents so their information can be subject to a query search, and request query evaluation to identify and retrieve documents most closely related to the subject matter of a user query.

Claims

1. A computer system comprising:

a central processing unit;

a user interface; and

a main memory having an operating system that supports an object oriented programming environment containing a framework that provides an extensible information retrieval system that evaluates a user query by comparing information contained in the user query with information contained in one or more documents stored in the computer system, such that the documents relevant to the user query will be identified, thereby providing a query result, wherein the framework includes;

at least one object oriented index class object that includes at least one word index object that maps each word contained in a document to the document containing the word by performing a preprocessing operation on the document to generate the at least one word index object; and

at least one object oriented query index object that processes a user query so as to produce a query result from comparison of the user query and the word index objects in response to a user query.

2. A computer system as defined in claim 1, wherein the framework includes an object oriented build index object that processes a document so as to create the word index objects in response to a user build index request.

3. A computer system as defined in claim 1, wherein the framework includes an object oriented posting list class of objects that contain a frequency count that indicates the number of times a word appears in a word index object of a document.

4. A computer system as defined in claim 1, wherein the framework includes an object oriented document table class of objects that map a word index object to the indexed document from which it was preprocessed.

5. A computer system as defined in claim 1, wherein the preprocessing operation includes a parsing method that identifies text words from other text characters.

6. A computer system as defined in claim 1, wherein the preprocessing operation includes a stoplist method that identifies text words not containing sufficient information to be useful in providing a query result and deletes such text words.

7. A computer system as defined in claim 1, wherein the preprocessing operation includes a stemming method that identifies text word stems of which a text word is a formative, and replaces the text word with the stem.

8. A computer system as defined in claim 1, wherein the documents are stored in the computer system as instances of an object oriented extensible binary-large-object class having document objects containing text information and binary document objects containing a digital representation of information other than text.

9. A computer system as defined in claim 8, wherein each binary document object is linked to a document object containing text information relating to the information other than text that is contained in the binary document object.

10. A computer system as defined in claim 8, wherein the digital representation of information other than text comprises image data, video data, or audio data.

11. A computer system as defined in claim 1, wherein the framework includes an object oriented load document object that stores documents into the memory of the system in response to a user storage request.

12. An object oriented framework for use in a computer system having an operating system that supports an object oriented programming environment that defines an index class that includes word index objects and provides an extensible information retrieval system that evaluates a user query by comparing information contained in the user query with information contained in the word index objects relating to documents stored in the computer system, wherein the information in each word index object is produced by a preprocessing operation on a corresponding document, such that the documents relevant to the user query will be identified, thereby providing a query result, wherein the framework includes:

at least one object oriented index class object that includes at least one word index object that maps each word contained in a document to the document containing the word by performing a preprocessing operation on the document to generate the at least one word index object; and

at least one object oriented query index object that processes a user query to produce a query result from comparison of the user query and the word index objects in response to a user query.

13. An object oriented framework as defined in claim 12, wherein the framework includes an object oriented build index object that processes a document so as to create the word index objects in response to a user build index request.

14. An object oriented framework as defined in claim 12, wherein the framework includes an object oriented posting list class of objects that contain a frequency count that indicates the number of times a word appears in a word index object of a document.

15. An object oriented framework as defined in claim 12, wherein the framework includes an object oriented document table class of objects that map a word index object to the indexed document from which it was preprocessed.

16. An object oriented framework as defined in claim 12, wherein the preprocessing operation of the information retrieval system includes a parsing method that identifies text words from other text characters.

17. An object oriented framework as defined in claim 12, wherein the preprocessing operation of the information retrieval system includes a stoplist method that identifies text words not containing sufficient information to be useful in providing a query result and deletes such text words.

18. An object oriented framework as defined in claim 12, wherein the preprocessing operation of the information retrieval system includes a stemming method that identifies text word stems of which a text word is a formative, and replaces the text word with the stem.

19. An object oriented framework as defined in claim 12, wherein the documents are stored in the computer system as instances of an object oriented extensible binary-large-object class having document objects containing text information and binary document objects containing a digital representation of information other than text.

20. An object oriented framework as defined in claim 19, wherein each binary document object is linked to a document object containing text information relating to the information other than text that is contained in the binary document object.

21. An object oriented framework as defined in claim 19, wherein the digital representation of information other than text comprises image data, video data, or audio data.

22. An object oriented framework as defined in claim 12, wherein the framework includes an object oriented load document object that stores documents into the memory of the system in response to a user storage request.

23. A program product for use in a computer system having an operating system that supports an object-oriented programming environment, the program product comprising:

a signal bearing media; and

a framework recorded on the signal bearing media, the framework providing an extensible information retrieval system that evaluates a user query by comparing information contained in the user query with information contained in object oriented index class objects that relate to one or more documents stored in the computer system, such that the documents relevant to the user query will be identified, thereby providing a query result, wherein the framework includes;

at least one object oriented index class object that includes at least one word index object that maps each word contained in a document to the document containing the word by performing a preprocessing operation on the document to generate the at least one word index object; and

at least one object oriented query index object that processes a user query to produce a query result from comparison of the user query and the word index objects in response to a user query.

24. A program product as defined in claim 23, wherein the framework includes an object oriented build index object that processes a document so as to create the word index objects in response to a user build index request.

25. A program product as defined in claim 23, wherein the framework includes an object oriented posting list class of objects that contain a frequency count that indicates the number of times a word appears in a word index object of a document.

26. A program product as defined in claim 23, wherein the framework includes an object oriented document table class of objects that map a word index object to the indexed document from which it was preprocessed.

27. A program product as defined in claim 23, wherein the preprocessing operation includes a parsing method that identifies text words from other text characters.

28. A program product as defined in claim 23, wherein the preprocessing operation includes a stoplist method that identifies text words not containing sufficient information to be useful in providing a query result and deletes such text words.

29. A program product as defined in claim 23, wherein the preprocessing operation includes a stemming method that identifies text word stems of which a text word is a formative, and replaces the text word with the stem.

30. A program product as defined in claim 23, wherein the documents are stored in the computer system as instances of an object oriented extensible binary-large-object class having document objects containing text information and binary document objects containing a digital representation of information other than text.

31. A program product as defined in claim 30, wherein each binary document object is linked to a document object containing text information relating to the information other than text that is contained in the binary document object.

32. A program product as defined in claim 30, wherein the digital representation of information other than text comprises image data, video data, or audio data.

33. A program product as defined in claim 23, wherein the framework includes an object oriented load document object that stores documents into the memory of the system in response to a user storage request.

34. A program product as defined in claim 23, wherein the signal bearing media comprises recordable media.

35. A program product as defined in claim 23, wherein the signal bearing media comprises transmission media.

36. A method of executing an application program in a computer system having a central processing unit that controls processing in the computer system, a user interface, and a main memory having an operating system that supports an object oriented programming environment, the method comprising the steps of:

providing an object oriented framework that provides an extensible information retrieval system, the framework including at least one core function that cannot be modified by a user and at least one extensible function defined by a user to customize the framework and thereby define a desired information retrieval system, the framework including;

at least one object oriented index class object that includes at least one word index object that maps each word contained in a document to the document containing the word by performing a preprocessing operation on the document to generate the at least one word index object; and

at least one object oriented query index object that processes a user query to produce a query result from comparison of the user query and the word index objects in response to a user query; and

evaluating a user query by comparing information contained in the user query with information contained in the at least one index class object that relates to one or more documents stored in the computer system, such that the documents relevant to the user query will be identified, thereby providing a query result.

37. A method as defined in claim 36, wherein the provided framework includes an object oriented build index object that processes a document so as to create the word index objects in response to a user build index request.

38. A method as defined in claim 36, wherein the provided framework includes an object oriented posting list class of objects that contain a frequency count that indicates the number of times a word appears in a word index object of a document.

39. A method as defined in claim 36, wherein the provided framework includes an object oriented document table class of objects that map a word index object to the indexed document from which it was preprocessed.

40. A method as defined in claim 36, wherein the preprocessing operation of the provided framework includes a parsing method that identifies text words from other text characters.

41. A method as defined in claim 36, wherein the preprocessing operation of the provided framework includes a stoplist method that identifies text words not containing sufficient information to be useful in providing a query result and deletes such text words.

42. A method as defined in claim 36, wherein the preprocessing operation of the provided framework includes a stemming method that identifies text word stems of which a text word is a formative, and replaces the text word with the stem.

43. A method as defined in claim 36, wherein the documents are stored in the computer system as instances of an object oriented extensible binary-large-object class having document objects containing text information and binary document objects containing a digital representation of information other than text.

44. A method as defined in claim 43, wherein each binary document object is linked to a document object containing text information relating to the information other than text that is contained in the binary document object.

45. A method as defined in claim 43, wherein the digital representation of information other than text comprises image data, video data, or audio data.

46. A method as defined in claim 36, wherein the provided framework includes an object oriented load document object that stores documents into the memory of the system in response to a user storage request.

47. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a user-extensible object oriented framework residing in the memory, the framework including at least one core function that cannot be modified by a user and at least one extensible function defined by a user to customize the framework and thereby define a desired information retrieval system, the framework including;

a load document processor for loading and preprocessing a plurality of documents, the load document processor comprising;

a parser for separating text words in at least one of the plurality of documents from other text characters to generate a parsed word list;

a stoplist processor for deleting from the parsed word list any words that are defined in a stoplist collection to be too common to be useful in searching; and

a stemmer processor for mapping a plurality of words with a common stem to the common stem;

an index processor for creating at least one word index corresponding to the plurality of documents; and

a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result.

48. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a user-extensible object oriented framework residing in the memory, the framework including at least one core function that cannot be modified by a user and at least one extensible function defined by a user to customize the framework and thereby define a desired information retrieval system, the framework including;

a load document processor for loading and preprocessing a plurality of documents;

an index processor for creating at least one word index corresponding to the plurality of documents, the index processor comprising;

a word index mechanism for loading a word index with words from the plurality of documents;

a posting list mechanism that contains a list of the plurality of documents and corresponding counts for words and word stems in each of the plurality of documents; and

a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result.

49. An apparatus comprising:

at least one processor;

a memory coupled to the at least one processor;

a user-extensible object oriented framework residing in the memory, the framework including at least one core function that cannot be modified by a user and at least one extensible function defined by a user to customize the framework and thereby define a desired information retrieval system, the framework including;

a load document processor for loading and preprocessing a plurality of documents;

an index processor for creating at least one word index corresponding to the plurality of documents; and

a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result, the query processor comprising;

a parser for separating text words in the query from other text characters in the query;

a stoplist processor for deleting from the text words in the parsed query any words that are defined in a stoplist collection to be too common to be useful in searching; and

a stemmer processor for mapping a plurality of the text words in the query with a common stem to the common stem.

50. A program product comprising:

(A) a user-extensible object oriented framework mechanism comprising;

(1) a load document processor for loading and preprocessing a plurality of documents, the load document processor comprising;

(a) a parser for separating text words in at least one of the plurality of documents from other text characters to generate a parsed word list;

(b) a stoplist processor for deleting from the parsed word list any words that are defined in a stoplist collection to be too common to be useful in searching; and

(c) a stemmer processor for mapping a plurality of words with a common stem to the common stem;

(2) an index processor for creating at least one word index corresponding to the plurality of documents; and

(3) a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result; and

(B) signal bearing media bearing the framework mechanism.

51. The program product of claim 50 wherein the signal bearing media comprises recordable media.

52. The program product of claim 50 wherein the signal bearing media comprises transmission media.

53. A program product comprising:

(A) a user-extensible object oriented framework mechanism comprising;

(1) a load document processor for loading and preprocessing a plurality of documents;

(2) an index processor for creating at least one word index corresponding to the plurality of documents, the index processor comprising;

(a) a word index mechanism for loading a word index with words from the plurality of documents;

(b) a posting list mechanism that contains a list of the plurality of documents and corresponding counts for words and word stems in each of the plurality of documents; and

(3) a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result; and

(B) signal bearing media bearing the framework mechanism.

54. The program product of claim 53 wherein the signal bearing media comprises recordable media.

55. The program product of claim 53 wherein the signal bearing media comprises transmission media.

56. A program product comprising:

(A) a user-extensible object oriented framework mechanism comprising;

(1) a load document processor for loading and preprocessing a plurality of documents;

(2) an index processor for creating at least one word index corresponding to the plurality of documents; and

(3) a query processor for receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result, the query processor comprising;

(a) a parser for separating text words in the query from other text characters in the query;

(b) a stoplist processor for deleting from the text words in the parsed query any words that are defined in a stoplist collection to be too common to be useful in searching; and

(c) a stemmer processor for mapping a plurality of the text words in the query with a common stem to the common stem; and

(B) signal bearing media bearing the framework mechanism.

57. The program product of claim 56 wherein the signal bearing media comprises recordable media.

58. The program product of claim 56 wherein the signal bearing media comprises transmission media.

59. A method of retrieving information from a plurality of documents comprising the step of:

providing a user-extensible object oriented framework mechanism, the framework mechanism performing the steps of;

(A) loading and preprocessing a plurality of documents, the step of loading and preprocessing the plurality of documents includes the steps of;

separating text words in at least one of the plurality of documents from other text characters to generate a parsed word list;

deleting from the parsed word list any words that are defined in a stoplist collection to be too common to be useful in searching; and

mapping a plurality of words with a common stem to the common stem;

(B) creating at least one word index corresponding to the plurality of documents; and

(C) receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result.

60. A method of retrieving information from a plurality of documents comprising the step of:

providing a user-extensible object oriented framework mechanism, the framework mechanism performing the steps of;

(A) loading and preprocessing a plurality of documents;

(B) creating at least one word index corresponding to the plurality of documents, the step of creating the at least one word index corresponding to the plurality of documents includes the steps of;

loading a word index with words from the plurality of documents;

generating a posting list mechanism that contains a list of the plurality of documents and corresponding counts for words and word stems in each of the plurality of documents; and

(C) receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result.

61. A method of retrieving information from a plurality of documents comprising the step of:

providing a user-extensible object oriented framework mechanism, the framework mechanism performing the steps of;

(A) loading and preprocessing a plurality of documents;

(B) creating at least one word index corresponding to the plurality of documents; and

(C) receiving a query and determining if any of the plurality of documents match the query by processing the query and comparing the processed query to the plurality of words in the at least one word index, thereby providing a query result, the step of receiving a query and determining if any of the plurality of documents match the query includes the steps of;

separating text words in the query from other text characters in the query;

deleting from the text words in the parsed query any words that are defined in a stoplist collection to be too common to be useful in searching; and

mapping a plurality of the text words in the query with a common stem to the common stem.