Process and apparatus for automatic retrieval from a database and for automatic enhancement of such database

• US 20030204500A1
• Filed: 04/24/2003
• Published: 10/30/2003
• Est. Priority Date: 04/25/2002
• Status: Abandoned Application

• Current Assignee: Commissariat a L'Energie Atomique (French Alternative Energies and Atomic Energy Commission), Jean-Francois Delpech
• Original Assignee: Commissariat a L'Energie Atomique (French Alternative Energies and Atomic Energy Commission), Jean-Francois Delpech
• Inventors: Delpech, Jean-Francois
• Application Number:
  US10/422,222
• Publication Number:
  US 20030204500A1
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The system comprises an interface for acquiring an initial request in natural language, which request is sent to a server of a main database, a storage memory for storing information searching methods, specialized modules for implementing searching methods, a module for responding to the initial request to produce metarequests, said module comprising a unit for extracting the most meaningful words or expressions from the initial request, a search engine for accessing an additional database, a unit for processing the metarequests in order to adapt them to the search engine, a unit for sending the processed metarequests to the search engine giving access to the additional database in order to obtain additional documents corresponding to the initial request, and a unit for transmitting additional documents to the specialized module for processing and formatting which then transmits processed and formatted information to the server in order to enrich the main database.

Claims

1. / A method of automatically extracting information contained in a main database and of automatically enriching the content of the main database, the method comprising the following steps:

a) presenting and sending to a server at least one initial request in natural language and of arbitrary length;

b) implementing a search method in a specialized module, the search method being suitable for defining a list of documents obtained from the main database in the context of the initial request;

c) on the basis of the initial request, producing at least one metarequest constructed by extracting the most meaningful words or expressions from the initial request, the metarequest serving to perform an additional search on at least one other, additional database, using at least one search engine, to find additional documents corresponding to the initial request; and

d) transmitting said additional documents to a specialized processing and formatting module which transmits processed and formatted information to the server in order to enrich the main database.

2. / A method according to claim 1, wherein the other, additional database is accessible over a network of the Intranet or Internet type.

3. / A method according to claim 1, wherein the various metarequests used for additional searches in at least one other, additional database using at least one search engine are sent in an order that is arbitrary and random so as to prevent unauthorized reconstruction of an initial request solely from the stream of metarequests.

4. / A method according to claim 3, including a test step consisting, for each initial request, in authorizing metarequests to be sent to at least one other, additional database using at least one search engine only if the initial request in question enables a number N of different metarequests to be generated where the number N is greater than or equal to a predetermined integer N₀.

5. / A method according to claim 3, including a step of verifying the mixing of metarequests in order to prevent a metarequest from a given initial request being sent to at least one other, additional database, unless said metarequest is present simultaneously with a plurality of other metarequests corresponding to a plurality of other different initial requests.

6. / A method according to claim 1, wherein the additional documents relating to a plurality of requests are transmitted by a server to a specialized module for deferred processing and formatting.

7. / A method according to claim 1, wherein the metarequests are generated by analyzing the language of the request to identify words and groups of words or “

expressions”

that have logical connections between one another, and then retaining only those words or groups of words that are representative of the request.

8. / A method according to claim 1, wherein after the step of presenting and sending at least one initial request to a server, an automatic selection step is performed by the server to select an information searching method from a set of different searching methods as a function of the type and the presentation of the initial request, after which a specialized module is used to implement the selected searching method in order to define a list of documents obtained from the main database in the context of the initial request.

9. / A method according to claim 8, wherein the set of different searching methods comprises at least a Boolean or extended Boolean type searching method and a statistical type searching method.

10. / A method according to claim 1, wherein a step is also performed of establishing “

extemporaneously”

a summary of each of the documents in the list of documents obtained in the context of the initial request, the summary being obtained by extracting sentences or parts of sentences that are the most meaningful compared with the initial request.

11. / A method according to claim 1, wherein the search is performed in iterative manner, by successive approximations, by presenting new requests that take account of the responses received from the preceding request.

12. / A method according to claim 10, wherein the search is performed in iterative manner, by successive approximations, by presenting new requests that take account of the responses received from the preceding request, and wherein the summary of a document obtained in the context of the preceding request is used as the new request.

13. / A method according to claim 1, wherein the presentation of a request in natural language of arbitrary length is performed by an operation of the “

cut/paste”

type starting from a pertinent text.

14. / A method according to claim 8, wherein the selected information searching method is selected as a function of a criterion constituted by the number of meaningful words in the request concerned.

15. / A method according to claim 14, wherein the meaningful nature of a word is determined on the basis of the rarity of said word in the database.

16. / A method according to claim 10, wherein the summary of a document is drawn up “

extemporaneously”

by making use of statistical data associated with the document and stored in the same file.

17. / A method according to claim 1, wherein the information for enriching the main database as processed and formatted by the specialized processing and formatting module is transmitted to the server if the additional documents specified by the search engine, after comparison with the initial request, presents a pertinent index that exceeds a predetermined threshold.

18. / A method according to claim 10, wherein prior to drawing up a summary of each of the documents in the list of documents obtained in the context of a request, each document that has been gathered has been put into canonical form with a first version that is in its original form, a second version transformed into ordinary text suitable for drawing up a summary and for indexing, an http header, and line by line indexing, these various kinds of information being compressed in a single file.

19. / A method according to claim 7, wherein the words or groups of words that are retained as being representative of the request are selected as a function of their rarity.

20. / A system for automatically extracting information contained in a main database and for automatically enriching said database, the system comprising:

a) means for acquiring at least one initial request in natural language and of arbitrary length;

b) means for sending said request to a server of the main database;

c) at least one storage memory for storing at least one information searching method;

d) at least one specialized module for implementing at least one information searching method stored in said storage memory;

e) a module for responding to an initial request by producing at least one metarequest, the module comprising means for extracting the most meaningful words or expressions from the initial request;

f) at least one other, additional database source;

g) at least one search engine for accessing said other, additional database source;

h) means for processing metarequests to adapt them to said search engine;

i) means for sending processed metarequests to said search engine that provides access to said other, additional database source in order to obtain additional documents corresponding to the initial request; and

j) a unit for transmitting additional documents to a specialized processing and formatting module which transmits processed and formatted information to the server for enriching the main database.

21. / A system according to claim 20, further comprising mixer means connected to the module for producing metarequests in order to send different metarequests for use in additional searches on at least one other, additional database source by means of at least one search engine in an order that is arbitrary and random.

22. / A system according to claim 21, including a unit for counting the number N of metarequests generated for a single initial request by the metarequest production module, a comparator unit for comparing the number N of generated metarequests with a predetermined number N₀, and an authorization unit connected to the comparison unit to authorize or not authorize the sending of the N metarequests generated for a single initial request, as a function of the result supplied by the comparator unit.

23. / A system according to claim 21, wherein the mixer means comprise a unit for verifying the mixture of metarequests, said means including means for identifying the initial request to which each metarequest belongs.

24. / A system according to claim 20, including a plurality of specialized modules for implementing different information searching methods stored in said storage memory, and means for automatically selecting from said memory in order to enable the server to act as a function of the type and the presentation of the initial request to select an information searching method from the set of different information searching methods stored in said memory.

25. / A system according to claim 20, further comprises a module for establishing “

extemporaneously”

a summary for each of the documents in the list of documents obtained in the context of the initial request, said module comprising means for extracting the sentences or parts of sentences that are the most meaningful compared with the initial request.