Method, system and software for searching, identifying, retrieving and presenting electronic documents

• US 10,140,295 B2
• Filed: 03/30/2015
• Issued: 11/27/2018
• Est. Priority Date: 03/29/2014
• Status: Active Grant

• Current Assignee: Camelot UK Bidco Limited (Clarivate Plc)
• Original Assignee: Camelot UK Bidco Limited (Clarivate Plc)
• Inventors: Ryger, Raphael Shmuel, Suvorova, Ekaterina
• Primary Examiner(s): Jalil, Neveen Abel
• Assistant Examiner(s): Chiu, Katriel Y
• Application Number:
  US14/673,789
• Publication Number:
  US 20150310114A1
• Time in Patent Office: 1,338 Days
• Field of Search: None
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

The present invention provides a method and system for identifying, retrieving and presenting electronic documents responsive to user queries. The three distinct inventive concepts are relevancy ranking of responsive documents based on component query technique; cross-lingual searching; and search expansion using analytics of initial results to derive and generate a modified query. Each of these inventions enhances document search and retrieval systems and the three solutions may be used separately or in any combination. The three inventions apply in layers above an underlying search system, controlling the submission of requests to the underlying system in support of received search requests, typically originating with an end user. Invention (I) involves use of focus-spectrum expanded search queries and either of two general types of enhanced OR operator that may be offered by the underlying search system. Processing of the content domain informs automated generation of elaborated queries using these operators for submitting to such an underlying system so as to yield improved relevance ranking.

Claims

1. A computer-implemented method of improving performance of a computer to enhance relevance ranking of results in a computer-implemented search of a corpus of documents having in part homogeneous structured data having a field structure, using an underlying search engine offering a distribution-biased OR operator, the method comprising:

a. as a preparatory phase, before a request function receives any queries, based on knowledge of the field structure of the documents to be searched and the content of some of the fields, determining a mapping or policy for automatically, by a processor, replacing a received search clause with a sequence of multiple search clauses of anticipated nested respective responsive scopes, such that a set of documents responsive to one search clause in the sequence of search clauses is predicted to be a subset of the set of documents responsive to the next search clause in the sequence of search clauses;

b. on an ongoing basis, operating at a search-controller level to mediate between the request function and an underlying search engine, and electronically receiving a query comprising at least one search clause from the request function;

c. by the processor, according to the mapping or policy determined in preparatory-phase step (a), transforming at least one of the at least one received search clause received in step (b) into an automatically generated, intentionally redundant sequence of search clauses representing the anticipated nesting of respective responsive scopes designed in the step (a) mapping or policy, so as to correspond to gradations of focus on a topic of interest interpreted to be suggested by the at least one received search clause;

d. combining the generated sequence of search clauses with a distribution-biased OR operator and submitting the combination to the underlying search engine to be executed against a database associated with the search engine, wherein scoring of the distribution-biased OR operation reflects not only aggregate scoring from respective counts of matches for the alternate clauses under the distribution-biased OR but also the evenness of distribution of that aggregate across its contributing component scores coming from those respective alternate clauses;

e. automatically executing the submitted combination of search clauses, by the underlying search engine, such that the nesting of the individual clauses'"'"' respective responsive scopes induces scopes of focus induce the underlying search engine to return a set of responsive documents for the entire OR combination that corresponds to the responsive set for the broadest of the submitted, automatically generated sequence of individual search clauses; and

f. automatically relevance-scoring the broad set of responsive documents returned in step (e), via the underlying search engine'"'"'s processing of the distribution-biased OR-connected sequence of search clauses automatically generated and submitted in steps (c) and (d), yielding scoring more sensitive to gradations of focus on a topic of interest as suggested in the at least one search clause received in step (b).

2. The method of claim 1, wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated sequence of search clauses replacing the at least one received search clause are in the form <

value>

in <

field_1>

;

<

value>

in <

field 2>

;

. . . <

value>

in <

field N>

, where <

field 1>

, <

field 2>

, . . . <

field N>

are each distinct sets of one or more fields replacing the original <

field> and

are automatically chosen based on the original <

field> and

the mapping determined in advance of the request function receiving any queries so as to achieve a desired nesting of scopes.

3. The method of claim 1, wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated sequence of search clauses replacing the at least one received search clause are in the form <

value_1>

in <

field>

;

<

value_2>

in <

field>

;

. . . <

value_N>

in <

field>

, where <

value_1>

, <

value_2>

, . . . <

value_N>

are each distinct sets of one or more values replacing the original <

value> and

are automatically chosen based on the original <

value>

.

4. The method of claim 1, wherein the underlying search engine processes the distribution-biased OR-connected sequence of search clauses in part by accepting and respecting numerical weights attached to individual search clauses in the automatically generated sequence of search clauses, the assignment of such weights having been pre-tuned in advance of the request function receiving any queries.

5. An apparatus that enhances relevance ranking of results in a search of a corpus of documents having in part homogeneous structured data having a field structure, using an underlying search engine offering a distribution-biased OR operator, the apparatus comprising:

processing circuitry configured to

a. as a preparatory phase, before a request function receives any queries, based on knowledge of the field structure of the documents to be searched and the content of some of the fields, determine a mapping or policy for automatically, by a processor, replacing a received search clause with a sequence of multiple search clauses of anticipated nested respective responsive scopes, such that a set of documents responsive to one search clause in the sequence of search clauses is predicted to be a subset of the set of documents responsive to the next search clause in the sequence of search clauses;

b. on an ongoing basis, operate at a search-controller level to mediate between the request function and an underlying search engine, and electronically receive a query comprising at least one search clause from the request function;

c. transform, according to the mapping or policy determined in preparatory-phase step (a), at least one of the at least one received search clause received in step (b) into an automatically generated, intentionally redundant sequence of search clauses representing the anticipated nesting of respective responsive scopes designed in the step (a) mapping or policy, so as to correspond to gradations of focus on a topic of interest interpreted to be suggested by the at least one received search clause;

d. combine the generated sequence of search clauses with a distribution-biased OR operator and submit the combination to the underlying search engine to be executed against a database associated with the search engine, wherein scoring of the distribution-biased OR operation reflects not only aggregate scoring from respective counts of matches for the alternate clauses under the distribution-biased OR but also the evenness of distribution of that aggregate across its contributing component scores coming from those respective alternate clauses;

e. automatically execute the submitted combination of search clauses, by the underlying search engine, such that the nesting of the individual clauses'"'"' respective responsive scopes induces the underlying search engine to return a set of responsive documents for the entire OR combination that corresponds to the responsive set for the broadest of the submitted, automatically generated sequence of individual search clauses; and

f. automatically relevance-score the broad set of responsive documents returned in step (e), via the underlying search engine'"'"'s processing of the distribution-biased OR-connected sequence of search clauses automatically generated and submitted in steps (c) and (d), yielding scoring more sensitive to gradations of focus on a topic of interest as suggested in the at least one search clause received in step (b).

6. The apparatus of claim 5 wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated set of search clauses replacing the at least one received search clause are in the form <

value>

in <

field_1>

;

<

value>

in <

field_2>

;

. . . <

value>

in <

field_N>

, where <

field_1>

, <

field_2>

, <

field_N>

are each distinct sets of one or more fields replacing the original <

field> and

are automatically chosen based on the original <

field> and

the mapping determined in advance of the request function receiving any queries so as to achieve a desired nesting of scopes.

7. The apparatus of claim 5 wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated sequence of search clauses replacing the at least one received search clause are in the form <

value_1>

in <

field>

;

<

value_2>

in <

field>

;

. . . <

value_N>

in <

field>

, and where <

value_1>

, <

value_2>

, . . . <

value_N>

are each distinct sets of one or more values replacing the original <

value> and

are automatically chosen based on the original <

value>

.

8. The apparatus of claim 5 wherein the underlying search engine processes the distribution-biased OR-connected sequence of search clauses in part by accepting and respecting numerical weights attached to individual search clauses in the automatically generated sequence of clauses, the assignment of such weights having been pre-tuned in advance of the request function receiving any queries.

9. A non-transitory computer-readable medium having stored therein computer-readable instructions which when executed by a computer cause the computer to perform a computer-implemented method of improving performance of a computer to enhance relevance ranking of results in a computer-implemented search of a corpus of documents having in part homogeneous structured data having a field structure, using an underlying search engine offering a distribution-biased OR operator, the method comprising:

a. as a preparatory phase, before a request function receives any queries, based on knowledge of the field structure of the documents to be searched and the content of some of the fields, determining a mapping or policy for automatically, by a processor, replacing a received search clause with a sequence of multiple search clauses of anticipated nested respective responsive scopes, such that a set of documents responsive to one search clause in the sequence of search clauses is predicted to be a subset of the set of documents responsive to the next search clause in the sequence of search clauses;

b. on an ongoing basis, operating at a search-controller level to mediate between the request function and an underlying search engine and electronically receiving a query comprising at least one search clause from the request function;

c. by the processor, according to the mapping or policy determined in preparatory-phase (a), transforming at least one of the at least one received search clause received in step (b) into an automatically generated, intentionally redundant sequence of search clauses representing the anticipated nesting of respective responsive scopes of focus on a topic of interest interpreted to be suggested by the at least one received search clause;

d. combining the generated sequence of search clauses with a distribution-biased OR operator and submitting the combination to the underlying search engine to be executed against a database associated with the search engine, wherein scoring of the distribution-biased OR operation reflects not only aggregate scoring from respective counts of matches for the alternate clauses under the distribution-biased OR but also the evenness of distribution of that aggregate across its contributing component scores coming from those respective alternate clauses;

e. automatically executing the submitted combination of search clauses, by the underlying search engine, such that the nesting of the individual clauses'"'"' respective responsive scopes induces the underlying search engine to return a set of responsive documents for the entire OR combination that corresponds to the responsive set for the broadest of the submitted, automatically generated sequence of individual search clauses; and

f. automatically relevance-scoring the broad set of responsive documents returned in step (e), via the underlying search engine'"'"'s processing of the distribution-biased OR-connected sequence of search clauses automatically generated and submitted in steps (c) and (d), yielding scoring more sensitive to gradations of focus on a topic of interest as suggested in the at least one search clause received in step (b).

10. The non-transitory computer-readable medium of claim 9, wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated sequence of search clauses replacing the at least one received search clause are in the form <

value>

in <

field_1>

;

<

value>

in <

field_2>

;

. . . <

value>

in <

field_N>

, where <

field_1>

, <

field_2>

, . . . <

field_N>

are each distinct sets of one or more fields replacing the original <

field> and

are automatically chosen based on the original <

field> and

the mapping determined in advance of the request function receiving any queries so as to achieve a desired nesting of scopes.

11. The non-transitory computer-readable medium of claim 9, wherein the at least one received search clause is in the form <

value>

in <

field> and

the automatically generated sequence of search clauses replacing the at least one received search clause are in the form <

value_1>

in <

field>

;

<

value_2>

in <

field>

;

. . . <

value_N>

in <

field>

, where <

value_1>

, <

value_2>

, . . . <

value_N>

are each distinct sets of one or more values replacing the original <

value> and

are automatically chosen based on the original <

value>

.

12. The non-transitory computer-readable medium of claim 9, wherein the underlying search engine processes the distribution-biased OR-connected sequence of search clauses in part by accepting and respecting numerical weights attached to individual search clauses in the automatically generated sequence of search clauses, the assignment of such weights having been pre-tuned in advance of the request function receiving any queries.