Compensating for bias in search results

• US 10,007,730 B2
• Filed: 01/30/2015
• Issued: 06/26/2018
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Horvitz, Eric Joel, Awadallah, Ahmed Hassan, White, Ryen William
• Primary Examiner(s): Ly, Anh
• Application Number:
  US14/611,085
• Publication Number:
  US 20160224666A1
• Time in Patent Office: 1,243 Days
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Biases in search and retrieval (i.e., situations where searchers seek or are presented with information that significantly deviates from the truth) may be detected by comparison to one or more authoritative sources. Once bias or potential bias is detected, techniques may be applied to indicate and/or compensate for the bias. Such techniques may allow users to more easily assess the veracity of search results, and increase the chances that users will locate accurate answers to their questions.

Claims

1. A search engine computing device comprising:

one or more communication connections;

one or more processors; and

memory communicatively coupled to the one or more communication connections and the one or more processors, the memory storing a ranking algorithm and instructions that, when executed, configure the one or more processors to perform operations comprising;

receiving a search query from a client device via the one or more communication connections;

obtaining search results responsive to the search query based at least in part on the ranking algorithm, the search results including multiple entries;

detecting potential bias in at least one entry of the multiple entries based at least in part on a first term in a caption of the at least one entry, the first term previously determined to be biasing;

modifying the search results to generate modified search results that at least partially compensate for the potential bias in the at least one entry, wherein generating the modified search results includes replacing the first term of the caption with a second term, the second term previously determined to be non-biasing; and

outputting the modified search results via the one or more communication connections for presentation at the client device.

2. The search engine computing device of claim 1, wherein:

the first term is previously determined to be biasing as a result of the first term previously being determined to be less likely to be authoritative than the second term.

3. The search engine computing device of claim 1, wherein the modifying the search results comprises appending an annotation to the search results indicating that the at least one entry is potentially biased.

4. The search engine computing device of claim 1, the operations further comprising determining that the search query relates to diagnosing a condition, wherein the modifying the search results comprises appending an annotation to the search results including one or more normative base rates, each normative base rate indicating a likelihood of the condition given one or more symptoms.

5. The search engine computing device of claim 1, the operations further comprising ranking the search results based on the potential bias, wherein the modifying the search results comprises reordering the search results based on the potential bias to present entries having a least potential bias earlier in the modified search results.

6. The search engine computing device of claim 1, the operations further comprising outputting the search results obtained responsive to the search query, without modification, for presentation at the client device.

7. The search engine computing device of claim 1, the operations further comprising:

receiving user input responsive to presentation of the modified search results, the user input indicating bias of the modified search results; and

updating the ranking algorithm, based at least in part on the user input, to down sample one or more entries of the modified search results.

8. A method comprising:

under control of one or more search engine computing devices;

receiving a search query from a client device;

selecting at least one of a first ranking algorithm and a second ranking algorithm based on at least one of a subject of the search query and a user-specified preference;

based on a result of the selecting, performing at least one of;

obtaining first search results responsive to the search query based at least in part on the first ranking algorithm, and

obtaining second search results responsive to the search query based at least in part on the second ranking algorithm, the second ranking algorithm differing from the first ranking algorithm, and wherein one of the first ranking algorithm or the second ranking algorithm compensates for bias of search results; and

outputting the first search results and the second search results for presentation at the client device.

9. The method of claim 8, wherein one of the first ranking algorithm or the second ranking algorithm is based at least in part on veracity of respective search results, and the other of the first ranking algorithm or the second ranking algorithm is not.

10. The method of claim 8, wherein one of the first ranking algorithm or the second ranking algorithm weights popularity of respective search results more heavily than veracity of the respective search results, and the other of the first ranking algorithm or the second ranking algorithm weights veracity of other respective search results more heavily than popularity of the other respective search results.

11. The method of claim 8, wherein outputting both the first search results and the second search results for presentation at the client device comprises outputting both the first search results and the second search results in a composite format for presentation concurrently.

12. The method of claim 8, further comprising outputting a control with the first search results and the second search results for presentation at the client device, the control usable to toggle display between the first search results and the second search results.

13. The method of claim 8, wherein outputting the first search results and the second search results comprises:

outputting the first search results for presentation at the client device;

outputting a control with the first search results for presentation at the client device usable to view the second search results;

receiving user input indicating selection of the control; and

outputting the second search results for presentation at the client device.

14. A method comprising:

under control of one or more search engine computing devices;

receiving a search query from a client device;

obtaining first search results responsive to the search query based at least in part on a first ranking algorithm;

obtaining second search results responsive to the search query based at least in part on a second ranking algorithm, the second ranking algorithm differing from the first ranking algorithm, and wherein one of the first ranking algorithm or the second ranking algorithm compensates for bias of search results; and

outputting the first search results and the second search results for presentation at the client device,

wherein outputting both the first search results and the second search results for presentation at the client device comprises outputting both the first search results and the second search results in a composite format for presentation concurrently, and

wherein the composite format comprises a blended format in which entries of the first search results are interleaved with entries of the second search results.

15. One or more computer-readable media storage devices storing instructions that when executed by one or more processors of one or more search engine computing devices configure the one or more search engine computing devices to perform operations comprising:

receiving a search query from a client device;

obtaining search results responsive to the search query based at least in part on a ranking algorithm implemented by the one or more processors of the one or more search engine computing devices, the search results including multiple entries;

detecting, from among the multiple entries, at least one entry that is potentially biased in the search results;

annotating the search results by associating the detected at least one entry, with an indication identifying that the detected at least one entry is potentially biased in the search results; and

outputting the search results for presentation at the client device along with the associated indication, such that the detected at least one entry is presented at the client device in association with the indication to identify that the detected at least one entry is potentially biased in the search results.

16. The one or more computer-readable media of claim 15, wherein the indication of the one or more potentially biased entries comprises a visual indicator presented in association with each of the one or more potentially biased entries.

17. The one or more computer-readable media of claim 16, wherein each visual indicator comprises at least one of:

a confidence level that the respective potentially biased entry is biased;

or

a degree of bias of the respective potentially biased entry.

18. The one or more computer-readable media of claim 15, wherein detecting the one or more potentially biased entries comprises detecting one or more predetermined terms in captions of the one or more potentially biased entries, wherein the captions include at least one or more titles.

19. The one or more computer-readable media of claim 15, wherein detecting the one or more potentially biased entries comprises:

comparing each of the multiple entries to one or more authoritative sources; and

identifying entries that differ substantively from the one or more authoritative sources as being potentially biased entries.

20. One or more computer-readable media storage devices storing instructions that when executed by one or more processors of one or more search engine computing devices configure the one or more search engine computing devices to perform operations comprising:

receiving a search query from a client device;

obtaining search results responsive to the search query based at least in part on a ranking algorithm implemented by the one or more processors of the one or more search engine computing devices, the search results including multiple entries;

detecting one or more potentially biased entries from among the multiple entries;

annotating the search results with an indication of the one or more potentially biased entries; and

outputting the search results for presentation at the client device along with the indication of the one or more potentially biased entries,

wherein;

the search query relates to diagnosing a condition; and

the indication of the one or more potentially biased entries comprises one or more normative base rates, each normative base rate indicating a likelihood of the condition given one or more symptoms.