Consistent user experience in information retrieval systems

• US 8,429,171 B2
• Filed: 08/19/2008
• Issued: 04/23/2013
• Est. Priority Date: 08/20/2007
• Status: Active Grant

• Current Assignee: Nexidia Incorporated (Nice Ltd)
• Original Assignee: Nexidia Incorporated (Nice Ltd)
• Inventors: Morris, Robert W., Gavalda, Marsal
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US12/193,896
• Publication Number:
  US 20090055360A1
• Time in Patent Office: 1,708 Days
• Field of Search: 704250-257, 707713-719, 707748-751, 707759-769
• US Class Current: 707/748
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

An information retrieval system for search a corpus is configured to, for each of a plurality of searches of at least a subset of the corpus, generate a confidence score for each of a plurality of putative occurrences of a search query in the at least a subset of the corpus. The system is further configured to adjust a threshold to maintain a consistent user experience across the plurality of searches according to a consistency criterion. The system is further configured to display putative occurrences of the search query having a confidence score greater than the threshold.

Claims

1. A computer implemented information retrieval system for searching a corpus, the system comprising:

a computer;

a scoring module configured to, for each of one or more searches of at least a subset of the corpus, generate a confidence score for each of one or more putative occurrences of a search query in the at least a subset of the corpus;

a threshold module configured to adjust a threshold to maintain a consistent user experience across the one or more searches according to a consistency criterion, wherein the threshold is adjusted according to at least one of a duration of the at least a subset of the corpus or an audio quality of the at least a subset of the corpus; and

a display module configured to display putative occurrences of the search query having a confidence score greater than the threshold;

wherein the consistency criterion is selected from the group consisting of

a constant number of false alarms in the putative occurrences of the search query for each of the one or more searches,

a constant false alarm rate in the putative occurrences of the search query for each of the one or more searches,

a constant number of true occurrences of the search query for each of the one or more searches,

a constant precision value in the putative occurrences of the search query for each of the one or more searches,

a constant number of missed occurrences of the search query for each of the one or more searches, and

a constant recall value in the putative occurrences of the search query for each of the one or more searches.

2. The system of claim 1, wherein the consistency criterion comprises a constant number of false alarms in the putative occurrences of the search query for each of the one or more searches.

3. The system of claim 1, wherein the consistency criterion comprises a constant false alarm rate in the putative occurrences of the search query for each of the one or more searches.

4. The system of claim 1, wherein the consistency criterion comprises a constant number of true occurrences of the search query for each of the one or more searches.

5. The system of claim 1, wherein the consistency criterion comprises a constant precision value in the putative occurrences of the search query for each of the one or more searches.

6. The system of claim 1, wherein the consistency criterion comprises a constant number of missed occurrences of the search query for each of the one or more searches.

7. The system of claim 1, wherein the consistency criterion comprises a constant recall value in the putative occurrences of the search query for each of the one or more searches.

8. The system of claim 1, wherein the information retrieval system comprises a wordspotter.

9. The system of claim 1, wherein the active corpus comprises a corpus defined by metadata.

14. The system of claim 1, wherein the threshold is adjusted according to a characteristic of the user.

15. The system of claim 1, wherein the threshold is adjusted according to the search query.

16. The system of claim 1, wherein the putative occurrences of a search query include portions of an audio file.

10. A computer implemented information retrieval system for searching a corpus, the system comprising:

a computer;

a scoring module configured to, for each of one or more searches of at least a subset of the corpus, generate a confidence score for each of one or more putative occurrences of a search query in the at least a subset of the corpus;

a threshold module configured to adjust a threshold to maintain a consistent user experience across the one or more searches according to a consistency criterion, wherein the threshold is adjusted according to at least one of a duration of the at least a subset of the corpus or an audio quality of the at least a subset of the corpus; and

a display module configured to display putative occurrences of the search query having a confidence score greater than the threshold;

wherein

the retrieval system is further configured to modify the search algorithm to optimize the consistency criterion, and

the consistency criterion is selected from the group consisting of

a constant computation duration for each of the plurality of searches, and

a constant amount of memory used for each of the one or more searches.

11. The system of claim 10, wherein the system is configured to account for an audio quality of the active corpus in optimizing the consistency criterion.

12. The system of claim 10, wherein the consistency criterion comprises a constant computation duration for each of the plurality of searches.

13. The system of claim 10, wherein the consistency criterion comprises a constant amount of memory used for each of the one or more searches.

17. A computer implemented information retrieval system for searching a corpus, the system comprising:

a computer;

a scoring module configured to, for each of one or more searches of at least a subset of the corpus, generate a confidence score for each of one or more putative occurrences of a search query in the at least a subset of the corpus;

a threshold module configured to adjust a threshold to maintain a consistent user experience across the one or more searches according to a consistency criterion, wherein the threshold is adjusted according to the search query; and

a display module configured to display putative occurrences of the search query having a confidence score greater than the threshold;

wherein the consistency criterion is selected from the group consisting of

a constant number of false alarms in the putative occurrences of the search query for each of the one or more searches,

a constant false alarm rate in the putative occurrences of the search query for each of the one or more searches,

a constant number of true occurrences of the search query for each of the one or more searches,

a constant precision value in the putative occurrences of the search query for each of the one or more searches,

a constant number of missed occurrences of the search query for each of the one or more searches, and

a constant recall value in the putative occurrences of the search query for each of the one or more searches.

18. A computer implemented information retrieval system for searching a corpus, the system comprising:

a computer;

a scoring module configured to, for each of one or more searches of at least a subset of the corpus, generate a confidence score for each of one or more putative occurrences of a search query in the at least a subset of the corpus;

a threshold module configured to adjust a threshold to maintain a consistent user experience across the one or more searches according to a consistency criterion, wherein the threshold is adjusted according to the search query; and

a display module configured to display putative occurrences of the search query having a confidence score greater than the threshold;

wherein

the retrieval system is further configured to modify the search algorithm to optimize the consistency criterion, and

the consistency criterion is selected from the group consisting of

a constant computation duration for each of the plurality of searches, and

a constant amount of memory used for each of the one or more searches.