RETRIEVAL SYSTEMS AND METHODS EMPLOYING PROBABILISTIC CROSS-MEDIA RELEVANCE FEEDBACK

• US 20120054130A1
• Filed: 08/31/2010
• Published: 03/01/2012
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Csurka, Gabriela, Mensink, Thomas, Verbeek, Jakob
• Granted Patent: US 8,538,896 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor

Abstract

In a retrieval application, a document relevance scoring function comprises a weighted combination of scoring components including at least one of a pseudo-relevance scoring component and a cross-media relevance scoring component. Weights of the document relevance scoring function are optimized to generate a trained document relevance scoring function. The optimizing is respective to a set of training documents including at least some multimedia training documents and a set of training queries and corresponding training document relevance annotations. A retrieval operation is performed for an input query respective to a database using the trained document relevance scoring function to retrieve one or more documents from the database.

Claims

1. A method comprising:

optimizing weights of a document relevance scoring function to generate a trained document relevance scoring function, wherein;

the document relevance scoring function comprises a weighted combination of scoring components including at least one of a pseudo-relevance scoring component and a cross-media relevance scoring component, and

the optimizing is respective to a set of training documents including at least some multimedia training documents and a set of training queries and corresponding training document relevance annotations; and

performing a retrieval operation for an input query respective to a database using the trained document relevance scoring function to retrieve one or more documents from the database;

wherein the optimizing and the performing are performed by a digital processor.

2. The method as set forth in claim 1, wherein the document relevance scoring function comprises a weighted linear combination of scoring components including at least one pseudo-relevance scoring component, at least one cross-media relevance scoring component, and at least one direct relevance scoring component.

3. The method as set forth in claim 1, wherein:

the document relevance scoring function includes at least one of a pseudo-relevance textual scoring component and a pseudo-relevance image scoring component; and

the document relevance scoring function includes at least one of a cross-media relevance scoring component having a text query modality and an image feedback modality and a cross-media relevance scoring component having an image query modality and a text feedback modality.

4. The method as set forth in claim 1, wherein the optimizing comprises:

optimizing a distribution-matching objective function respective to matching between;

a distribution of document relevance computed using the document relevance scoring function for the set of training queries and the set of training documents, and

a distribution of the training document relevance annotations corresponding to the set of training queries.

5. The method as set forth in claim 1, wherein the optimizing comprises:

training a classifier employing the document relevance scoring function to predict document relevance for an input query, the training using the set of training documents and the set of training queries and corresponding training document relevance annotations.

6. The method as set forth in claim 1, wherein the optimizing comprises:

training a classifier employing the document relevance scoring function to predict a set of most relevant documents of the set of training documents for an input query, the training using the set of training documents and the set of training queries and corresponding training document relevance annotations.

7. The method as set forth in claim 1, wherein the optimizing comprises:

for each training query of the set of training queries, computing training document relevance values for the training documents using the document relevance scoring function; and

scaling the computed training document relevance values using training query-dependent scaling factors.

8. The method as set forth in claim 7, wherein the optimizing also optimizes the training query-dependent scaling factors.

9. The method as set forth in claim 7, wherein the training query-dependent scaling factors include a linear scaling factor α

_q for each training query.

11. The method as set forth in claim 7, wherein the trained document relevance scoring function does not include the training query-dependent scaling factors.

12. The method as set forth in claim 7, wherein each pseudo-relevance scoring component or cross-media relevance scoring component comprises a weighted linear combination of an ordered list of top k most similar feedback scoring sub-components weighted by a feedback ordinal position weighting, and the optimizing further comprises:

optimizing at least one parameter controlling the feedback ordinal position weighting.

13. The method as set forth in claim 1, wherein each pseudo-relevance scoring component or cross-media relevance scoring component comprises a weighted linear combination of an ordered list of top k most similar feedback scoring sub-components weighted by a feedback ordinal position weighting, and the optimizing further comprises:

optimizing at least one parameter controlling the feedback ordinal position weighting.

14. The method as set forth in claim 1, wherein the document relevance scoring function comprises a weighted linear combination of scoring components including at least one of a pseudo-relevance scoring component and a cross-media relevance scoring component.

15. The method as set forth in claim 1, wherein the database includes annotated images, the retrieval operation is performed for an input query image and the method further comprises:

constructing an annotation for the input query image based on annotations of annotated images retrieved from the database by the retrieval operation.

16. A storage medium storing instructions executable by a digital processor to perform a method as set forth in claim 1.

10. The method as set forth in claim 10, wherein the training query-dependent scaling factors further include an offset scaling factor β

_q for each training query.

17. An apparatus comprising:

a digital processor configured to train a document relevance scoring function to generate a trained document relevance scoring function, wherein;

the document relevance scoring function comprises a weighted linear combination of scoring components including at least one of a pseudo-relevance scoring component and a cross-media relevance scoring component,

the training adjusts weights of the weighted linear combination of scoring components, and

the training is respective to a set of training documents including at least some multimedia training documents and a set of training queries and corresponding training document relevance annotations.

18. The apparatus as set forth in claim 17, wherein:

the document relevance scoring function includes at least one of a pseudo-relevance textual scoring component and a pseudo-relevance image scoring component;

the document relevance scoring function includes at least one of a cross-media relevance scoring component having a text query modality and an image feedback modality and a cross-media relevance scoring component having an image query modality and a text feedback modality; and

the linear combination of scoring components further includes at least one of a direct relevance text scoring component and a direct relevance image scoring component.

19. The apparatus as set forth in claim 17, wherein the digital processor is configured to train the document relevance scoring function by a process including:

for each training query of the set of training queries, computing training document relevance values for the training documents using the document relevance scoring function; and

adjusting weights of the weighted linear combination of scoring components to optimize an objective function or classifier measuring an aggregate similarity between the computed training document relevance values and the corresponding training document relevance annotations.

20. The apparatus as set forth in claim 17, wherein the digital processor is configured to train the document relevance scoring function by a process including:

for each training query of the set of training queries, computing training document relevance values for the training documents using the document relevance scoring function;

scaling the computed training document relevance values using training query-dependent scaling factors; and

adjusting (i) weights of the weighted linear combination of scoring components and (ii) the training query-dependent scaling factors to optimize an objective function or classifier measuring an aggregate similarity between the computed training document relevance values and the corresponding training document relevance annotations.

21. The apparatus as set forth in claim 20, wherein the training query-dependent scaling factors include a linear scaling factor for each training query of the set of training queries.

22. The apparatus as set forth in claim 20, wherein trained document relevance scoring function does not include the training query-dependent scaling factors, and the digital processor is further configured to perform a retrieval operation using the trained document relevance scoring function.