Methods and apparatuses for facilitating content-based image retrieval

• US 8,571,358 B2
• Filed: 12/30/2010
• Issued: 10/29/2013
• Est. Priority Date: 12/30/2009
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Nokia Corporation
• Inventors: Wang, Hao, Zhao, Feng
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US12/982,698
• Publication Number:
  US 20110158558A1
• Time in Patent Office: 1,034 Days
• Field of Search: 382/165, 382/181, 382/282, 382/305, 382/306, 382/312, 382/173, 707/707, 707/713, 707/705, 707/721, 707/728
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship

Abstract

Methods and apparatuses are provided for facilitating content-based image retrieval. A method may include determining a selected target image. The method may further include generating a candidate region of interest set including one or more regions of interest within the target image. The method may additionally include determining a recommended region of interest set including one or more recommended regions of interest selected from the candidate region of interest set based at least in part upon evaluation criteria. The evaluation criteria may be determined based at least in part upon analysis of maintained region of interest-based searching history. The method may also include providing the recommended region of interest set for user selection of one or more target regions of interest from the recommended region of interest set as query criteria for searching an image library for one or more result images. Corresponding apparatuses are also provided.

Claims

1. A method comprising:

determining a selected target image;

generating a candidate region of interest set, the candidate region of interest set comprising one or more regions of interest within the target image;

determining a recommended region of interest set, the recommended region of interest set comprising one or more recommended regions of interest selected from the candidate region of interest set based at least in part upon evaluation criteria, the evaluation criteria being determined based at least in part upon analysis of maintained region of interest-based searching history; and

providing the recommended region of interest set for user selection of one or more target regions of interest from the recommended region of interest set as query criteria for searching an image library for one or more result images.

2. The method of claim 1, wherein determining the recommended region of interest set comprises:

computing a recommendation score for each candidate region of interest in the candidate region of interest set using the evaluation criteria; and

determining one or more recommended regions of interest from the candidate region of interest set based at least in part upon the computed recommendation scores.

3. The method of claim 1, wherein the maintained region of interest-based searching history comprises a history of one or more of:

previously recommended regions of interest;

regions of interest previously selected as target regions of interest;

historical relationships between target regions of interest selected as query criteria and corresponding search result images;

or

feedback on previous search result images indicating correspondence between the search result images and corresponding query criteria used to identify the search result images.

4. The method of claim 1, further comprising:

determining a selection of one or more target regions of interest within the selected target image as query criteria;

searching at least a portion of the image library to identify a result image corresponding to the query criteria;

providing the identified result image to the user;

determining feedback on the identified result image, the feedback comprising an indication of whether the identified result image relates to a selected target region of interest; and

updating the region of interest-based searching history to include information about the feedback.

5. The method of claim 4, further comprising updating the region of interest-based searching history to include one or more of:

information about the identified result image and the selected one or more target regions of interest;

or

information about a relationship between the identified result image and the selected one or more target regions of interest.

6. The method of claim 4, wherein searching the at least a portion of the image library comprises searching at least a portion of the image library to identify a plurality of result images corresponding to the query criteria;

and further comprising;

filtering the plurality of result images based at least in part upon the maintained region of interest-based searching history by filtering out any result images known from the maintained region of interest-based searching history to not be related to a selected target region of interest.

7. The method of claim 4, wherein searching the at least a portion of the image library comprises:

determining a region of interest class for a selected target region of interest;

analyzing the maintained region of interest-based searching history to determine images within the image library known to comprise a region of interest having the determined region of interest class; and

searching the determined images to identify a result image corresponding to the query criteria.

8. The method of claim 4, further comprising

determining a second selected target image;

and

determining, prior to searching the at least a portion of the image library, a selection of one or more target regions of interest within the second selected target image, wherein the query criteria further comprises the one or more target regions of interest selected from the second selected target image.

9. An apparatus comprising at least one processor and at least one memory storing computer program code, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to at least:

determine a selected target image;

generate a candidate region of interest set, the candidate region of interest set comprising one or more regions of interest within the target image;

determine a recommended region of interest set, the recommended region of interest set comprising one or more recommended regions of interest selected from the candidate region of interest set based at least in part upon evaluation criteria, the evaluation criteria being determined based at least in part upon analysis of maintained region of interest-based searching history; and

provide the recommended region of interest set for user selection of one or more target regions of interest from the recommended region of interest set as query criteria for searching an image library for one or more result images.

10. The apparatus of claim 9, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to determine the recommended region of interest set at least in part by:

computing a recommendation score for each candidate region of interest in the candidate region of interest set using the evaluation criteria; and

determining one or more recommended regions of interest from the candidate region of interest set based at least in part upon the computed recommendation scores.

11. The apparatus of claim 9, wherein the maintained region of interest-based searching history comprises a history of one or more of:

previously recommended regions of interest;

regions of interest previously selected as target regions of interest;

historical relationships between target regions of interest selected as query criteria and corresponding search result images;

or

feedback on previous search result images indicating correspondence between the search result images and corresponding query criteria used to identify the search result images.

12. The apparatus of claim 9, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

determine a selection of one or more target regions of interest within the selected target image as query criteria;

search at least a portion of the image library to identify a result image corresponding to the query criteria;

provide the identified result image to the user;

determine feedback on the identified result image, the feedback comprising an indication of whether the identified result image relates to a selected target region of interest; and

update the region of interest-based searching history to include information about the feedback.

13. The apparatus of claim 12, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to update the region of interest-based searching history to include one or more of:

information about the identified result image and the selected one or more target regions of interest;

or

information about a relationship between the identified result image and the selected one or more target regions of interest.

14. The apparatus of claim 12, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to search the at least a portion of the image library by searching at least a portion of the image library to identify a plurality of result images corresponding to the query criteria;

and wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to;

filter the plurality of result images based at least in part upon the maintained region of interest-based searching history by filtering out any result images known from the maintained region of interest-based searching history to not be related to a selected target region of interest.

15. The apparatus of claim 12, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to search the at least a portion of the image library at least in part by:

determining a region of interest class for a selected target region of interest;

analyzing the maintained region of interest-based searching history to determine images within the image library known to comprise a region of interest having the determined region of interest class;

and searching the determined images to identify a result image corresponding to the query criteria.

16. The apparatus of claim 12, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

determine a second selected target image; and

determine, prior to searching the at least a portion of the image library, a selection of one or more target regions of interest within the second selected target image, wherein the query criteria further comprises the one or more target regions of interest selected from the second selected target image.

17. The apparatus of claim 9, wherein the apparatus comprises or is embodied on a mobile phone, the mobile phone comprising user interface circuitry and user interface software stored on one or more of the at least one memory;

wherein the user interface circuitry and user interface software are configured to;

facilitate user control of at least some functions of the mobile phone through use of a display; and

cause at least a portion of a user interface of the mobile phone to be displayed on the display to facilitate user control of at least some functions of the mobile phone.

18. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program instructions stored therein, the computer-readable program instructions comprising:

program instructions configured to determine a selected target image;

program instructions configured to generate a candidate region of interest set, the candidate region of interest set comprising one or more regions of interest within the target image;

program instructions configured to determine a recommended region of interest set, the recommended region of interest set comprising one or more recommended regions of interest selected from the candidate region of interest set based at least in part upon evaluation criteria, the evaluation criteria being determined based at least in part upon analysis of maintained region of interest-based searching history; and

program instructions configured to provide the recommended region of interest set for user selection of one or more target regions of interest from the recommended region of interest set as query criteria for searching an image library for one or more result images.

19. The computer program product of claim 18, wherein the program instructions configured to determine the recommended region of interest set comprise:

program instructions configured to compute a recommendation score for each candidate region of interest in the candidate region of interest set using the evaluation criteria; and

program instructions configured to determine one or more recommended regions of interest from the candidate region of interest set based at least in part upon the computed recommendation scores.

20. The computer program product of claim 18, wherein the maintained region of interest-based searching history comprises a history of one or more of:

previously recommended regions of interest;

regions of interest previously selected as target regions of interest;

historical relationships between target regions of interest selected as query criteria and corresponding search result images;

or feedback on previous search result images indicating correspondence between the search result images and corresponding query criteria used to identify the search result images.