HUMAN PHOTO SEARCH SYSTEM

• US 20130301938A1
• Filed: 08/30/2012
• Published: 11/14/2013
• Est. Priority Date: 05/11/2012
• Status: Abandoned Application

• Current Assignee: National Taiwan University (Government of The Republic of China)
• Original Assignee: National Taiwan University (Government of The Republic of China)
• Inventors: Chen, Yin-Ying, Hsu, Winston H., Lei, Yu-Heng
• Application Number:
  US13/599,127
• Publication Number:
  US 20130301938A1
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A human photo search system is provided. A user can search for a human photo using a canvas interactive interface on a user device, such as a touch panel or a computer. The user composes his/her impression of a desired photo on a query canvas to generate query semantics, which are then sent to a photo search server. The photo search server then searches a human photo database for candidate photos corresponding to the query semantics, and ranks the candidate photos according to relevance. Finally, the photo search server sends the sorted candidate photos back to the user device for display. Accordingly, the human photo search system of the present invention can search possible photos by the positions, the sizes, and the human attributes of the people in the desired photo, for which the user composes his/her impression on the query canvas without entering any text tags.

Claims

1. A human photo search system, comprising:

a user device including a canvas interactive interface, the canvas interactive interface including a query canvas area for allowing a user to compose human content and human layout therein to generate query semantics; and

a photo search server including;

a human photo database for storing a plurality of human photos and building a block-based index based on position and size information;

a search module for receiving the query semantics from the user device and retrieving candidate photos pointed to by the block-based index of the human photo database based on the query semantics;

a ranking module for generating a score for each of the candidate photos based on relevance, and sorting all of the candidate photos according to the scores therefor; and

a display module for returning the sorted candidate photos back to the user device.

2. The human photo search system of claim 1, wherein the human content includes at least one selected from the group consisting of gender, age, race, facial expression, hairstyle, and accessories.

3. The human photo search system of claim 1, wherein the human content further includes facial appearance similarity, whose source image is selected from the candidate photos or a facial photo input by the user.

4. The human photo search system of claim 1, wherein the human layout includes at least one selected from the group consisting of positions, sizes, angles, and the number of people in the query canvas area.

5. The human photo search system of claim 1, wherein the relevance takes into account errors between the query semantics generated by the query canvas and a candidate photo, the errors including human attributes, facial appearances, positions, sizes, angles, or the number of people.

6. The human photo search system of claim 1, wherein the block-based index further includes human attribute scores, facial appearance similarity scores, and photo aesthetic scores.

7. The human photo search system of claim 6, wherein the photo search server further includes a human attribute detection module for performing attribute detection on a person in the human photo to generate attribute scores of the person.

8. The human photo search system of claim 6, wherein the photo search server further includes a facial appearance similarity estimation module for obtaining a specific representation of a human face in the human photo to evaluate the facial appearance similarity between two faces.

9. The human photo search system of claim 6, wherein the photo search server further includes an aesthetics assessment module for performing aesthetics assessment on a human photo in the human photo database to generate an aesthetic score for the photo.

10. The human photo search system of claim 9, wherein the photo search server further includes an aesthetic filtering module that performs filtering on the aesthetic scores of the candidate photos, such that the display module displays only those candidate photos with aesthetic scores higher than a predetermined value.