PHOTOGRAPHY ASSISTANT AND METHOD FOR ASSISTING A USER IN PHOTOGRAPHING LANDMARKS AND SCENES

• US 20110314049A1
• Filed: 06/22/2010
• Published: 12/22/2011
• Est. Priority Date: 06/22/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Poirier, Hervé, Perronnin, Florent, Jarmasz, Mario Agustin Ricardo
• Granted Patent: US 8,332,429 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: H04N 1/00183 Photography assistance, e.g. displaying suggestions to the user
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A method and system to help photographers to take better quality pictures of landmarks and scenes are disclosed. A user is guided with examples of existing quality images, which are extracted from a database, of the same or similar landmarks or scenes. The method includes taking a query photograph that may include an image associated with a GPS location and other metadata, and using information extracted from the image to retrieve existing, similar images. The images retrieved may be ordered according to different criteria. When a user selects one as a model image, the user is provided with assistance for taking a target photograph of similar quality.

Claims

1. A method for assisting a user in taking photographs comprising:

receiving a query photograph comprising an image;

with a processor, extracting an image signature from the query photograph based on content-related features of the image;

querying a database of photographs based on the image signature to retrieve photographs similar to the query image;

presenting at least a subset of the retrieved photographs for the user to select a model photograph from among the presented photographs;

based on a user-selected model photograph, generating instructions for taking a target photograph; and

presenting the instructions to the user for taking the target photograph.

2. The method of claim 1, wherein the presenting of the at least a subset of the images includes presenting images based on at least one sorting criterion.

3. The method of claim 1, wherein the sorting criterion is selected from the group consisting of a quality indication, a visual similarity, location information, setting information for an image capture device, timing information, and combinations thereof.

4. The method of claim 1, wherein the query photograph comprises a landmark or a scene.

5. The method of claim 1, wherein the query photograph comprises metadata, and wherein the querying of the database is based on at least some of the metadata.

6. The method of claim 5, wherein the metadata comprises at least one of:

a global positioning system (GPS) location, settings of an image capture device which generated the photograph, model information of the image capture device, timing information, and information related to the query photograph.

7. The method of claim 1, wherein presenting the instructions comprises at least one of:

a) where the model photograph is associated with a GPS location, at least one of;

indicating to the user on a map a location where the model photograph was taken, and

providing directions to that location;

b) where the model photograph is associated with settings, providing suggestions to the user related to at least one setting for taking the target photograph, based on the settings of the model photograph;

or

c) presenting the model photograph in one display of a double display or displaying a landmark or scene within the model photograph in a background while the target photograph is concurrently displayed on the foreground using a transparency effect.

8. The method of claim 1, wherein presenting the instructions comprises computing geometrical transformation data between a view within a frame of an image capture device and the model photograph and at least one of:

presenting directions for increasing a similarity level between the view and the model photograph to the user, and

providing a signal when the view within the frame of the image capture device is determined to be substantially similar to the model photo.

9. The method of claim 1, wherein presenting the assistance comprises:

detecting a conflict for taking the target photograph having a similar quality to the model photograph, based on time information associated with the model photograph, and

proposing to the user a suitable future time to take the target photograph when environmental conditions are predicted to be similar to those when the model photograph was taken.

10. The method of claim 9, wherein the environmental conditions comprise at least one of an amount of sunlight and a weather condition.

11. The method of claim 1, wherein the querying of the database comprises comparing the image signature of the query photograph to image signatures of images in the database.

12. The method of claim 1 further comprising filtering the retrieved images based on at least one of a GPS location associated with the images and comments associated with the images which relate to at least one of a location of the image and a quality of the image.

13. The method of claim 1, wherein the image capture device comprises a camera or a mobile phone capable of communicating the query image and receiving retrieved images over a wireless network.

14. A computer program product encoding instructions which, when executed by a computing device, perform the method of claim 1.

15. A system for assisting a user in taking photographs, comprising:

memory which stores;

a user selection component for receiving a user'"'"'s selection of a query photograph and of a model photograph;

an image shuttle for transferring the query photograph to an associated query system for querying a database based on an image signature extracted from the query image and for receiving images retrieved from the database in response to the query;

a display generator for generating a display of at least a subset of the retrieved images on an associated display device from which the model photograph is selected; and

an instruction generator configured to present instructions to the user for taking a target photograph based on the selected model photograph; and

a processor in communication with the memory for implementing the user selection component, image shuttle, display generator, instruction generator.

16. The system of claim 15, wherein the memory further stores a sorting component for sorting the retrieved photographs based on at least one of a quality indication, a similarity weighting, a location information, setting information for an image capture device, and a timing information related to the query photograph.

17. The system of claim 15, wherein the query system queries the database based on metadata associated with the image.

18. The system of claim 15, wherein the image signature further comprises information variables having at least one of a viewpoint, zooming, light, occlusion and quality information.

19. The system of claim 15, further comprising a geometrical transformation engine configured to compute a geometrical transformation between a view in a frame of an image capture device and the model photograph and for assisting a user in aligning the view with the model photograph.

20. The system of claim 15, further comprising an image categorization mechanism coupled to the instruction generator and configured to categorize the model photograph and the target photograph into a time of day, and/or weather information for providing suggestions to the user.

21. An image capture device comprising the system of claim 15, and the display device.

22. The image capture device of claim 21, wherein the image capture device comprises a camera or a mobile phone capable of communicating the query photograph and receiving the retrieved photographs over a wireless network.

23. A method for assisting a user in taking photographs executed by at least one processor with at least one memory storing executable instructions for performing the method, comprising:

receiving a selected query photograph comprising at least one image;

retrieving photographs from a database based on a comparison of the information extracted from the query photograph with information extracted from photographs in the database;

presenting at least a subset of the retrieved photographs for the user to select a model photograph therefrom;

receiving a selection of a model photograph from among the presented photographs; and

generating instructions to guide the user for taking a target photograph comprising a similar quality to the selected model photograph.

24. The method of claim 23, wherein the information comprises image signatures respectively extracted from the query photograph and database photographs based on features of the respective photographs.

25. The method of claim 24, wherein the information further comprises metadata associated with the query image and database images respectively.