Context dependent intelligent thumbnail images

• US 8,009,921 B2
• Filed: 02/19/2008
• Issued: 08/30/2011
• Est. Priority Date: 02/19/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Csurka, Gabriela
• Primary Examiner(s): Alavi; Amir
• Application Number:
  US12/033,434
• Publication Number:
  US 20090208118A1
• Time in Patent Office: 1,288 Days
• Field of Search: 382171-173, 382224-225, 382/228, 382/270, 382/272, 382276-277, 382282-283, 382291-292, 382/298, 382/309, 345/419, 345/421, 345/473, 345/619, 345/620, 358/538, 704/245
• US Class Current: 382/228
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 1/3873 defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming

Abstract

An apparatus and method are disclosed for context dependent cropping of a source image. The method includes identifying a context for the source image, identifying a visual class corresponding to the identified context from a set of visual classes, applying a class model to the source image to identify a candidate region of the image based on its relevance to the visual class, and identifying a subpart of the source image for cropping, based on the location of the candidate region.

Claims

1. A method for cropping a source image comprising:

identifying a context for the source image;

identifying a visual class corresponding to the identified context from a set of visual classes;

with a processor, applying a class model to the source image to identify a candidate region of the image based on its relevance to the visual class; and

identifying a subpart of the source image for cropping based on the candidate region.

2. The method of claim 1, wherein the context is identified from at least one of text, tags, and metadata of a document comprising the source image.

3. The method of claim 1, wherein the context is identified from at least one of text, tags, and metadata of a document which is designated to receive the cropped image.

4. The method of claim 1, wherein the context is derived from a search query and the source image is in a document retrieved as responsive to the search query.

5. The method of claim 1, wherein the applying of the class model includes generating a probability map in which pixels of the source image or an image derived therefrom are each associated with a probability of being in the identified visual class.

6. The method of claim 5, wherein each pixel probability is based on a probability that a region in which the pixel is located is in the visual class.

7. The method of claim 5, wherein at least some of the visual classes are object classes and wherein the class models are trained on a set of training images which have been assigned to the class.

8. The method of claim 5, wherein the applying of the class model includes applying a first class model to generate a first probability map in which pixel probabilities are based on a first feature, applying at least a second class model to generate at least a second probability map in which pixel probabilities are based on a second or subsequent feature, and combining the first and at least second probability maps.

9. The method of claim 5, further comprising thresholding the pixel probabilities to identify pixels which at least meet the threshold.

10. The method of claim 1, wherein the applying of the class model comprises identifying at least one candidate region in the source image in which pixels have at least a threshold probability for the visual class.

11. The method of claim 1, wherein a location of the candidate region is used to identify the subpart.

12. The method of claim 1, further comprising cropping the source image and forming a context dependent thumbnail image from a cropped subpart.

13. The method of claim 12, further comprising resizing the cropped subpart to form the context dependent thumbnail image.

14. The method of claim 1, wherein the class model comprises a generic visual classifier which determines features vectors for a plurality of regions in the image.

15. The method of claim 1, wherein at least a first of the visual classes relates to people and at least a second of the visual classes relates to a class of inanimate objects.

16. The method of claim 1, wherein the source image comprises at least one of the group consisting of photographs, graphics, images with associated text, tables, graphs, medical images, maps, composite images, sets of images, video images, and combinations and multiples thereof.

17. The method of claim 1, further comprising displaying a context dependent thumbnail derived from the cropped subpart.

18. A computer program product encoding instructions stored in a non-transitory computer readable medium, which when executed on a computer causes the computer to perform the method of claim 1.

19. An image cropping apparatus comprising a processor which executes instructions stored in memory for performing the method of claim 1.

20. An apparatus for cropping a source image comprising:

a context finder which automatically identifies a context for the source image;

a visual categorizer which automatically identifies a visual class corresponding to the identified context from a set of visual classes; and

applies a class model to the source image to identify a candidate region of the image based on its relevance to the visual class;

an image cropper which crops a subpart of the source image based on the candidate region;

a processor which implements the context finder visual categorizer and image cropper; and

memory which stores the source image to be cropped.

21. The apparatus of claim 20, wherein the visual categorizer comprises a set of class models, each of the class models being associated with a different one of the visual classes.

22. The apparatus of claim 20, wherein the apparatus is capable of cropping a first subpart for a first identified context and a second subpart, different from the first subpart, for a second identified context.

23. A computer implemented method of variable printing comprising:

identifying a context for a document to receive a cropped image;

identifying a visual class corresponding to the identified context from a set of visual classes;

applying a class model to a source image to identify a candidate region of the image based on its relevance to the visual class;

cropping a subpart of the source image based on the candidate region; and

incorporating the cropped subpart into the document.

24. A method for assisting a user in assessing the relevance of documents responsive to a query comprising:

receiving a user query;

retrieving a document responsive to the query;

identifying a context based on at least one of the user query and the retrieved document;

identifying a visual class corresponding to the identified context from a set of visual classes;

with a processor, applying a class model to a source image in the retrieved document to identify a candidate region of the source image based on its relevance to the visual class;

identifying a subpart of the source image for cropping based on the candidate region; and

displaying a context-dependent thumbnail derived from the identified subpart to assist a user in assessing the relevance of the document.

25. A computer program product comprising a non-transitory medium on which a control program is recorded which when executed by a computer apparatus, performs a method for cropping a source image comprising:

identifying a context for the source image;

identifying a visual class corresponding to the identified context from a set of visual classes;

applying a class model to the source image to identify a candidate region of the image based on its relevance to the visual class; and

identifying a subpart of the source image for cropping based on the candidate region.

26. The computer program product of claim 25, wherein the non-transitory medium on which the control program is recorded comprises a disk, a hard drive, a magnetic tape, a RAM, a PROM, an EPROM, a FLASH-EPROM, other memory chip, or a cartridge.