Finding images with multiple people or objects

• US 20070177805A1
• Filed: 01/27/2006
• Published: 08/02/2007
• Est. Priority Date: 01/27/2006
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Gallagher, Andrew
• Granted Patent: US 7,711,145 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06V 10/84 using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

A method of searching through a collection of images, includes providing a list of individuals of interest and features associated with such individuals; detecting people in the collection images; determining the likelihood(s) for each listed individual of appearing in each collection image in response to the detected people and the features associated with the listed individuals; and selecting in response to the determined likelihood(s) a number of collection images such that each individual from the list appears in the selected number of collection images.

Claims

1. A method of searching through a collection of images, comprising:

(a) providing a list of individuals of interest and features associated with such individuals;

(b) detecting people in the collection images;

(c) determining the likelihood(s) for each listed individual of appearing in each collection image in response to the detected people and the features associated with the listed individuals; and

(d) selecting in response to the determined likelihood(s) a subset of one or more collection images such that at least two individuals from the list appear in the selected number of collection images.

2. The method of claim 1, wherein the list of individuals includes a number of times c_r that an individual should appear in selected subset of collection images, and the selected subset of collection images is chosen such that each individual from the list appears at least the number of times c_r in the selected number of collection images.

3. The method of claim 1, wherein the selected subset of collection images are used to produce an image product.

4. The method of claim 3, wherein the image product is a framed photograph set, a photo calendar, an album, a slide show, a web page, a mug, a t-shirt, a mouse pad, or a puzzle.

5. The method of claim 1, wherein the step of determining the likelihood(s) further includes:

i) using a first classifier to determine the likelihood that a detected person is a particular individual from the list individuals of interest; and

ii) using a second classifier and the likelihoods from c) to determine the likelihood that a particular image contains a particular individual from the list.

6. The method of claim 1, wherein at least one image in the collection of images contains more than one person.

7. The method of claim 1, wherein the selected subset of images is displayed to a user for review.

8. The method of claim 7, wherein the user selects an image from the selected subset of images and that image is replaced with another image from the digital image collection.

9. The method of claim 1, wherein the step of determining the likelihood(s) further includes:

i) detecting features for the detected people;

ii) forming a probability network of links and nodes using the detected features for the detected people; and

iii) using the probability network and the features associated with individuals of interest to determine the likelihood(s) for each listed individual of appearing in each collection image.

10. The method of claim 9, wherein the detected features include local features associated with the detected person and global features that are independent of the identity or position of the individual in the image associated with the detected person.

11. The method of claim 1, wherein the likelihoods are displayed to a user and the user provides input indicating the correctness of the labels.

12. The method of claim 12, wherein the likelihoods are updated based on the user input.

13. A method of searching through a collection of images, comprising:

(a) providing a list of individuals of interest and features associated with such individuals;

(b) detecting people in the collection images;

(c) determining the likelihood(s) for each listed individual of appearing in each collection image in response to the detected people and the features associated with the listed individuals;

(d) selecting in response to the determined likelihood(s) one or more subset(s) of collection images such that each individual from the list appears in each subset of collection images;

(e) displaying the one or more subsets of collection images to a user; and

(f) the user selecting one of the subsets of collection images.

14. The method of claim 13, wherein the images from the selected subset of collection images from f) are used to produce an image product.

15. The method of claim 14, wherein the image product is a framed photograph set, a photo calendar, an album, a slide show, a web page, a mug, a t-shirt, a mouse pad, or a puzzle.

16. A method of determining the identities of individuals in images from an image collection having two or more individuals, comprising:

(a) providing a list of individuals of interest and features associated with such individuals;

(b) detecting people in the collection images;

(c) using a first classifier to determine the likelihood that a detected person is a particular individual from the list individuals of interest; and

(d) using a second classifier and the likelihoods from c) to determine the likelihood that a particular image contains a particular individual from the list.

17. A method of determining the identities of individuals in images from an image collection having two or more individuals, comprising:

(a) providing a list of individuals of interest and features associated with such individuals;

(b) detecting people in the collection images;

(c) using a first classifier to determine the likelihood that a detected person is a particular individual from the list individuals of interest; and

(d) using the likelihood(s) from c) and a rule that an individual cannot appear more than once in an image and can only be one individual from the list to determine the likelihood that a particular image contains a particular individual from the list.

18. A method of searching through a collection of images, comprising:

(a) providing a list of objects of interest and features associated with such objects;

(b) detecting objects in the collection images;

(c) determining the likelihood(s) for each listed object of appearing in each collection image in response to the detected objects and the features associated with the listed objects; and

(d) selecting in response to the determined likelihood(s) a subset of one or more collection images such that each object from the list appears in the selected number of collection images.

19. The method of claim 18, wherein the images from the selected subset of collection images are used to produce an image product.

20. The method of claim 19, wherein the image product is a frames photograph set, a photo calendar, an album, a slide show, a web page, a mug, a t-shirt, a mouse pad, or a puzzle.

21. A method of searching through a collection of images, comprising:

(a) providing a list of individuals of interest and features associated with such individuals;

(b) detecting people in the collection images;

(c) determining the likelihood(s) for each listed individual of appearing in each collection image in response to the detected people and the features associated with the listed individuals;

(d) selecting in response to the determined likelihood(s) a subset of one or more collection images; and

(e) determining a likelihood that the selected number of images contains each of the individuals of interest.