SYSTEMS AND METHODS FOR IMAGE MANAGEMENT

• US 20140093174A1
• Filed: 09/28/2012
• Published: 04/03/2014
• Est. Priority Date: 09/28/2012
• Status: Abandoned Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Denney, Bradley Scott, Lu, Juwei, Zhang, Liyan
• Application Number:
  US13/629,948
• Publication Number:
  US 20140093174A1
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene

Abstract

Systems and methods for organizing images extract low-level features from an image of a collection of images of a specified event, wherein the low-level features include visual characteristics calculated from the image pixel data, and wherein the specified event includes two or more sub-events; extract a high-level feature from the image, wherein the high-level feature includes characteristics calculated at least in part from one or more of the low-level features; identify a sub-events in the image based on the high-level feature and a predetermined model of the specified event, wherein the predetermined model describes a relationship between two or more sub-events; and annotate the image based on the identified sub-event.

Claims

1. A method comprising:

extracting low-level features from an image of a collection of images of a specified event, wherein the low-level features include visual characteristics calculated from the image pixel data, and wherein the specified event includes two or more sub-events;

extracting a high-level feature from the image, wherein the high-level feature includes characteristics calculated at least in part from one or more of the low-level features of the image;

identifying a sub-event in the image based on the high-level feature and a predetermined model of the specified event, wherein the predetermined model describes a relationship between two or more sub-events; and

annotating the image based on the identified sub-event.

2. The method of claim 1, wherein identifying the sub-event in the image is further based at least in part on a respective sub-event score of the image that is based on the low-level features.

3. The method of claim 2, wherein the sub-event score is a sub-event probability.

4. The method of claim 3, wherein the sub-event probability based on the low-level features is determined using a probability mixture model trained with a second collection of sub-event-labeled images.

5. The method of claim 2, wherein the low-level features are represented with a lower dimensional representation, wherein the dimensionality of the low-level features in the lower dimensional representation is reduced using principal component analysis.

6. The method of claim 1, wherein the low-level features include one or more of a color-based feature, a texture-based feature, an edge-based feature, and a local image descriptor.

7. The method of claim 1, wherein the low-level features include one or more of time, geo-location, ISO setting, aperture, exposure, focus, flash, camera mode, and camera model.

8. The method of claim 1, wherein the high-level feature is an adjusted time, a classifier-based location determination, a face detection, a face clustering, or an activity determination.

9. The method of claim 1, wherein identifying the sub-event further comprises:

training a hidden Markov model using a second collection of ordered sub-event-labeled images; and

estimating a sub-event sequence from the hidden Markov model.

10. The method of claim 1, further comprising choosing representative images of the sub-event from a plurality of images in the collection of images for inclusion in an image summary collection.

11. A system for organizing images, the system comprising:

at least one computer-readable medium configured to store images; and

one or more processors configured to cause the system to

extract low-level features from a collection of images of an event, wherein the specified event includes one or more sub-events;

extract a high-level feature from one or more images based on the low-level features;

identify one or more sub-events corresponding to one or more images in the collection of images based on the high-level feature and a predetermined model of the event, wherein the predetermined model defines the one or more sub-events; and

label the one or more images based on the recognized corresponding sub-events.

12. The system of claim 11, wherein the predetermined model of the event describes one or more of a temporal order of sub-events and respective high-level features that are associated with the one or more sub-events.

13. The system of claim 12, wherein the respective high-level features that are associated with the one or more sub-events include one or more of location of an image, time an image was captured, people in an image, non-people objects in an image, and activities in an image.

14. The system of claim 11, wherein the system uses a Hidden Markov Model to recognize the one or more sub-events, wherein observed states correspond to high-level features and unobserved states correspond to the sub-events.

15. One or more computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

quantifying low-level features of images of a collection of images of an event;

quantifying one or more high-level features of the images based on the low-level features; and

associating images with respective sub-events based on the one or more high-level features of the images and a predetermined model of the event that defines the sub-events.

16. The one or more computer-readable media of claim 15, wherein the operations further comprise training the predetermined model of the event based on a training set of images that are labeled according to the sub-events.

17. The one or more computer-readable media of claim 15, wherein the operations further comprise modeling respective relationships between the low-level features and the sub-events, and wherein the images are associated with the respective sub-events based on the respective relationships between the low-level features and the sub-events.

18. The one or more computer-readable media of claim 15, wherein the operations further comprise selecting respective representative images for the sub-events.

19. The one or more computer-readable media of claim 15, wherein the operations further comprise locating example images of the respective sub-events.

20. The one or more computer-readable media of claim 15, wherein the operations further comprise generating a series of camera setting groups for an expected sub-event based on the example images, wherein each camera setting group includes one or more setting parameters that are different from the settings in the other camera setting groups.