AUTOMATIC STORY CREATION USING SEMANTIC CLASSIFIERS FOR DIGITAL ASSETS AND ASSOCIATED METADATA

• US 20080304808A1
• Filed: 11/06/2007
• Published: 12/11/2008
• Est. Priority Date: 06/05/2007
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Wood, Mark D., Costello, Kathleen M., Poetker, Robert B., Newell, Catherine D.
• Granted Patent: US 8,934,717 B2
• Time in Patent Office: Days
• US Class Current: 386/52
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded

Abstract

A method and system for automatically creating an image product based on media assets stored in a database. A number of stored digital media files are analyzed to determine their semantic relationship to an event and are classified according to requirements and semantic rules for generating an image product. Rule sets are applied to assets for finding one or more assets that can be include in a story product. The assets, which best meet the requirements and rules of the image product are included.

Claims

1. A computer implemented method for automatically producing a story product comprising the steps of:

automatically applying a rule set to a plurality of digital image assets;

identifying ones of the digital image assets that satisfy the rule set;

automatically producing a story product containing identified ones of the digital image assets; and

notifying a user that the story product has been generated.

2. The method of claim 1, further comprising the step of indexing the digital image assets before the step of applying a rule set.

3. The method of claim 1, further comprising the steps of:

producing a story product representation of the story product before the step of producing the story product;

notifying the user that the representation has been produced before the step of producing the story product; and

receiving an approval of the representation from the user before the step of producing the story product.

4. The method of claim 2, wherein the step of indexing the digital image assets includes a step selected from the steps consisting of:

scene classifying the digital image assets including storing scene metadata with a corresponding asset;

event classifying the digital image assets including storing event metadata with a corresponding asset; and

person classifying the digital image assets including storing person identification metadata with a corresponding asset.

5. The method of claim 4, wherein the step of scene classifying the digital image assets further includes a step selected from the steps consisting of:

classifying as indoor;

classifying as outdoor;

classifying as natural;

classifying as manmade;

classifying as sunset;

classifying as beach;

classifying as fall foliage;

classifying as field;

classifying as mountain;

classifying as sky;

classifying as grass;

classifying as snow;

classifying as water;

classifying as sand; and

classifying as clouds.

6. The method of claim 4, wherein the step of event classifying the digital image assets further includes a step selected from the steps consisting of:

classifying as vacation;

classifying as sports;

classifying as party; and

classifying as family moments.

7. The method of claim 4, wherein the step of person classifying the digital image assets further includes the step of storing with a corresponding digital image asset a name of a person who appears in the corresponding digital image asset.

8. The method of claim 1 wherein the step of applying the rule set is preceded by the step of monitoring a storage device for an upload of the digital image assets and beginning the applying step in response thereto.

9. The method of claim 1 wherein the step of applying the rule set is preceded by the step of detecting an upcoming holiday and beginning the applying step in response thereto, and wherein the step of applying the rule set includes the step of selecting a rule set for application that corresponds to the detected upcoming holiday.

10. The method of claim 1 wherein the step of applying the rule set is preceded by the step of detecting an upcoming calendar date and beginning the applying step in response thereto, and wherein the step of applying the rule set includes the step of selecting a rule set for application that corresponds to the detected calendar date.

11. The method of claim 1 wherein the step of notifying the user includes the step of transmitting the story product to a user'"'"'s device.

12. The method of claim 11 wherein the user'"'"'s device is selected from the group consisting of:

a set top box;

a cell phone;

a PDA;

a home media server;

a digital picture frame;

a printer; and

a PC.

13. The method of claim 1 wherein the rule set includes a step selected from the group consisting of:

requiring a depiction of a particular person in a digital image asset;

requiring a depiction of a particular person and a relative of the particular person in a digital image asset;

requiring a depiction of a particular person and a friend of the particular person in a digital image asset;

requiring a particular date range of a digital image asset;

requiring a minimum value index of a digital image asset;

requiring a particular range of playback time of a digital image asset;

requiring a particular owner of a digital image asset;

selecting a best one of a group of digital image assets;

requiring an event classification of a digital image asset;

requiring a scene classification of a digital image asset;

requiring a depiction of a particular logo in a digital image asset;

requiring a depiction of a particular number of people in a digital image asset;

requiring an absence of persons in a digital image asset;

requiring that a digital image asset is a member of a particular class of assets; and

requiring that a recipient enjoys an event class in which a digital image asset belongs.

25. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform the method steps of claim 1.

14. A computer system comprising:

an asset uploader for receiving a plurality of digital image assets to be stored, the digital image assets including metadata;

an asset store for storing a plurality of digital assets;

a database for storing the metadata;

a semantic indexer for indexing the digital assets according to their content and for storing indexing information as metadata in the database associated with corresponding digital assets;

a rule base comprising a plurality of story rule sets each of which is capable of being satisfied by metadata stored in the database; and

a product generator for generating a story product that includes at least one digital asset whose metadata satisfies one of the story rule sets.

15. The computer system of claim 14 wherein the semantic indexer comprises:

a scene classifier for categorizing a scene depicted in a digital image asset into at least one scene type and for storing the scene type in the database in association with a corresponding digital image asset.

16. The computer system of claim 14 wherein the semantic indexer comprises:

an event classifier for categorizing an event depicted in a digital image asset into at least one event type and for storing the event type in the database in association with a corresponding digital image asset.

17. The computer system of claim 14 wherein the semantic indexer comprises:

a people recognition component for identifying a person depicted in a digital image asset and for storing a name of the person in the database in association with a corresponding digital image asset.

18. The computer system according to claim 14 further comprising:

an inferencing engine for evaluating the rule sets to identify one or more of the digital image assets having metadata associated therewith that satisfy a story rule set.

19. The computer system according to claim 18 wherein one of the rule sets comprises a plurality of pre-selected story rules for identifying one or more of the digital image assets that satisfy a pre-selected theme.

20. The computer system according to claim 18 further comprising an asset selector for identifying one or more of the digital image assets that satisfy a story rule set that also fail an image value rule.

21. The computer system according to claim 19 wherein the pre-selected theme comprises a theme selected from the group consisting of:

celebratory;

religious;

interest group;

pets;

holiday;

anniversary;

solemn;

birthday;

sports;

vacation;

hobby;

leisure; and

family.

22. The computer system according to claim 14 further comprising a story notifier service for notifying a user that the story product has been generated.

23. The computer system according to claim 22 wherein the story notifier service includes a device for transmitting a viewable representation of the story product to a user device having a display for a user'"'"'s approval.

24. The computer system according to claim 14 further comprising:

a musical asset uploader for receiving a plurality of digital music assets to be stored, the digital music assets including metadata, and wherein the asset store includes the plurality of digital music assets, the database includes the digital music asset metadata, the rule base includes a story rule set capable of being satisfied by digital music asset metadata stored in the database, and wherein the product generator includes a music product generator for including in the story product at least one digital music asset whose metadata satisfies the story rule set capable of being satisfied by digital music asset metadata.