Automated Content Detection, Analysis, Visual Synthesis and Repurposing

• US 20110093798A1
• Filed: 10/15/2009
• Published: 04/21/2011
• Est. Priority Date: 10/15/2009
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Shahraray, Behzad, Liu, Zhu, Renger, Bernard S., Basso, Andrea, Gibbon, David C., Begeja, Lee
• Granted Patent: US 9,167,189 B2
• Time in Patent Office: Days
• US Class Current: 715/763
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A content summary is generated by determining a relevance of each of a plurality of scenes, removing at least one of the plurality of scenes based on the determined relevance, and creating a scene summary based on the plurality of scenes. The scene summary is output to a graphical user interface, which may be a three-dimensional interface. The plurality of scenes is automatically detected in a source video and a scene summary is created with user input to modify the scene summary. A synthetic frame representation is formed by determining a sentiment of at least one frame object in a plurality of frame objects and creating a synthetic representation of the at least one frame object based at least in part on the determined sentiment. The relevance of the frame object may be determined and the synthetic representation is then created based on the determined relevance and the determined sentiment.

Claims

1. A method of automatically generating a summary of content comprising:

determining a relevance of each of a plurality of scenes of the content;

extracting at least one of the plurality of scenes based on the determined relevance; and

generating a scene summary based on the extracted at least one of the plurality of scenes.

2. The method of claim 1, wherein the scene summary is output to a graphical user interface.

3. The method of claim 2, wherein the graphical user interface is a three-dimensional interface.

4. The method of claim 1, further comprising:

automatically detecting the plurality of scenes in a source video.

5. The method of claim 1, wherein creating a scene summary based on the plurality of scenes further comprises:

receiving an indication from a user to modify the scene summary.

6. The method of claim 1, wherein said step of determining a relevance of each of a plurality of scenes of the content comprises:

detecting at least one feature for each of the plurality of scenes; and

determining the relevance of each of the plurality of scenes based on the detected at least one feature.

7. A method of synthetic frame representation comprising:

determining a sentiment of at least one frame object in a plurality of frame objects; and

creating a synthetic representation of the at least one frame object based at least in part on the determined sentiment.

8. The method of claim 7, further comprising:

determining a relevance of the at least one frame object in a plurality of frame objects;

creating the synthetic representation of the at least one frame object based at least in part on the determined relevance and the determined sentiment.

9. The method of claim 7, further comprising:

determining the plurality of frame objects in a source video.

10. The method of claim 7, further comprising:

outputting the synthetic representation to a graphical user interface.

11. The method of claim 7, further comprising:

storing a plurality of synthetic frame representations;

indexing the stored synthetic frame representations based on the sentiment represented by each of the synthetic frame representations; and

searching the stored synthetic frame representations based at least in part on a sentiment specified in a received user request.

12. A computer readable medium comprising computer program instructions for automatically generating a summary of content, the computer program instructions capable of being executed in a processor and defining the steps of:

determining a relevance of each of a plurality of scenes of the content;

extracting at least one of the plurality of scenes based on the determined relevance; and

generating a scene summary based on the extracted at least one of the plurality of scenes.

13. The computer readable medium of claim 12, wherein the instructions further define the step of:

outputting the scene summary to a graphical user interface.

14. The computer readable medium of claim 12, wherein the instructions further define the step of:

automatically detecting the plurality of scenes in a source video.

15. The computer readable medium of claim 12, wherein the instructions for creating a scene summary based on the plurality of scenes further define the step of:

receiving an indication from a user to modify the scene summary.

16. The computer readable medium of claim 12, wherein the instructions for determining a relevance of each of a plurality of scenes of the content further defined the steps of:

detecting at least one feature for each of the plurality of scenes; and

determining the relevance of each of the plurality of scenes based on the detected at least one feature

17. A computer readable medium comprising computer program instructions capable of being executed in a processor and defining the steps of:

determining a sentiment of at least one frame object in a plurality of frame objects; and

creating a synthetic representation of the at least one frame object based at least in part on the determined sentiment.

18. The computer readable medium of claim 17, wherein the instructions further define the step of:

determining a relevance of the at least one frame object in a plurality of frame objects;

creating the synthetic representation of the at least one frame object based at least in part on the determined relevance and the determined sentiment.

19. The computer readable medium of claim 17, wherein the instructions further define the step of:

determining the plurality of frame objects in a source video.

20. The computer readable medium of claim 17, wherein the instructions further define the step of:

outputting the synthetic representation to a graphical user interface.