System and Method for Deriving Timeline Metadata for Video Content

• US 20150254341A1
• Filed: 03/10/2014
• Published: 09/10/2015
• Est. Priority Date: 03/10/2014
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Rai, Pete, Richardson, David
• Granted Patent: US 10,349,093 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

In one embodiment, a method implemented on a computing device for deriving timeline metadata for video content includes: capturing timeline elements through analysis of at least one of audio, visual or language aspects of the video content, interpreting the timeline elements according to associated inferences as indicative of timeline states, evaluating combinations of the timeline states according to a set of rules to derive timeline metadata, where the timeline metadata is associated with at least a portion of the video content.

Claims

1. A method implemented on a computing device for deriving timeline metadata for video content, the method comprising:

capturing timeline elements through analysis of at least one of audio, visual or language aspects of said video content;

interpreting said timeline elements according to associated inferences as indicative of timeline states; and

evaluating combinations of said timeline states according to a set of rules to derive timeline metadata, wherein said timeline metadata is associated with at least a portion of said video content.

2. The method according to claim 1 and also comprising:

augmenting said timeline elements with information from other data sources.

3. The method according to claim 1 and wherein said evaluating comprises:

assigning a probability of correctness to said derived timeline metadata.

4. The method according to claim 3 and also comprising:

receiving feedback indicative of said correctness; and

adjusting said probability in light of said feedback.

5. The method according to claim 4 and wherein said receiving and said adjusting are performed iteratively.

6. The method according to claim 1 and also comprising:

detecting recurring said timeline elements for which there are no associated inferences.

7. The method according to claim 6 and also comprising:

determining timeline states for said recurring timeline elements based on at least the timeline states of other timeline elements that have been observed in proximity to said recurring elements.

8. The method according to claim 1 and also comprising:

detecting recurring combinations of said timeline elements which are not addressed in said set of rules.

9. The method according to claim 8 and also comprising:

determining new rules for said recurring combination of elements based on at least one existing rule for similar said combinations of said timeline states, wherein at least one timeline state of said similar combinations is associated with a said timeline element in said recurring combination of elements.

10. The method according to claim 4 wherein said feedback comprises feedback regarding at least two versions of said timeline metadata.

11. The method according to claim 1 and wherein said evaluating comprises:

determining a context for said at least a portion of said video content based at least on said timeline metadata.

12. The method according to claim 11 and wherein said determining is performed iteratively.

13. A timeline metadata derivation system implemented on a computing device, the system comprising:

means for capturing timeline elements through analysis of at least one of audio, visual or language aspects of said video content;

means for interpreting said timeline elements according to associated inferences as indicative of timeline states; and

means for evaluating combinations of said timeline states according to a set of rules to derive timeline metadata, wherein said timeline metadata is associated with at least a portion of said video content.

14. A video content timeline metadata generation system implemented on a computing device, the system comprising:

at least one timeline element capture module operative to analyze video content items in order to at least detect timeline elements associated with at least points in time in said video content item;

a timeline element interpreter operative to at least interpret said timeline elements as timeline states in accordance with associated inferences; and

a timeline metadata engine operative to evaluate combinations of said timeline states according to a set of rules to derive timeline metadata, wherein said timeline metadata is associated with at least portions of said video content items.

15. The system according to claim 14 and wherein said at least one timeline element capture module is at least one of a video analyzer, an audio analyzer, a language analyzer or a metadata reader.

16. The system according to claim 14 and wherein said timeline metadata engine is operative to associate a probability for correctness for said derived timeline metadata.

17. The system according to claim 16 and wherein said timeline metadata engine comprises a feedback unit operative:

to receive feedback from external feedback sources regarding said probability of correctness; and

to adjust said probability of correctness in view of at least said feedback.

18. The system according to claim 17 and wherein said feedback unit is operative to receive said feedback and to adjust said probability of correctness iteratively.

19. The system according to claim 14 and also comprising a strand detector operative to detect recurring said timeline elements for which there are said associated inferences.

20. The system according to claim 19 and wherein said strand detector is also operative to detect recurring combinations of said timeline elements which are not addressed in said set of rules.