Method and apparatus for augmenting media content

• US 9,451,335 B2
• Filed: 04/29/2014
• Issued: 09/20/2016
• Est. Priority Date: 04/29/2014
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Gopalan, Raghuraman, Begeja, Lee, Gibbon, David Crawford, Liu, Zhu, Reibman, Amy Ruth, Renger, Bernard S., Shahraray, Behzad, Zavesky, Eric
• Primary Examiner(s): FLYNN, RANDY A
• Application Number:
  US14/264,580
• Publication Number:
  US 20150312649A1
• Time in Patent Office: 875 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 21/231 Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/4542 Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8405 : represented by keywords
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

Aspects of the subject disclosure may include, for example, generating narrative descriptions corresponding to visual features, visual events, and interactions there between for media content, where the narrative descriptions are associated with time stamps of the media content, and presenting the media content and an audio reproduction of the narrative descriptions, wherein the audio reproduction is synchronized to video of the media content according to the time stamps. Other embodiments are disclosed.

Claims

1. A device, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising;

scanning a plurality of images from a media content item to detect a plurality of visual features present within the plurality of images;

analyzing interactions between the plurality of visual features to determine a plurality of visual events that are depicted in the media content item;

generating a plurality of narrative descriptions corresponding to the plurality of visual features and the plurality of visual events, wherein the plurality of narrative descriptions are associated with a plurality of time stamps of the media content item;

determining visual capabilities of a user for viewing the media content item;

filtering the plurality of narrative descriptions according to requirements of an application and according to the visual capabilities of the user of the media content item to generate a plurality of filtered descriptions;

presenting the media content item and a first portion of the plurality of filtered descriptions via the application;

determining if a feature in the plurality of narrative descriptions has been provided in the first portion of the filtered description; and

presenting a shortened description of the feature in a second portion of the filtered description responsive to determining that the feature has been provided in the first portion, wherein the shortened description occupies a smaller memory space than a non-shortened description of the feature provided in the first portion of the filtered description.

2. The device of claim 1, wherein the filtered descriptions comprise emotional states in the media content item, and wherein the operations further comprise:

recognizing facial expressions of persons in the visual features in the media content item; and

determining the emotional states in the media content according to the facial expressions that are recognized.

3. The device of claim 1, wherein the operations further comprise:

searching the plurality of filtered descriptions based on a search term;

selecting a video clip from the media content item, wherein a filtered description of the plurality of filtered descriptions substantially matches the search term;

generating a speech rendering corresponding to the filtered descriptions; and

presenting the video clip and the speech rendering via the application.

4. The device of claim 1, wherein the operations further comprise filtering the plurality of visual features according to second requirements of the application.

5. The device of claim 1, wherein the plurality of narrative descriptions comprise natural language descriptions of the plurality of visual features and the plurality of visual events.

6. The device of claim 1, wherein the operations further comprise analyzing audio content of the media content item to extract audio information associated with the plurality of visual features, wherein the step of analyzing interactions further includes the audio information.

7. The device of claim 1, wherein the operations further comprise analyzing lighting characteristics of the plurality of visual features to extract mood information, wherein the step of analyzing interactions further includes the mood information.

8. The device of claim 1, wherein the filtering of the narrative descriptions according to second requirements of a profile of user associated with the application.

9. The device of claim 1, wherein the operations further comprise:

comparing the plurality of narrative descriptions to a plurality of target event descriptions;

extracting a portion of the media content item corresponding to a narrative description of the plurality of narrative descriptions that substantially matches a target event description of the plurality of target event descriptions; and

presenting the portion of the media content item.

10. The device of claim 1, wherein the operations further comprise generating a graphical representation of a portion of the plurality of narrative descriptions.

11. The device of claim 1, wherein the operations further comprise:

receiving the media content item via a first channel of a media source network;

receiving a second media content item via a second channel of the media source network; and

presenting the second media content item, wherein the plurality of narrative descriptions associated with the media content item is generated during the presenting of the second media content item.

12. The device of claim 1, wherein the operations further comprise:

comparing the plurality of narrative descriptions to a plurality of objectionable event descriptions; and

censoring a portion of the media content item corresponding to a narrative description of the plurality of narrative descriptions that substantially matches an objectionable event description of the plurality of objectionable event descriptions.

13. The device of claim 12, wherein the censoring comprises removing the portion of the media content item.

14. The device of claim 12, wherein the censoring comprises generating an advisory of the objectionable event.

15. The device of claim 1, wherein the operations further comprise receiving the media content item via a video capturing device, wherein the plurality of narrative descriptions are generated in nearly real-time with the receiving of the media content item.

16. The device of claim 15, wherein the video capturing device comprises a wearable camera device.

17. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:

determining a plurality of visual events in first media content according to interactions between a plurality of visual features in the first media content;

generating a first group of narrative descriptions corresponding to the plurality of visual features and the plurality of visual events, wherein the first group of narrative descriptions are associated with a plurality of time stamps of the first media content;

comparing the first group of narrative descriptions to a second group of narrative descriptions associated with second media content to determine a semantic correlation between the first media content and the second media content;

generating a recommendation for the first media content according to the semantic correlation and user affinity information associated with the second media content;

receiving information that is associated with visual capabilities of a viewer of the first media content;

transmitting a first portion of a filtered description of a plurality of narrative descriptions that are associated with the first media content according to the visual capabilities of the viewer;

determining if a feature in the plurality of narrative descriptions has been provided in the first portion of the filtered description; and

presenting a shortened description of the feature in a second portion of the filtered description responsive to determining that the feature has been provided in the first portion, wherein the shortened description occupies a smaller memory space than a non-shortened description of the feature provided in the first portion of the filtered description.

18. The non-transitory machine-readable storage medium of claim 17, wherein the operations further comprise:

comparing the first group of narrative descriptions to a target descriptor; and

determining a genre category for the first media content according to a portion of the first group of narrative descriptions substantially matching the target descriptor, wherein the recommendation comprises the genre category.

19. A method, comprising:

generating, by a system comprising a processor, narrative descriptions corresponding to visual features, visual events, and interactions there between for media content, wherein the narrative descriptions are associated with time stamps of the media content;

determining, by the system, visual capabilities of a user for viewing the media content;

presenting, by the system, the media content and a first portion of an audio reproduction of the narrative descriptions, wherein the audio reproduction is synchronized to video of the media content according to the time stamps, and wherein the audio reproduction comprises information consistent with the visual capabilities of the user;

determining if a feature in the narrative descriptions has been provided in the first portion of the audio reproduction of the narrative descriptions; and

presenting a shortened description of the feature in a second portion of the audio reproduction responsive to determining that the feature has been provided in the first portion, wherein the shortened description occupies a smaller memory space than a non-shortened description of the feature provided in the first portion of the audio reproduction of the narrative descriptions.

20. The method of claim 19, comprising:

comparing the narrative descriptions to target event descriptions;

extracting a portion of the media content corresponding to a narrative description of media content that substantially matches a target event description; and

marking the portion of the media content for selective presentation.