Method and apparatus for augmenting media content

• US 9,769,524 B2
• Filed: 07/06/2016
• Issued: 09/19/2017
• Est. Priority Date: 04/29/2014
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Gopalan, Raghuraman, Begeja, Lee, Gibbon, David Crawford, Liu, Zhu, Reibman, Amy Ruth, Renger, Bernard S., Shahraray, Behzad, Zavesky, Eric
• Primary Examiner(s): FLYNN, RANDY A
• Application Number:
  US15/203,336
• Publication Number:
  US 20160316265A1
• Time in Patent Office: 440 Days
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 21/231 Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/4542 Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8405 : represented by keywords
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

Aspects of the subject disclosure may include, for example, generating narrative descriptions corresponding to visual features, visual events, and interactions there between for media content, where the narrative descriptions are associated with time stamps of the media content, and presenting the media content and an audio reproduction of the narrative descriptions, wherein the audio reproduction is synchronized to video of the media content according to the time stamps. Other embodiments are disclosed.

Claims

1. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, comprising:

determining visual capabilities of a user viewing a media content item at a device;

filtering a plurality of narrative descriptions associated with the media content item according to the visual capabilities of the user viewing the media content item to generate a plurality of filtered descriptions;

determining if a feature in the plurality of narrative descriptions has been provided in a first portion of the filtered description; and

replacing a non-shortened description of the feature with a shortened description of the feature in a second portion of the filtered description responsive to determining that the feature has been provided in the first portion, wherein the shortened description occupies a smaller memory space than the non-shortened description of the feature provided in the first portion of the filtered description; and

transmitting the media content item, the first portion of the plurality of filtered descriptions, and the second portion of the plurality of filtered descriptions to the device for presentation via an application.

2. The non-transitory machine-readable storage medium of claim 1, wherein the narrative description is further filtered according to requirements of the application.

3. The non-transitory machine-readable storage medium of claim 1, wherein the plurality of narrative descriptions are associated with a plurality of time stamps of the media content item.

4. The non-transitory machine-readable storage medium of claim 1, wherein the operations further comprise:

scanning a plurality of images from the media content item to detect a plurality of visual features present within the plurality of images;

analyzing interactions between the plurality of visual features to determine a plurality of visual events that are depicted in the media content item; and

generating the plurality of narrative descriptions corresponding to the plurality of visual features and the plurality of visual events.

5. The non-transitory machine-readable storage medium of claim 4, wherein the filtered descriptions comprise emotional states in the media content item, and wherein the operations further comprise:

recognizing facial expressions of persons in the visual features in the media content item; and

determining the emotional states in the media content according to the facial expressions that are recognized.

6. The non-transitory machine-readable storage medium of claim 4, wherein the operations further comprise filtering the plurality of visual features according to second requirements of the application.

7. The non-transitory machine-readable storage medium of claim 4, wherein the plurality of narrative descriptions comprise natural language descriptions of the plurality of visual features and the plurality of visual events.

8. The non-transitory machine-readable storage medium of claim 4, wherein the operations further comprise analyzing audio content of the media content item to extract audio information associated with the plurality of visual features, wherein the step of analyzing interactions further includes the audio information.

9. The non-transitory machine-readable storage medium of claim 4, wherein the operations further comprise analyzing lighting characteristics of the plurality of visual features to extract mood information, wherein the step of analyzing interactions further includes the mood information.

10. The non-transitory machine-readable storage medium of claim 1, wherein the operations further comprise:

searching the plurality of filtered descriptions based on a search term;

selecting a video clip from the media content item, wherein a filtered description of the plurality of filtered descriptions substantially matches the search term;

generating a speech rendering corresponding to the filtered descriptions; and

presenting the video clip and the speech rendering via the application.

11. The non-transitory machine-readable storage medium of claim 1, wherein the filtering of the narrative descriptions according to second requirements of a profile of user associated with the application.

12. The non-transitory machine-readable storage medium of claim 1, wherein the operations further comprise:

comparing the plurality of narrative descriptions to a plurality of objectionable event descriptions; and

censoring a portion of the media content item corresponding to a narrative description of the plurality of narrative descriptions that substantially matches an objectionable event description of the plurality of objectionable event descriptions.

13. The non-transitory machine-readable storage medium of claim 12, wherein the censoring comprises removing the portion of the media content item.

14. The non-transitory machine-readable storage medium of claim 12, wherein the censoring comprises generating an advisory of the objectionable event.

15. A communication device, comprising:

a processing system including a processor; and

a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, comprising;

providing, to a server, visual capabilities of a user viewing a media content item;

receiving, from the server, the media content item, a first portion of a plurality of filtered descriptions associated with the media content item, and a second portion of the plurality of filtered descriptions associated with the media content item, a wherein the filter descriptions are generated from a plurality of narrative descriptions associated with the media content according to the visual capabilities of the user of the media content item, and wherein the first portion includes a non-shortened description of a feature and the second portion includes a shortened description of the feature that occupies a smaller memory space than the non-shortened description; and

presenting the media content item, the first portion of the plurality of filtered descriptions associated with the media content item, and the second portion of the plurality of filtered descriptions associated with the media content item via an application.

16. The communication device of claim 15, wherein the operations further comprise:

receiving a portion of the media content item from the server, wherein the portion is extracted from the media content item corresponding to a narrative description of the plurality of narrative descriptions that substantially matches a target event description of a plurality of target event descriptions; and

presenting the portion of the media content item via the application.

17. The communication device of claim 15, wherein the operations further comprise receiving a graphical representation of a portion of the plurality of narrative descriptions.

18. The communication device of claim 15, wherein the narrative description is further filtered according to requirements of the application.

19. The communication device of claim 15, wherein the plurality of narrative descriptions are associated with a plurality of time stamps of the media content item.

20. A method, comprising:

providing, by a processing system including a processor, to a server, visual capabilities of a user viewing a media content item;

receiving, by the processing system, from the server, the media content item, a first portion of a plurality of filtered descriptions associated with the media content item, and a second portion of the plurality of filtered descriptions associated with the media content item, a wherein the filter descriptions are generated from a plurality of narrative descriptions associated with the media content according to the visual capabilities of the user of the media content item, and wherein the first portion includes a non-shortened description of a feature and the second portion includes a shortened description of the feature that occupies a smaller memory space than the non-shortened description; and

presenting, by the processing system, the media content item, the first portion of the plurality of filtered descriptions associated with the media content item, and the second portion of the plurality of filtered descriptions associated with the media content item via an application.