System for Social Media Tag Extraction

• US 20140366052A1
• Filed: 06/05/2014
• Published: 12/11/2014
• Est. Priority Date: 06/05/2013
• Status: Active Grant

• Current Assignee: TVEyes Inc.
• Original Assignee: TVEyes Inc.
• Inventors: Ives, David J., Seltzer, David B., Hayter, James H., Oei, Maxim
• Granted Patent: US 9,148,675 B2
• Time in Patent Office: Days
• US Class Current: 725/19
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/231 Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A system for extracting and monitoring media tags within video content includes at least one server in communication with a plurality of content sources, the server receiving video content from the content sources, a recorder saving the video content, a detector receiving at least one frame of the video content, the detector detecting one or more unknown text within the frame and creating one or more images, each image associated with one of the one or more unknown text, the detector generating metadata associated with the one or more unknown text appearing in the frame, and an optical character recognition engine scanning the one or more images and converting the one or more images into one or more known text. The server further determines that the one or more known text is a media tag.

Claims

1. A system for extracting and tracking media tags within video content, comprising:

at least one server in communication with a plurality of content sources, the server receiving video content from the content sources;

a recorder saving the video content;

a detector receiving at least one frame of the video content, the detector detecting one or more unknown text within the frame and creating one or more images, each image associated with one of the one or more unknown text, the detector generating metadata associated with the one or more unknown text appearing in the frame; and

an optical character recognition engine scanning the one or more images and converting the one or more images into one or more known text, each of the one or more known text associated with one of the one or more unknown text;

wherein the server determines whether the one or more known text is a media tag.

2. The system of claim 1, wherein the media tag is a trademark, logo, social media identifier, or uniform resource locator address.

3. The system of claim 1, wherein the detector comprises a metadata formatter for generating the metadata associated with the one or more unknown text, wherein the metadata comprises time and source information.

4. The system of claim 3, wherein the metadata comprises at least one data value from the following:

a channel or station that displayed the unknown text;

a date and time when the unknown text was displayed;

a length of time that the unknown text was displayed;

a location of the unknown text within the frame;

a size of the unknown text within the frame;

a color of the unknown text; and

dimensions of the frame.

5. The system of claim 1, wherein the server monitors social media services over a network, identifies social media events generated after the media tag is displayed in the video content, and provides analytics measurements of the social media events.

6. The system of claim 5, wherein the server receives a request from the client computer over a network to search for a term appearing in the video content, the server providing search results which comprise a media tag indicative of the term, metadata corresponding to the media tag indicative of the term, and analytics measurements of media events associated with the media tag indicative of the term.

7. The system of claim 5, wherein the server assesses a sentiment of the social media events.

8. The system of claim 1, further comprising a database for storing the media tag, the metadata associated with the media tag, the frame having the media tag, and a video clip comprising a plurality of frames before and after the frame having the media tag.

9. The system of claim 1, wherein the detector comprises:

a text locator which identifies one or more regions of the frame where the one or more unknown text is displayed, the text locator utilizing at least one of stroke width transform, edge detection, glyph detection, or logo detection to detect the one or more unknown text and to identify the one or more regions, and

a text extractor which extracts the one or more regions from the frame for creating the one or more images.

10. The system of claim 1, further comprising an image processing unit which reduces noise in the one or more images by performing at least one of converting the one or more images to grayscale, determining the average brightness of the one or more images, inverting dark areas of the one or more images, and sharpening the one or more images, and further converts the one or more images to monochrome.

11. The system of claim 1, further comprising an analyzer in communication with the optical character recognition engine, wherein the analyzer performs at least one of the following:

determines the media tag from the one or more known text, identifies that the one or more known text refers to one or more social media services, and validates whether the optical character recognition engine accurately generated the one or more known text from the one or more images using at least the metadata.

12. The system of claim 11, wherein the accuracy of the optical character recognition engine in generating the one or more known text is validated by comparing known text that appeared in previous frames of the video content.

13. The system of claim 1, wherein the recorder, detector, and optical character recognition engine comprise different software components executing on computer readable medium.

14. A system for extracting and tracking media tags within video content, comprising:

a media capture server and an analysis server, the media capture server and the analysis server being in communication with each other;

the media capture server in communication with a plurality of content sources, the media capture server receiving video content from the content sources and having a recorder which saves the video content;

a detector receiving at least one frame of the video content, the detector detecting one or more unknown text within the frame and creating one or more images, each of the one or more images associated with one of the one or more unknown text, the detector generating metadata associated with the one or more unknown text appearing in the frame;

an optical character recognition engine scanning the one or more images and converting the one or more images into one or more known text, each of the one or more known text associated with one of the one or more unknown text;

the analysis server having an analyzer which receives the one or more known text from the optical character recognition engine, the analyzer determining whether the one or more known text is a media tag; and

an interface in communication with the media capture server and the analysis server, the interface receiving a request to monitor for the media tag in the video content from a user computer;

wherein the analysis server generates alerts notifying when the media tag is detected in the video content.

15. The system of claim 14, further comprising a database in communication with the analysis server, the database storing the media tag, the metadata associated with the media tag, the frame having the media tag, and a video clip comprising a plurality of frames before and after the frame having the media tag.

16. The system of claim 14, wherein the media capture server comprises two or more media capture servers in communication with the analysis server, the media capture servers transmitting processed video content to the analysis server for centralized analysis of the media tag and social media events generated after the media tag is displayed in the video content, the media capture servers each receiving video content from different content sources.

17. The system of claim 14, wherein the detector comprises a text locator which identifies one or more regions of the frame where the one or more unknown text is displayed, and a text extractor which extracts the one or more regions from the frame for creating the one or more images.

18. The system of claim 14, wherein the at least one media capture server further comprises a content decoder in communication with the recorder, the content decoder decodes video signals from the content sources to obtain the video content and extracts the at least one frame for output to the detector.

19. The system of claim 14, wherein the analyzer validates whether the optical character recognition engine accurately generated the one or more known text from the one or more images using at least the metadata.

20. The system of claim 14, further comprising a metadata formatter, the metadata formatter packaging the one or more images and the metadata associated with the one or more unknown text into a data package file for transmission between the media capture server and the analysis server.

21. The method for extracting and tracking media tags within video content, comprising the steps of:

receiving video content with a server from one or more content sources;

saving the video content in a storage;

extracting on the server at least one frame of the video content;

detecting on the server one or more unknown text within the frame and creating one or more images, each of the one or more images associated with one of the one or more unknown text;

retrieving metadata associated with the one or more unknown text appearing in the frame;

scanning the one or more images using optical character recognition and converting the one or more images into one or more known text, each of the one or more known text associated with one of the one or more unknown text; and

determining whether the one or more known text is a media tag.

22. The method of claim 21, wherein the step of detecting comprises the steps of:

identifying one or more regions of the frame where the one or more unknown text is displayed; and

extracting the one or more regions from the frame for creating the one or more images.

23. The method of claim 21, further comprising the steps of:

processing the one or more images by reducing noise in the one or more images;

converting the one or more images to monochrome before scanning the one or more images with optical character recognition; and

packaging the one or more processed images and the metadata associated with the one or more unknown text into a data package file for transmission to an optical character recognition engine.

24. The method of claim 21, wherein the step of determining whether the one or more known text is a media tag comprises:

identifying that the one or more known text refers to one or more social media services; and

validating whether the scanning accurately generated the one or more known text from the one or more images using at least the metadata.

25. The method of claim 21, further comprising the steps of:

monitoring social media services over a network;

identifying social media events generated with the social media services after the media tag is displayed in the video content;

generating measurements and analytics of the social media events; and

displaying the media tag, the metadata, and the measurements and analytics for viewing on a user computer.

26. The method of claim 21, providing an alert when a particular media tag being monitored is detected.