METHODS AND SYSTEMS FOR AGGREGATION AND ORGANIZATION OF MULTIMEDIA DATA ACQUIRED FROM A PLURALITY OF SOURCES

• US 20180109843A1
• Filed: 09/08/2017
• Published: 04/19/2018
• Est. Priority Date: 10/10/2013
• Status: Active Application

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: CHANG, Shih-Fu, JOU, Brendan, LI, Hongzhi, ELLIS, Joseph G., MOROZOFF-ABEZGAUZ, Daniel, KARAMAN, Svebor
• Granted Patent: US 10,277,946 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: H04N 21/4332 by placing content in organized collections, e.g. local EPG data repository interfaces, Database management systems or updating for information retrieval G06F16/23; details of retrieval of video data and associated meta data in video database G06F16/739
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4821 using a grid, e.g. sorted out by channel and broadcast time
• CPC Class Codes: H04N 21/4826 using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments.

Claims

1. A system for aggregation and organization of multimedia data acquired from a plurality of sources, comprising:

a data collection element configured to receive multimedia data;

a multimedia segmentation element, coupled to the data collection element, and configured to separate the multimedia data into a plurality of multimedia data segments;

a memory configured to store hierarchical category information having a plurality of categories;

an extraction element, coupled to the multimedia segmentation element and the memory, configured to associate at least one of the categories with each of the multimedia data segments; and

a user interface, coupled to the extraction element, configured to provide a user with at least one of the multimedia data segments associated with a corresponding one of the categories in the hierarchical category information.

2. The system of claim 1, wherein the data collection element further comprises a recorder configured to record audiovisual data.

3. The system of claim 2, wherein the recorder is configured to record at least one of analog video content and digital video content.

4. The system of claim 2, wherein the recorder further comprises a scheduling routine configured to query an electronic program guide feed and schedule a recording job for a news program.

5. The system of claim 1, wherein the data collection element further comprises a web crawler.

6. The system of claim 1, wherein the multimedia segmentation element further comprises a closed caption reader configured to identify multimedia data boundaries.

7. The system of claim 6, wherein the multimedia segmentation element further comprises a dynamic time warping algorithm configured to align audiovisual data and closed caption data.

8. The system of claim 1, wherein the multimedia segmentation element further comprises a speech recognition element.

9. The system of claim 1, wherein the multimedia segmentation element further comprises a shot detection element configured to identify shot boundaries.

10. The system of claim 1, wherein at least one of the plurality of categories comprises a long-term news event.

11. The system of claim 1, wherein at least one of the plurality of categories comprises a breaking news event.

12. The system of claim 1, wherein the extraction element further comprises a news video frame layout identifier.

13. The system of claim 1, wherein the extraction element further comprises a visual matching element.

14. The system of claim 1, wherein the extraction element further comprises a text-matching element.

15. The system of claim 14, wherein the text-matching algorithm further comprises a term-frequency and inverse-document-frequency (TF-IDF) algorithm.

16. The system of claim 1, wherein the extraction element further comprises a video character recognition element configured to recognize text superimposed on video frames.

17. The system of claim 1, wherein the extraction element further comprises a speaker identification element.

18. The system of claim 1, wherein the user interface further comprises a text entry box.

19. The system of claim 1, wherein the user interface further comprises a cross-domain bookmarklet feature.

20. The system of claim 1, wherein the user interface further comprises a filter feature.

21. -34. (canceled)