System and method for indexing and annotation of video content

• US 9,407,942 B2
• Filed: 11/07/2012
• Issued: 08/02/2016
• Est. Priority Date: 10/03/2008
• Status: Expired due to Fees

• Current Assignee: Finitiv Corporation
• Original Assignee: Finitiv Corporation
• Inventors: Brenneman, Andrew
• Primary Examiner(s): Bycer, Eric J
• Application Number:
  US13/671,513
• Publication Number:
  US 20130067333A1
• Time in Patent Office: 1,364 Days
• Field of Search: 715230-233, 715719-726, 37524001-24029, 382173-180, 348461-468, 725109-113, 386/240, 386/241, 386/343, 386/344
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/105 Arrangements for software license management or administration, e.g. for managing licenses at corporate level
• CPC Class Codes: G06F 21/12 Protecting executable software
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: G11B 27/327 Table of contents
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 1/00214 details of transmission
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Embodiments in accordance with the present invention relate to methods and systems for the indexing and annotation of video content, the retrieval and display of such video content and its annotations, and the conversion and transmission of such video content and associated data. In certain embodiments, the system processes and cross-references video content with data associated with that content and with standard ontologies. Correlating these information sources produces a system of indexing and annotation of the content. In certain embodiments, this indexing and annotation may facilitate the discovery, retrieval, and management of video content, and may facilitate the integration of video content with other media and other systems.

Claims

1. A method comprising:

receiving digital media data including codes as a first input;

defining segments of the digital media data;

generating a first index of the segments;

receiving associated data relating to the digital media data as a second input;

aligning elements of the associated data to relevant codes of the digital media data to generate a second index of the digital media data including a set of annotations of the digital media data associated with the codes to which they apply;

combining the first index, the second index including the set of annotations, and the codes to which they are associated, to produce an annotated index, wherein the annotated index includes a term of a permissions license granted by a third-party rights holder to a publishing organization for an element of the digital media data originating from the third-party rights holder and incorporated during a product development process; and

storing the annotated index in a computer-readable storage medium for access by the publishing organization in executing the product development process.

2. The method of claim 1 wherein the defining of segments is further based upon an optical text transcript of the digital media data, and the second index including the annotations.

3. The method of claim 2 wherein the digital media data comprises a digital media container file.

4. The method of claim 3 wherein the digital media data is configured to represent graphical and/or text content.

5. The method of claim 1 wherein the aligning of elements of the associated data is based on a comparison of the associated data with a speech transcript of the digital media data.

6. The method of claim 1 wherein the aligning of elements of the associated data is based on a comparison of the associated data with an optical text transcript of the digital media data.

7. The method of claim 1 wherein the aligning of elements of the associated data is based on a comparison of the associated data with data embedded in a container file of the digital media data.

8. The method of claim 1 further comprising tagging the second index according to a standard set of terms to create a second set of annotations, wherein the combining includes the second set of annotations of the second index.

9. The method of claim 1 further comprising:

receiving from an operator an instruction identifying a portion of the digital media data and an intended recipient for the portion;

processing the annotated index according the instruction to produce a portion of the annotated index corresponding to the identified portion of the digital media data; and

transmitting the portion of the digital media data and the portion of the annotated index to the recipient.

10. The method of claim 9 wherein the portion of the digital media data and the portion of the annotated index are transmitted to the recipient according to a delivery format previously specified for the recipient.

11. The method of claim 10 further comprising:

storing on a content server, the annotated index, the digital media data, and the associated materials;

receiving from an operator a fourth input specifying a desired delivery format; and

converting data of the annotated index, the digital media data, and/or the associated materials, to the delivery format.

12. The method of claim 11 further comprising:

identifying misalignment between data embedded in the digital media container file and data in the associated material; and

revising the associated material through comparison of data in the associated material and embedded data extracted from the digital media container file.

13. The method of claim 1 further comprising, prior to the aligning, cross-referencing components of the associated data to establish relationships between portions of the associated data, wherein the relationships are included in the second index.

14. The method of claim 1 wherein the code comprises time codes.

15. The method of claim 14 wherein the digital media data includes audio data.

16. The method of claim 1 wherein the annotated index includes a set of textual data derived from an optical text transcript of the digital media data, textual data embedded in the digital media container file, and textual data from associated data relating to the digital media.

17. The method of claim 1 wherein the digital media data comprises interactive content and the associated data relates to the interactive content.

18. The method of claim 17 wherein the codes describe points in a simulated three dimensional space.

19. The method of claim 1 wherein the annotated index includes a GPS coordinate data for geographic locations referenced by or associated with the digital media data.

20. The method of claim 19 further comprising:

causing a GPS content matching system to interoperate with the annotated index and a mobile device;

causing the GPS content matching system to request and receive from the mobile device, current GPS coordinates corresponding to a location of the mobile device;

causing the GPS content matching system to compare the current GPS coordinates to the GPS coordinate data; and

causing the GPS content matching system to select a clip when the GPS coordinate data are proximate to the current GPS coordinates.

21. The method of claim 1 further comprising:

causing a content matching system to interoperate with the annotated index and a social media application;

causing the content matching system to extract data from the social media application;

causing the content matching system to compare the extracted data to data elements of the annotated index; and

causing the content matching system to select a clip when data elements in the annotated index correspond to the extracted data.