Highly meaningful multimedia metadata creation and associations

• US 7,921,116 B2
• Filed: 06/16/2006
• Issued: 04/05/2011
• Est. Priority Date: 06/16/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Finch, Gregory D., Finkelstein, Charles A.
• Primary Examiner(s): Alam; Hosain T
• Assistant Examiner(s): Oberly; Van H
• Application Number:
  US11/424,689
• Publication Number:
  US 20070294295A1
• Time in Patent Office: 1,754 Days
• Field of Search: 707/736, 707/746
• US Class Current: 707/746
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/70 of video data

Abstract

Systems and methods for generating temporal metadata for a media source as a function of detected content. An analysis engine detects content within a media stream of a media source. A metadata creation engine generates temporal metadata as a function of the detected content. The generated temporal metadata may be used to create a media log for the media source, generate metadata subsequent detected content and to compare to media sources.

Claims

1. A system for creating temporal metadata from a media source comprising:

an import engine for obtaining a media stream from the media source, said media stream including content recorded by the media source, said import engine generating a media sequence from the media stream;

a user interface for receiving user-identified content corresponding to the generated media sequence wherein the user provides metadata identifying the content of the generated media sequence, said user-identified content comprising content discernable to the user during rendering of the media sequence;

an analysis engine for automatically analyzing the user-identified content within the media sequence without user interaction, said analysis engine comprising the following components;

music beat detection, audio dynamic peaks detection, speech detection, word boundary detection, word and phrase detection and audio fingerprint detection, wherein said analysis engine components analyze the audio content of the user-identified content within the media sequence, said word and phrase detection component generating text metadata corresponding to words and phrases detected in the analyzed user-identified content;

a temporal metadata creation engine for automatically creating temporal metadata for the media sequence without user interaction as a function of the user-provided metadata for the user-identified content and the analyzed user-identified content, said creation engine automatically determining a location within the media stream suitable for inserting an advertisement into the media sequence without user interaction as a function of the created temporal metadata; and

a metadata storage component for storing the temporal metadata and the generated text metadata for the media source in a searchable database, said searchable database including information indicating the determined location in the media file suitable for inserting an advertisement.

2. The system of claim 1, further comprising a user interface component for modifying a threshold setting of the analysis engine, modifying the created temporal metadata, and searching the stored temporal metadata.

3. The system of claim 1, wherein the media stream comprises at least one of the following types:

an audio stream, a video stream and a still video stream.

4. The system of claim 1, wherein the media sequence comprises at least one of the following types:

an audio sequence, a video sequence and a still video sequence.

5. The system of claim 1, wherein the media source comprises at least one of the following:

a motion video without sound, a motion video with sound, a sound recording and a digital photograph.

6. The system of claim 1, wherein the media source is generated by at least one of the following:

a digital camera, a cellular phone, a sound recorder, a video camera, a scanner, a computer, a digital video recording device and a digital audio player.

7. The system of claim 1, wherein the analysis engine further comprises at least one of the following components:

dynamic image peaks, dynamic image change detection, and image pattern recognition.

8. The system of claim 1, wherein one of the components of the analysis engine is bypassed as a function of the type of media sequence.

9. The system of claim 1, wherein the metadata comprises at least one of the following:

alphanumeric text description of the detected content, motion video thumbnails of the detected content, audio thumbnail of the detected content, and still thumbnail of the detected content.

10. The system of claim 1, wherein the automatically created temporal metadata corresponds to at least one of closed captioning and subtitles.

11. The system of claim 1, wherein the searchable database includes information indicating a location of adult content.

12. The system of claim 1, wherein the temporal metadata creation engine aggregates the metadata of the detected content for the media sequence and generates a log for the media source as a function of the aggregated metadata.

13. The system of claim 1, further comprising a metadata tagging component for creating metadata compatible with at least of the following:

online video repositories and media asset applications.

14. A method of comparing a first media stream and second media stream comprising:

providing a first media stream comprising a first content discernable during rendering of the first media stream;

providing a second media stream comprising a second content discernable during rendering of the second media stream, wherein the second content is different than the first content;

analyzing the first media stream for the first content within the first media stream;

determining a first characteristic of the analyzed first content;

generating a first set temporal metadata based on the analyzed first content;

analyzing the second media stream for the second content within the second media stream;

determining a second characteristic of the analyzed second media stream;

generating a second set temporal metadata based on the analyzed second content;

comparing the first media stream to the second media stream as of function of the first characteristic of the first media stream and the second characteristic of the second media stream;

providing an evaluation to detect copyright infringement of the analyzed first content of the first media stream by the analyzed second content of the second media stream as a function of the comparison, the first set of temporal metadata and the second set of temporal metadata, said evaluation to detect copyright infringement including one or more of the following;

beat detection within the analyzed first content and the analyzed second content, voice recognition within the analyzed first content and the analyzed second content, and an audio fingerprint comparison of the analyzed first content and the analyzed second content, said provided evaluation comprising information indicating a similarity indicative of copyright of the first content by the second content; and

including in the second set temporal metadata information indicating the appropriate location in the second stream to insert the advertisement; and

inserting an advertisement into the second media stream according to the included temporal metadata.

15. The method of claim 14, wherein at least one of the first characteristic and the second characteristic indicates at least one of the following:

a viewer rating, and an adult content.