KEYWORD AUTOMATION OF VIDEO CONTENT

• US 20110218994A1
• Filed: 03/05/2010
• Published: 09/08/2011
• Est. Priority Date: 03/05/2010
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Holladay, Christopher E., Shaouy, William P.
• Application Number:
  US12/717,988
• Publication Number:
  US 20110218994A1
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data

Abstract

A system and associated method for automatically processing keyword for video content. The video content contains image frames and an audio stream. An image pattern table for image patterns from the image frames and a word pattern table for word patterns from the audio stream are generated by use of respective pattern names provided by pattern recognition tools. Each pattern is associated with a respective count indicating a number of appearances of each pattern. A respective weight of each pattern is calculated as a relative frequency of each pattern. The image pattern table and the word pattern table are merged to generate a keyword list. A predefined number of most frequently appeared patterns are selected by examining the respective weight of each pattern and metadata associated with the video content are updated to utilize pattern names of the selected patterns as keyword for web searches.

Claims

1. A method for automatically processing keyword for video content, said method comprising:

a processor of a computer system loading said video content, said video content comprising at least one image frame and an audio stream;

said processor generating an image pattern table from said at least one image frame, wherein an entry of the image pattern table comprises attributes of image pattern identifier ID_I, image pattern name, image pattern count COUNT(ID_I), and image pattern weight WEIGHT(ID_I), wherein the image pattern identifier ID_I identifies an image pattern in said at least one image frame, wherein the image pattern name is an alphanumeric text representing the image pattern, wherein the image pattern count COUNT(ID_I) represents a number of appearances of the image pattern in said at least one image frame, and wherein the image pattern weight WEIGHT(ID_I) represents a relative frequency of the image pattern within said at least one image frame;

said processor generating a word pattern table from the audio stream, wherein an entry of the word pattern table comprises attributes of word pattern identifier ID_W, word pattern name, word pattern count COUNT(ID_W), and word pattern weight WEIGHT(ID_W), wherein the word pattern identifier ID_W identifies a word pattern in the audio stream, wherein the word pattern name is an alphanumeric text representing the word pattern, wherein the word pattern count COUNT(ID_W) represents a number of appearances of the word pattern in the audio stream, and wherein the word pattern weight WEIGHT(ID_W) represents a relative frequency of the word pattern within the audio stream;

said processor calculating the respective weight for all entries in the image pattern table and the word pattern table, wherein the respective weight is selected from the group consisting of the image pattern weight WEIGHT(ID_I) and the word pattern weight WEIGHT(ID_W);

said processor generating a keyword list from the image pattern table and the word pattern table based on the calculated weight, wherein an entry of the keyword list is selected from the group consisting of entries of the image pattern table and entries of the word pattern table, and wherein the entry of the keyword list comprises attributes of generic pattern identifier, generic pattern name, generic pattern count, and generic pattern weight; and

said processor integrating the generated keyword list into metadata of a web page associated with the video content such that the keyword list is utilized in web searches employing the metadata.

2. The method of claim 1, said generating the image pattern table comprising:

generating the image pattern identifier ID_I that uniquely identifies each image frame of the video content; and

assigning the image pattern name that has been provided by an image recognition tool as a result of analyzing each image frame of said at least one image frame of the video content, wherein said image recognition tool logically groups similar image patterns with an identical image pattern name.

3. The method of claim 1, said generating the word pattern table comprising:

generating the word pattern identifier ID_W that uniquely identifies each word pattern in the audio stream of the video content; and

assigning the word pattern name that has been provided by a speech recognition tool as a result of analyzing each word pattern of said audio stream, wherein said speech recognition tool logically groups similar word patterns with an identical word pattern name.

4. The method of claim 1, said calculating the respective weight comprising:

calculating the image pattern weight WEIGHT(ID_I) for each entry in the image pattern table via WEIGHT(ID_I)=COUNT(ID_I)/SUM_I, wherein SUM_I is the sum of all image pattern counts in the image pattern table; and

calculating the word pattern weight WEIGHT(ID_W) for each entry in the word pattern table via WEIGHT(ID_W)=COUNT(ID_W)/SUM_W, wherein SUM_W is the sum of all word pattern counts in the word pattern table.

5. The method of claim 4, said generating the keyword list comprising:

joining the image pattern table and the word pattern table into the keyword list by mapping, for each entry of the image pattern table, the image pattern identifier, the image pattern name, the image pattern count, and the image pattern weight attributes of said each entry of the image pattern table to the generic pattern identifier, the generic pattern name, the generic pattern count, and the generic pattern weight attributes of a corresponding entry of the keyword list, respectively, and by mapping, for each entry of the word pattern table, the word pattern identifier, the word pattern name, the word pattern count, and the word pattern weight attributes of said each entry of the word pattern table to the generic pattern identifier, the generic pattern name, the generic pattern count, and the generic pattern weight attributes of another corresponding entry of the keyword list, respectively;

selecting K number of entries of the keyword list that have largest values of the generic pattern weight, wherein K is a positive integer; and

storing generic pattern names of selected K number of entries as the keyword list to a computer readable storage medium coupled to said processor.

6. A computer program product comprising:

a computer readable storage medium having a computer readable program code embodied therein, said computer readable program code containing instructions that perform a method for automatically processing keyword for video content, said method comprising;

loading said video content, said video content comprising at least one image frame and an audio stream;

generating an image pattern table from said at least one image frame, wherein an entry of the image pattern table comprises attributes of image pattern identifier ID_I, image pattern name, image pattern count COUNT(ID_I), and image pattern weight WEIGHT(ID_I), wherein the image pattern identifier ID_I identifies an image pattern in said at least one image frame, wherein the image pattern name is an alphanumeric text representing the image pattern, wherein the image pattern count COUNT(ID_I) represents a number of appearances of the image pattern in said at least one image frame, and wherein the image pattern weight WEIGHT(ID_I) represents a relative frequency of the image pattern within said at least one image frame;

generating a word pattern table from the audio stream, wherein an entry of the word pattern table comprises attributes of word pattern identifier ID_W, word pattern name, word pattern count COUNT(ID_W), and word pattern weight WEIGHT(ID_W), wherein the word pattern identifier ID_W identifies a word pattern in the audio stream, wherein the word pattern name is an alphanumeric text representing the word pattern, wherein the word pattern count COUNT(ID_W) represents a number of appearances of the word pattern in the audio stream, and wherein the word pattern weight WEIGHT(ID_W) represents a relative frequency of the word pattern within the audio stream;

calculating the respective weight for all entries in the image pattern table and the word pattern table, wherein the respective weight is selected from the group consisting of the image pattern weight WEIGHT(ID_I) and the word pattern weight WEIGHT(ID_W);

generating a keyword list from the image pattern table and the word pattern table based on the calculated weight, wherein an entry of the keyword list is selected from the group consisting of entries of the image pattern table and entries of the word pattern table, and wherein the entry of the keyword list comprises attributes of generic pattern identifier, generic pattern name, generic pattern count, and generic pattern weight; and

integrating the generated keyword list into metadata of a web page associated with the video content such that the keyword list is utilized in web searches employing the metadata.

7. The computer program product of claim 6, said generating the image pattern table comprising:

generating the image pattern identifier ID_I that uniquely identifies each image frame of the video content; and

assigning the image pattern name that has been provided by an image recognition tool as a result of analyzing each image frame of said at least one image frame of the video content, wherein said image recognition tool logically groups similar image patterns with an identical image pattern name.

8. The computer program product of claim 6, said generating the word pattern table comprising:

generating the word pattern identifier ID_W that uniquely identifies each word pattern in the audio stream of the video content; and

assigning the word pattern name that has been provided by a speech recognition tool as a result of analyzing each word pattern of said audio stream, wherein said speech recognition tool logically groups similar word patterns with an identical word pattern name.

9. The computer program product of claim 6, said calculating the respective weight comprising:

calculating the image pattern weight WEIGHT(ID_I) for each entry in the image pattern table via WEIGHT(ID_I)=COUNT(ID_I)/SUM_I, wherein SUM_I is the sum of all image pattern counts in the image pattern table; and

calculating the word pattern weight WEIGHT(ID_W) for each entry in the word pattern table via WEIGHT(ID_W)=COUNT(ID_W)/SUM_W, wherein SUM_W is the sum of all word pattern counts in the word pattern table.

10. The computer program product of claim 9, said generating the keyword list comprising:

merging entries of the image pattern table and the word pattern table;

sorting the merged entries in a descending order of the generic pattern weight, wherein the generic pattern weight is equal to the image pattern weight WEIGHT(ID_I) if the merged entry is an entry of the image pattern table, and wherein the generic pattern weight is equal to the word pattern weight WEIGHT(ID_W) if the merged entry is an entry of the word pattern table;

selecting K number of entries from the top of the merged entries such that the selected K entries have K largest values of the generic pattern weight, wherein K is a positive integer, wherein the generic pattern identifier, the generic pattern name, and the generic pattern count is respectively mapped from the image pattern identifier, the image pattern name, and the image pattern count if the selected entry is an entry of the image pattern table, and wherein the generic pattern identifier, the generic pattern name, and the generic pattern count is respectively mapped from the word pattern identifier, the word pattern name, and the word pattern count if the selected entry is an entry of the word pattern table; and

adding generic pattern names of the selected K entries to the keyword list, wherein the keyword list is stored in the computer readable storage medium.

11. A computer system comprising a processor and a computer readable memory unit coupled to the processor, said computer readable memory unit containing instructions that when run by the processor implement a method for automatically processing keyword for video content, said method comprising:

loading said video content, said video content comprising at least one image frame and an audio stream;

generating an image pattern table from said at least one image frame, wherein an entry of the image pattern table comprises attributes of image pattern identifier ID_I, image pattern name, image pattern count COUNT(ID_I), and image pattern weight WEIGHT(ID_I), wherein the image pattern identifier ID_I identifies an image pattern in said at least one image frame, wherein the image pattern name is an alphanumeric text representing the image pattern, wherein the image pattern count COUNT(ID_I) represents a number of appearances of the image pattern in said at least one image frame, and wherein the image pattern weight WEIGHT(ID_I) represents a relative frequency of the image pattern within said at least one image frame;

generating a word pattern table from the audio stream, wherein an entry of the word pattern table comprises attributes of word pattern identifier ID_W, word pattern name, word pattern count COUNT(ID_W), and word pattern weight WEIGHT(ID_W), wherein the word pattern identifier ID_W identifies a word pattern in the audio stream, wherein the word pattern name is an alphanumeric text representing the word pattern, wherein the word pattern count COUNT(ID_W) represents a number of appearances of the word pattern in the audio stream, and wherein the word pattern weight WEIGHT(ID_W) represents a relative frequency of the word pattern within the audio stream;

calculating the respective weight for all entries in the image pattern table and the word pattern table, wherein the respective weight is selected from the group consisting of the image pattern weight WEIGHT(ID_I) and the word pattern weight WEIGHT(ID_W);

generating a keyword list from the image pattern table and the word pattern table based on the calculated weight, wherein an entry of the keyword list is selected from the group consisting of entries of the image pattern table and entries of the word pattern table, and wherein the entry of the keyword list comprises attributes of generic pattern identifier, generic pattern name, generic pattern count, and generic pattern weight; and

integrating the generated keyword list into metadata of a web page associated with the video content such that the keyword list is utilized in web searches employing the metadata.

12. The computer system of claim 11, said generating the image pattern table comprising:

generating the image pattern identifier ID_I that uniquely identifies each image frame of the video content; and

assigning the image pattern name that has been provided by an image recognition tool as a result of analyzing each image frame of said at least one image frame of the video content, wherein said image recognition tool logically groups similar image patterns with an identical image pattern name.

13. The computer system of claim 11, said generating the word pattern table comprising:

generating the word pattern identifier ID_W that uniquely identifies each word pattern in the audio stream of the video content; and

assigning the word pattern name that has been provided by a speech recognition tool as a result of analyzing each word pattern of said audio stream, wherein said speech recognition tool logically groups similar word patterns with an identical word pattern name.

14. The computer system of claim 11, said calculating the respective weight comprising:

calculating the image pattern weight WEIGHT(ID_I) for each entry in the image pattern table via WEIGHT(ID_I)=COUNT(ID_I)/SUM_I, wherein SUM_I is the sum of all image pattern counts in the image pattern table; and

calculating the word pattern weight WEIGHT(ID_W) for each entry in the word pattern table via WEIGHT(ID_W)=COUNT(ID_W)/SUM_W, wherein SUM_W is the sum of all word pattern counts in the word pattern table.

15. The computer system of claim 14, said generating the keyword list comprising:

joining the image pattern table and the word pattern table into the keyword list by mapping, for each entry of the image pattern table, the image pattern identifier, the image pattern name, the image pattern count, and the image pattern weight attributes of said each entry of the image pattern table to the generic pattern identifier, the generic pattern name, the generic pattern count, and the generic pattern weight attributes of a corresponding entry of the keyword list, respectively, and by mapping, for each entry of the word pattern table, the word pattern identifier, the word pattern name, the word pattern count, and the word pattern weight attributes of said each entry of the word pattern table to the generic pattern identifier, the generic pattern name, the generic pattern count, and the generic pattern weight attributes of another corresponding entry of the keyword list, respectively;

selecting K number of entries of the keyword list that have largest values of the generic pattern weight, wherein K is a positive integer; and

storing generic pattern names of selected K number of entries as the keyword list to a computer readable storage medium coupled to said processor.

16. A process for supporting computer infrastructure, said process comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a computing system, wherein the code in combination with the computing system is capable of performing a method for automatically processing keyword for video content, said method comprising:

loading said video content, said video content comprising at least one image frame and an audio stream;

generating an image pattern table from said at least one image frame, wherein an entry of the image pattern table comprises attributes of image pattern identifier ID_I, image pattern name, image pattern count COUNT(ID_I), and image pattern weight WEIGHT(ID_I), wherein the image pattern identifier ID_I identifies an image pattern in said at least one image frame, wherein the image pattern name is an alphanumeric text representing the image pattern, wherein the image pattern count COUNT(ID_I) represents a number of appearances of the image pattern in said at least one image frame, and wherein the image pattern weight WEIGHT(ID_I) represents a relative frequency of the image pattern within said at least one image frame;

generating a word pattern table from the audio stream, wherein an entry of the word pattern table comprises attributes of word pattern identifier ID_W, word pattern name, word pattern count COUNT(ID_W), and word pattern weight WEIGHT(ID_W), wherein the word pattern identifier ID_W identifies a word pattern in the audio stream, wherein the word pattern name is an alphanumeric text representing the word pattern, wherein the word pattern count COUNT(ID_W) represents a number of appearances of the word pattern in the audio stream, and wherein the word pattern weight WEIGHT(ID_W) represents a relative frequency of the word pattern within the audio stream;

calculating the respective weight for all entries in the image pattern table and the word pattern table, wherein the respective weight is selected from the group consisting of the image pattern weight WEIGHT(ID_I) and the word pattern weight WEIGHT(ID_W);

generating a keyword list from the image pattern table and the word pattern table based on the calculated weight, wherein an entry of the keyword list is selected from the group consisting of entries of the image pattern table and entries of the word pattern table, and wherein the entry of the keyword list comprises attributes of generic pattern identifier, generic pattern name, generic pattern count, and generic pattern weight; and

integrating the generated keyword list into metadata of a web page associated with the video content such that the keyword list is utilized in web searches employing the metadata.

17. The process of claim 16, said generating the image pattern table comprising:

generating the image pattern identifier ID_I that uniquely identifies each image frame of the video content; and

assigning the image pattern name that has been provided by an image recognition tool as a result of analyzing each image frame of said at least one image frame of the video content, wherein said image recognition tool logically groups similar image patterns with an identical image pattern name.

18. The process of claim 16, said generating the word pattern table comprising:

generating the word pattern identifier ID_W that uniquely identifies each word pattern in the audio stream of the video content; and

assigning the word pattern name that has been provided by a speech recognition tool as a result of analyzing each word pattern of said audio stream, wherein said speech recognition tool logically groups similar word patterns with an identical word pattern name.

19. The process of claim 16, said calculating the respective weight comprising:

calculating the image pattern weight WEIGHT(ID_I) for each entry in the image pattern table via WEIGHT(ID_I)=COUNT(ID_I)/SUM_I, wherein SUM_I is the sum of all image pattern counts in the image pattern table; and

calculating the word pattern weight WEIGHT(ID_W) for each entry in the word pattern table via WEIGHT(ID_W)=COUNT(ID_W)/SUM_W, wherein SUM_W is the sum of all word pattern counts in the word pattern table.

20. The process of claim 19, said generating the keyword list comprising:

merging entries of the image pattern table and the word pattern table;

sorting the merged entries in a descending order of the generic pattern weight, wherein the generic pattern weight is equal to the image pattern weight WEIGHT(ID_I) if the merged entry is an entry of the image pattern table, and wherein the generic pattern weight is equal to the word pattern weight WEIGHT(ID_W) if the merged entry is an entry of the word pattern table;

selecting K number of entries from the top of the merged entries such that the selected K entries have K largest values of the generic pattern weight, wherein K is a positive integer, wherein the generic pattern identifier, the generic pattern name, and the generic pattern count is respectively mapped from the image pattern identifier, the image pattern name, and the image pattern count if the selected entry is an entry of the image pattern table, and wherein the generic pattern identifier, the generic pattern name, and the generic pattern count is respectively mapped from the word pattern identifier, the word pattern name, and the word pattern count if the selected entry is an entry of the word pattern table; and

adding generic pattern names of the selected K entries to the keyword list, wherein the keyword list is stored in a computer readable storage medium coupled to the computer system.