HIERARCHICAL VIDEO CONCEPT TAGGING AND INDEXING SYSTEM FOR LEARNING CONTENT ORCHESTRATION

• US 20180063601A1
• Filed: 08/26/2016
• Published: 03/01/2018
• Est. Priority Date: 08/26/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chee, Yi-Min, Jagmohan, Ashish, Kokku, Ravindranath, Liu, Rong, Nitta, Satyanarayana V.
• Granted Patent: US 10,567,850 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/278 Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments

Abstract

A computer system, method, and computer-readable product for automatically generating a hierarchy of concepts described in a video using semantic concept matching that assists persons in searching for specific types of content within videos. A video segment is obtained, with the segment having a distinct running time with discrete information related to one or more topics within its content. The discrete information of the video segment is then parsed out and related to a predetermined relevant topic, and then one or more attributes for weighting each discrete information relevant topic within the video segment are determined. A hierarchy of concepts of the video segments is then created, based upon, at least, the weighting of the information topic within the running time of the video segment based upon the presence of the one or more attributes. Other information related to the video segment can also be used to modify the hierarchy.

Claims

1. A computer implemented method of automatically generating a hierarchy of concepts described in a video using semantic concept matching, comprising:

obtaining a video segment having a distinct running time including discrete information related to one or more topics;

parsing the discrete information of the video segment;

relating the discrete information to a predetermined relevant topic;

determining one or more attributes for weighting each of the discrete information related to the predetermined relevant topic within the video segment; and

creating a hierarchy of concepts of the video segment based upon, at least, the weighting of each of the discrete information related to the predetermined relevant topic within the video segment based upon the presence of the one or more attributes, wherein the discrete information is based on at least one of (i) spoken language in the video that is converted to text by the computer, and (ii) visual content in the video that is recognized by the computer.

2. The method of claim 1, further comprising:

obtaining topic tags created for the video segment; and

integrating the topic tags within the hierarchy of concepts.

3. The method of claim 1, wherein the video segment further includes related data of user comments related to the video segment, the user comments including discrete information, and further comprising:

parsing the discrete information from the user comments;

relating the discrete information to a predetermined relevant topic; and

modifying the hierarchy of concepts of the video segment based upon the discrete information of the relevant topic of the user comments.

4. The method of claim 1, wherein the discrete information of the video segment includes natural language, and further comprising parsing the natural language from the video segment, wherein the discrete information is created by analyzing the parsed natural language.

5. The method of claim 1, wherein the discrete information of the video segment includes visual content, and further comprising parsing the visual content from the video segment, wherein the discrete information is created by analyzing the parsed visual content.

6. The method of claim 1, further comprising:

creating a weighted graph wherein each concept of the hierarchy of concepts is a weighted node within the graph; and

storing the weighted graph for the video segment.

7. The method of claim 6 wherein the one or more attributes for weighting each discrete information relevant topic are one or more of:

the duration of the relevant topic within the video segment, the breadth of the relevant topic, the relatedness of the relevant topic to other relevant topics within the video segment, and the developmental pattern of the relevant topic within the video segment.

8. A computer system for automatically generating a hierarchy of concepts described in a video using semantic concept matching, the computer system comprising:

a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on at least one or more storage devices for execution by at least one or more processors via at least one or more memories, wherein the computer system is configured to;

obtain a video segment having a distinct running time including discrete information related to one or more topics;

parse the discrete information of the video segment;

relate the discrete information to a predetermined relevant topic;

determine one or more attributes for weighting each of the discrete information related to the predetermined relevant topic within the video segment; and

create a hierarchy of concepts of the video segment based upon, at least, the weighting of each of the discrete information related to the predetermined relevant topic within the running time of the video segment based upon the presence of the one or more attributes, wherein the discrete information is based on at least one of (i) spoken language in the video that is converted to text by the computer system, and (ii) visual content in the video that is recognized by the computer system.

9. The system of claim 8, wherein the computer system is further configured to:

obtain topic tags created for the video segment; and

integrate the topic tags within the hierarchy of concepts.

10. The system of claim 8, wherein the video segment further includes related data of user comments related to the video segment, the user comments including discrete information, and the computer system is further configured to:

parse the discrete information from the user comments;

relate the discrete information to a predetermined relevant topic; and

modify the hierarchy of concepts of the video segment based upon the discrete information of the relevant topic of the user comments.

11. The system of claim 8, wherein the discrete information of the video segment includes natural language, and the computer system is further configured to parse the natural language from the video segment, wherein the discrete information is created by analyzing the parsed natural language.

12. The system of claim 8, wherein the discrete information of the video segment includes visual content, and the computer system is further configured to parse the visual content from the video segment, wherein

the discrete information is created by analyzing the parsed visual content.

13. The system of claim 8, further configured to:

create a weighted graph, wherein each concept of the hierarchy of concepts is a weighted node within the graph; and

store the weighted graph for the video segment.

14. The system of claim 13, further configured to determine, as one or more attributes, one or more of:

the duration of the relevant topic within the video segment, the breadth of the relevant topic, the relatedness of the relevant topic to other relevant topics within the video segment, and the developmental pattern of the relevant topic within the video segment.

15. A computer program product stored on a non-transitory computer readable medium having computer readable program code embodied thereon that is executable by a computer for automatically generating a hierarchy of concepts described in a video using semantic concept matching, the computer program product comprising computer readable program code that when executed configures a computer to perform the acts of:

obtaining a video segment having a distinct running time including discrete information related to one or more topics;

parsing the discrete information of the video segment;

relating the discrete information to a predetermined relevant topic;

determining one or more attributes for weighting each of the discrete information related to the predetermined relevant topic within the video segment; and

creating a hierarchy of concepts of the video segment based upon, at least, the weight of each of the discrete information related to the predetermined relevant topic within the running time of the video segment based upon the presence of the one or more attributes, wherein the discrete information is based on at least one of (i) spoken language in the video that is converted to text by the computer, and (ii) visual content in the video that is recognized by the computer.

16. The computer program product of claim 15, wherein execution of the code further configures the computer to perform acts comprising:

obtaining topic tags created for the video segment; and

integrating the topic tags within the hierarchy of concepts.

17. The computer program product of claim 15, wherein the video segment further includes related data of user comments related to the video segment, the user comments including discrete information, and wherein execution of the code further configures the computer to perform acts comprising:

parsing the discrete information from the user comments;

relating the discrete information to a predetermined relevant topic; and

modifying the hierarchy of concepts of the video segment based upon the discrete information of the relevant topic of the user comments.

18. The computer program product of claim 15, wherein the discrete information of the video segment includes natural language, and wherein execution of the code further configures the computer to perform acts comprising parsing the natural language from the video segment, wherein the discrete information is created by analyzing the parsed natural language.

19. The computer program product of claim 15, wherein the discrete information of the video segment includes visual content, and wherein execution of the code further configures the computer to perform acts comprising parsing the visual content from the video segment, wherein the discrete information is created by analyzing the parsed visual content.

20. The computer program product of claim 15, further comprising computer readable program code for:

creating a weighted graph wherein each concept is a weighted node within the graph; and

storing the weighted graph for the video segment.