Named entity disambiguation for providing TV content enrichment

• US 10,652,592 B2
• Filed: 03/25/2018
• Issued: 05/12/2020
• Est. Priority Date: 07/02/2017
• Status: Active Grant

• Current Assignee: Comigo Ltd.
• Original Assignee: Comigo Ltd.
• Inventors: Geva, Guy, Lasser, Menahem
• Primary Examiner(s): Mendoza, Junior O
• Application Number:
  US15/935,000
• Publication Number:
  US 20190007711A1
• Time in Patent Office: 779 Days
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Methods and systems are disclosed for enriching a viewing experience of a user watching video content on a screen of a client terminal by increasing the relevance of additional media content proposed or provided to the user. Disambiguation of named entities detected in a video content item being played is performed by identifying and accessing an information source directly associated with the video content item, and/or by analyzing visual content of a segment of the video content item. Selecting, proposing and/or providing an additional media content item is based on the information source and/or on the analyzing.

Claims

1. A method for enriching a viewing experience of a user watching video content on a screen of a client terminal by increasing the relevance of additional media content proposed to the user, the method comprising:

a. providing at least a portion of a video content item to the client terminal, thereby causing playing the at least a portion of the video content item on the screen of the client terminal;

b. obtaining a segment of text that is spoken in a sound track of the video content item;

c. identifying an occurrence of an ambiguous reference to a named entity in the segment of text, the ambiguous reference matching multiple candidate named entities;

d. disambiguating the ambiguous reference, the disambiguating comprising;

i. identifying the video content item;

ii. identifying an information source, the information source being directly associated with the identified video content item; and

iii. assigning one candidate named entity of the multiple candidate named entities to the ambiguous reference, the assigning being based on information from the identified information source;

e. selecting one or more media content items that are related to the video content item, the selecting being based on the assigned one candidate named entity; and

f. providing one media content item of the one or more media content items, thereby causing displaying the one media content item on the screen of the client terminal.

2. The method of claim 1, wherein

i. the disambiguating additionally comprises analyzing visual content of a segment of video from the video content item, and

ii. the assigning is also being based on a result of the analyzing.

3. The method of claim 1, wherein:

i. the information from the identified information source comprises at least one of graphics and video,

ii. the disambiguating additionally comprises analyzing visual content of the at least one of graphics and video, and

iii. the information from the identified information source on which the assigning is based includes the analyzed visual content.

4. The method of claim 1, wherein the obtaining of the segment of text includes obtaining the segment of text from subtitles of the video content item.

5. The method of claim 1, wherein the obtaining of the segment of text includes converting speech appearing in the sound track of the video content item into text using a speech-to-text conversion engine.

6. The method of claim 1, wherein the disambiguating of the named entity is done in parallel to the playing of the at least a portion of the video content item.

7. The method of claim 1, wherein the information source directly associated with the identified video content item is a synopsis of the identified video content item.

8. The method of claim 1, wherein the information source directly associated with the identified video content item is an advertisement for the identified video content item.

9. The method of claim 1, wherein the information source directly associated with the identified video content item is a review of the identified video content item.

10. The method of claim 1, wherein the information source directly associated with the identified video content item is an entry in a source that is one of a group consisting of a database, a knowledge base, a dictionary and an encyclopedia, the entry corresponding to the identified video content item.

11. The method of claim 1, wherein the information source directly associated with the identified video content item is an account in a social network, the account being dedicated to the identified video content item.

12. A method for enriching a viewing experience of a user watching video content on a screen of a client terminal by increasing the relevance of additional media content proposed to the user, the method comprising:

a. providing at least a portion of a video content item to the client terminal, thereby causing playing the at least a portion of the video content item on the screen of the client terminal;

b. obtaining a segment of text that is spoken in a sound track of the video content item;

c. identifying an occurrence of an ambiguous reference to a named entity in the segment of text, the ambiguous reference matching multiple candidate named entities;

d. disambiguating the ambiguous reference, the disambiguating comprising;

i. analyzing visual content of a segment of video from the video content item; and

ii. assigning one candidate named entity of the multiple candidate named entities to the ambiguous reference to a named entity, the assigning being based on a result of the analyzing;

e. selecting one or more media content items that are related to the video content item, the selecting being based on the assigned one candidate named entity; and

f. providing one media content item of the one or more media content items, thereby causing displaying the one media content item on the screen of the client terminal.

13. The method of claim 12, wherein:

i. the disambiguating additionally comprises;

a. identifying the video content item; and

b. identifying an information source, the information source being directly associated with the identified video content item, and

ii. the assigning is also being based on information from the identified information source.

14. The method of claim 12, wherein the obtaining of the segment of text includes obtaining the segment of text from subtitles of the video content item.

15. The method of claim 12, wherein the obtaining of the segment of text includes converting speech appearing in the sound track of the video content item into text using a speech-to-text conversion engine.

16. The method of claim 12, wherein the segment of video does not include the identified occurrence of the ambiguous reference.

17. The method of claim 12, wherein at least a portion of the segment of text is not included in the segment of video.

18. The method of claim 12, wherein the disambiguating of the ambiguous reference is done in parallel to the playing of the at least a portion of the video content item.

19. A system for enriching a viewing experience of a user watching video content on a screen of a client terminal by increasing the relevance of additional media content proposed to the user, the system comprising:

a. one or more computer processors;

b. non-transitory computer-readable storage medium storing program instructions for execution by the one or more computer processors; and

c. one or more media content servers in data communication with the client terminal and operable to (i) provide at least a portion of a video content item to the client terminal, so as to cause playing the at least a portion of the video content item on the screen of the client terminal, and (ii) provide a media content item, so as to cause displaying the media content item on the screen of the client terminal, wherein the media content item is one of one or more media content items selected by executing, by the one or more computer processors, the program instructions stored in the non-transitory computer-readable storage medium, wherein the stored program instructions comprise;

i. first program instructions for identifying an occurrence of an ambiguous reference to a named entity in a segment of text that is spoken in a sound track of the video content item, the ambiguous reference matching multiple candidate named entities,

ii. second program instructions for disambiguating the ambiguous reference, the disambiguating comprising (A) identifying the video content item, (B) identifying an information source that is directly associated with the identified video content item, and (C) assigning one candidate named entity of the multiple candidate named entities to the ambiguous reference, the assigning being done based on information obtained from the identified information source, and

iii. third program instructions for selecting the one or more media content items, the one or more media content items being related to the video content item, the selecting being based on the assigned one candidate named entity.

20. A system for enriching a viewing experience of a user watching video content on a screen of a client terminal by increasing the relevance of additional media content proposed to the user, the system comprising:

a. one or more computer processors;

b. non-transitory computer-readable storage medium storing program instructions for execution by the one or more computer processors; and

c. one or more media content servers in data communication with the client terminal and operable to (i) provide at least a portion of a video content item to the client terminal, so as to cause playing the at least a portion of the video content item on the screen of the client terminal, and (ii) provide a media content item, so as to cause displaying the media content item on the screen of the client terminal, wherein the media content item is one of one or more media content items selected by executing, by the one or more computer processors, the program instructions stored in the non-transitory computer-readable storage medium,

wherein the stored program instructions comprise;

i. first program instructions for identifying an occurrence of an ambiguous reference to a named entity in a segment of text that is spoken in a sound track of the video content item, the ambiguous reference matching multiple candidate named entities,

ii. second program instructions for disambiguating the ambiguous reference, the disambiguating comprising (A) analyzing visual content of a segment of video from the video content item, and (B) assigning one candidate named entity of the multiple candidate named entities to the ambiguous reference to a named entity, the assigning being done based on a result of the analyzing, and

iii. third program instructions for selecting the one or more media content items, the one or more media content items being related to the video content item, the selecting being based on the assigned one candidate named entity.