Television key phrase detection

• US 10,326,829 B2
• Filed: 12/22/2017
• Issued: 06/18/2019
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: X Corp. (f/k/a Twitter, Inc.) (X Holdings Corp.)
• Original Assignee: X Corp. (f/k/a Twitter, Inc.) (X Holdings Corp.)
• Inventors: Carroll, Andrew J., Whitcomb, Jr., Richard Douglas, Fu, Wangfan, Jianrungsang, Thanabordi
• Primary Examiner(s): Meky, Moustafa M
• Assistant Examiner(s): Kassa, Elizabeth
• Application Number:
  US15/853,384
• Publication Number:
  US 20180124162A1
• Time in Patent Office: 543 Days
• Field of Search: 709203, 709217
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Images of key phrases or hashtags appear on televised feeds. Image processing techniques, such as feature locating algorithms or character recognition algorithms, can be used to locate the images of key phrases in the images. Then, character recognition algorithms can be used to generate a list of candidate key phrases for the key phrase in image format. However, identification of the key phrase in image format is not completely accurate with conventional methods. Social media content items associated with the televised feed are used to filter the list of candidate key phrases. Using known information about the televised feed as well as about key phrases in text format in the social media content items, candidate key phrases in the list of candidate key phrases can be scored and, thus, a final candidate key phrase selected based on the scores.

Claims

1. A computer-executed method for character recognition of an unidentified key phrase presented in a time-based media event, based on a set of social media content items associated with the time-based media event, the method comprising:

accessing a time-based media event including an image comprising an unidentified key phrase in image format, the time-based media event comprises an airing time stamp;

generating a list of candidate key phrases from the image including the unidentified key phrase;

determining a likelihood score for each of the candidate key phrases based on

a confidence score reflecting the likelihood that the unidentified key phrase is correctly interpreted as the candidate key phrase by a recognition process, and

a frequency score reflecting a number of content items in a set of social media content items that each (i) include the candidate key phrase and (ii) have a respective item time stamp that is within an interval amount of time from the airing time stamp of the media event;

selecting one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores;

selecting a content item to present to one or more users of a social media network based on the selected candidate key phrase; and

sending the selected content item to the one or more users of the social media network.

2. The computer-executed method of claim 1, wherein selecting one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores comprises:

selecting a candidate key phrase with a highest likelihood score.

3. The computer executed method of claim 1, wherein the unidentified key phrase includes a plurality of characters and the confidence score for each candidate key phrase is generated from a set of character confidence scores with each character confidence score reflecting the likelihood that a corresponding character of the plurality of characters in the unidentified key phrase is correctly interpreted in the candidate key phrase.

4. The computer-executed method of claim 3, wherein generating the list of candidate key phrases further comprises:

generating a set of likely characters for each character of the plurality of characters of the unidentified key phrase, each set of likely characters including characters with character confidence scores above a threshold; and

generating the list of candidate key phrases using the set of likely characters for each character of the plurality of characters, each candidate key phrase made up of a selection of one character from each set of likely characters.

5. The computer-executed method of claim 1, wherein the set of social media content items contains only content items that have a broadcast time stamp during or after an airing of the time-based media event.

6. The computer-executed method of claim 1, wherein the set of social media content items are associated with the accessed time-based media event based on an ontological concept shared between the content items and the time-based media event.

7. The computer-executed method of claim 1, wherein the image includes a second key phrase associated with a second time based media event and the selected content item is associated with the second time based media event.

8. The computer-executed method of claim 7, wherein the second key phrase appears in a threshold number of social media content items that include the selected candidate key phrase.

9. The computer-executed method of claim 8, wherein the one or more users of the social media network have authored a content item associated with the second key phrase.

10. The computer-executed method of claim 1, wherein the one or more users of the social network have authored a content item associated with the candidate key phrase.

11. The computer-executed method of claim 1, wherein the selected content item includes any of posts, videos, photos, links, status updates, blog entries, and tweets.

12. The method of claim 1, wherein each key candidate phrase in the list of candidate key phrases is generated by recognizing individual characters of the unidentified key phrase and associating each of the recognized characters with a character confidence score that indicates a likelihood that a respective character of the unidentified key phrase is recognized correctly.

13. The method of claim 12, wherein the list of candidate key phrases is generated by selecting the recognized characters that have character confidence scores exceeding a threshold value.

14. The method of claim 12, wherein the likelihood scores for the candidate key phrases are determined in response to identifying a predefined character among the recognized characters of the unidentified key phrase.

15. A non-transitory computer readable storage medium comprising computer program instructions that, when executed by a processor, cause the processor to perform operations for character recognition of an unidentified key phrase in a time-based media event based on a set of social media content items associated with the time-based media event, the operations comprising:

access a time-based media event including an image comprising an unidentified key phrase in image format, the time-based media event comprising an airing time stamp;

generate a list of candidate key phrases from the image including the unidentified key phrase;

determine a likelihood score for each of the candidate key phrases based on

a confidence score reflecting the likelihood that the unidentified key phrase is correctly interpreted as the candidate key phrase by a recognition process, and

a frequency score reflecting a number of content items in a set of social media content items that each (i) include the candidate key phrase, and (ii) have a respective item time stamp that is within an interval amount of time from the airing time stamp of the media event;

select one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores;

select a content item to present to one or more users of a social media network based on the selected candidate key phrase; and

send the selected content item to the one or more users of the social media network.

16. The non-transitory computer readable storage medium of claim 15, wherein selecting one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores further causes the processor to:

select a candidate key phrase with a highest likelihood score.

17. The non-transitory computer readable storage medium of claim 15, wherein the unidentified key phrase includes a plurality of characters and the confidence score for each candidate key phrase is generated from a set of character confidence scores with each character confidence score reflecting the likelihood that a corresponding character of the plurality of characters in the unidentified key phrase is correctly interpreted in the candidate key phrase.

18. The non-transitory computer readable storage medium of claim 17, wherein generating the list of candidate key phrases further causes the processor to:

generate a set of likely characters for each character of the plurality of characters of the unidentified key phrase, each set of likely characters including characters with character confidence scores above a threshold; and

generate the list of candidate key phrases using the set of likely characters for each character of the plurality of characters, each candidate key phrase made up of a selection of one character from each set of likely characters.

19. The non-transitory computer readable storage medium of claim 15, wherein the set of social media content items contains only content items that have a broadcast time stamp during or after an airing of the time-based media event.

20. The non-transitory computer readable storage medium of claim 15, wherein set of social media content items are associated with the accessed time-based media event based on an ontological concept shared between the content items and the time-based media event.

21. The non-transitory computer readable storage medium of claim 15, wherein the image includes a second key phrase associated with a second time based media event and the selected content item is associated with the second time based media event.

22. The non-transitory computer readable storage medium of claim 21, wherein the second key phrase appears in a threshold number of social media content items that include the selected candidate key phrase.

23. The non-transitory computer readable storage medium of claim 22, wherein the one or more users of the social media network have authored a content item associated with the second key phrase.

24. The non-transitory computer readable storage medium of claim 23, wherein the one or more users of the social network have authored a content item associated with the candidate key phrase.

25. A system comprising a processor and a memory storing computer program instructions that when executed by the processor cause the processor to perform operations for character recognition of an unidentified key phrase presented in a time-based media event, based on a set of social media content items associated with the time-based media event, the operations comprising:

access a time-based media event including an image comprising an unidentified key phrase in image format, the time-based media event comprising an airing time stamp;

generate a list of candidate key phrases from the image including the unidentified key phrase;

determine a likelihood score for each of the candidate key phrases based on

a confidence score reflecting the likelihood that the unidentified key phrase is correctly interpreted as the candidate key phrase by a recognition process, and

a frequency score reflecting a number of content items in a set of social media content items that each (i) include the candidate key phrase and (ii) have a respective item time stamp that is within an interval amount of time from the airing time stamp of the media event;

select one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores;

select a content item to present to one or more users of a social media network based on the selected candidate key phrase; and

send the selected content item to the one or more users of the social media network.

26. The system of claim 25, wherein the set of social media content items are associated with the accessed time-based media event based on an ontological concept shared between the content items and the time-based media event.

27. The system of claim 25, wherein the image includes a second key phrase associated with a second time based media event and the selected content item is associated with the second time based media event.

28. The system of claim 27, wherein the second key phrase appears in a threshold number of social media content items that include the selected candidate key phrase.

29. The system of claim 28, wherein the one or more users of the social media network have authored a content item associated with the second key phrase.

30. The system of claim 25, wherein the set of social media content items contains only content items that have a broadcast time stamp during or after an airing of the time-based media event.

31. The system of claim 30, wherein selecting one of the candidate key phrases as being the unidentified key phrase based on the likelihood scores further causes the processor to:

select a candidate key phrase with a highest likelihood score.

32. The system of claim 30, wherein the unidentified key phrase includes a plurality of characters and the confidence score for each candidate key phrase is generated from a set of character confidence scores with each character confidence score reflecting the likelihood that a corresponding character of the plurality of characters in the unidentified key phrase is correctly interpreted in the candidate key phrase.

33. The system of claim 32, wherein generating the list of candidate key phrases further causes the processor to:

generate a set of likely characters for each character of the plurality of characters of the unidentified key phrase, each set of likely characters including characters with character confidence scores above a threshold; and

generate the list of candidate key phrases using the set of likely characters for each character of the plurality of characters, each candidate key phrase made up of a selection of one character from each set of likely characters.