Identification and presentation of content associated with currently playing television programs

• US 10,701,440 B2
• Filed: 02/14/2019
• Issued: 06/30/2020
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Verrilli, Michael, Sandland, Nathan Leslie
• Primary Examiner(s): Flynn, Nathan J
• Assistant Examiner(s): Trinh, Tung T
• Application Number:
  US16/276,379
• Publication Number:
  US 20190182543A1
• Time in Patent Office: 502 Days
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: H04N 21/44204 Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image

Abstract

Features of various implementations are used to enable identifying and presenting, in real time, Internet-accessible and/or Internet-based functionality, content and/or services associated with a particular television program concurrently playing on a television or display, as a part of an interactive television suite of technologies. In some implementations, the process of identifying Internet-accessible and/or Internet-based functionality, content and/or services associated with a particular television program is enabled to reduce communication between a primary set-top box and a supplemental set-top box and/or television integrated device, thereby reducing the need for one or more provider specific APIs. For example, one aspect of the disclosure is a computer-implemented method of identifying a media program by capturing display data from the media program, and extracting information from the display data in response to determining that the display data includes a program information overlay associated with the media program.

Claims

1. A computer-implemented method, comprising:

at an electronic device that is communicatively coupled to a display device and a second screen client device, and while a media program is being provided to the display device for display thereon;

capturing screen display data associated with the media program displayed on the display device;

determining whether a program information overlay is being displayed on the display device, including analyzing the captured screen display data to determine whether the screen display data includes one or more predefined indicators of the program information overlay;

in accordance with a determination that the program information overlay is being displayed on the display device, processing information displayed on the program information overlay;

generating search terms from the processed information;

performing a search based on a subset of the generated search terms to identify content associated with the media program; and

while the media program is being displayed on the display device, transmitting search results of the search to at least one of the display device and the second screen client device for concurrent and synchronous display thereon.

2. The method of claim 1, wherein the search includes an Internet search selected from a group consisting of a general Internet search, a targeted search for associated news items, a targeted search for associated images, a targeted search for associated Internet-accessible media content, and a targeted search for associated social media content.

3. The method of claim 1, wherein the one or more predefined indicators include at least one of:

a color of the program information overlay, a percentage of display area the program information overlay typically covers when visible, an arrangement of text with the program information overlay, contrast between a portion of a display likely to be displaying the media program and another portion of the display likely to be displaying the program information overlay.

4. The method of claim 1, wherein capturing the screen display data includes performing a screen capture of the display device, and processing the information displayed on the program information overlay comprises applying an optical character recognition process to the captured screen display data.

5. The method of claim 1, wherein a set-top box is coupled to the display device, further comprising:

obtaining screen display data of the program information overlay from one of a display card of the set-top box, an operating system of the set-top box, and a television application running on the set-top box.

6. The method of claim 1, further comprising:

obtaining screen display data of the program information overlay by decoding the media program streamed by the electronic device to the display device.

7. The method of claim 1, further comprising:

comparing the processed information to electronic program guide data to confirm identification of the media program that is played on the display device.

8. The method of claim 1, further comprising:

transmitting a code that causes the display device to receive the program information overlay in the screen display data, and

capturing the screen display data associated with the media program in response to transmitting the code that causes the display device to receive the program information overlay in the screen display data.

9. An electronic device, wherein the electronic device is communicatively coupled to a display device and a second screen client device, comprising:

one or more processors; and

memory storing one or more programs, the one or more programs including instructions that, when executed by the one or more processors, cause the electronic device to implement operations for, while a media program is being provided to the display device for display thereon;

capturing screen display data associated with the media program displayed on the display device;

determining whether a program information overlay is being displayed on the display device, including analyzing the captured screen display data to determine whether the screen display data includes one or more predefined indicators of the program information overlay;

in accordance with a determination that the program information overlay is being displayed on the display device, processing information displayed on the program information overlay;

generating search terms from the processed information;

performing a search based on a subset of the generated search terms to identify content associated with the media program; and

while the media program is being displayed on the display device, transmitting search results of the search to at least one of the display device and the second screen client device for concurrent and synchronous display thereon.

10. The electronic device of claim 9, wherein generating search terms comprises:

identifying noun phrases in the processed information; and

selecting a threshold number of the noun phrases to be the search terms.

11. The electronic device of claim 10, wherein identifying noun phrases comprises:

removing program showing data from the processed information to produce one or more descriptive phrases; and

removing at least one verb from the one or more descriptive phrases to produce one or more noun phrases.

12. The electronic device of claim 11, wherein identifying noun phrases comprises comparing segments of the one or more descriptive phrases to known names of television shows, movies, sporting events and known proper names.

13. The electronic device of claim 10, the one or more programs further comprising instructions for refining each of the threshold number of the noun phrases that are selected to be the search terms by at least one of:

checking and correcting for typographical errors;

cross-referencing against known names of television shows, movies, sporting events and known proper names; and

cross-referencing against prior search data.

14. The electronic device of claim 10, wherein performing the search comprises performing a respective search for each of the threshold number of the noun phrases, and wherein each respective search produces corresponding search results.

15. The electronic device of claim 9, the one or more programs further comprising instructions for:

scoring the search results; and

enabling display of at least some of the search results based on the scoring.

16. The electronic device of claim 15, wherein the scoring comprises:

categorizing the search results;

assigning a query score based on at least one of location data, search history and user preferences;

assigning a proximity score to each of a threshold number of noun phrases selected as search terms based on an order in which the respective noun phrase appears in the processed information; and

multiplying each query score by a corresponding proximity score to produce a respective relevance score, wherein the search results are displayed based at least on the respective relevance scores.

17. A non-transitory computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device cause the electronic device to implement operations for:

at the electronic device, wherein the electronic device is communicatively coupled to a display device and a second screen client device, while a media program is being provided to the display device for display thereon;

capturing screen display data associated with the media program displayed on the display device;

determining whether a program information overlay is being displayed on the display device, including analyzing the captured screen display data to determine whether the screen display data includes one or more predefined indicators of the program information overlay;

in accordance with a determination that the program information overlay is being displayed on the display device, processing information displayed on the program information overlay;

generating search terms from the processed information;

performing a search based on a subset of the generated search terms to identify content associated with the media program; and

while the media program is being displayed on the display device, transmitting search results of the search to at least one of the display device and the second screen client device for concurrent and synchronous display thereon.

18. The non-transitory computer readable storage medium of claim 17, wherein the search is performed when the electronic device submits a search query based on the at least some of the generated search terms to a search engine that is configured to provide information stored on one or more third party content servers according to heuristic data of prior searches of a plurality of users and content providers.

19. The non-transitory computer readable storage medium of claim 17, the one or more programs further comprising instructions for:

receiving from the second screen client device a content search request associated with the media program displayed on the display device.

20. The non-transitory computer readable storage medium of claim 17, the one or more programs further comprising instructions for:

initiating the display of the program information overlay; and

ending the display of the program information overlay in response to capturing screen display data associated with the media program displayed on the display device.