Identification and presentation of internet-accessible content associated with currently playing television programs

• US 9,788,055 B2
• Filed: 03/14/2013
• Issued: 10/10/2017
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Verrilli, Michael, Sandland, Nathan Leslie
• Primary Examiner(s): Flynn, Nathan
• Assistant Examiner(s): TRINH, TUNG THANH
• Application Number:
  US13/831,166
• Publication Number:
  US 20140082647A1
• Time in Patent Office: 1,671 Days
• Field of Search: 725 62
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: H04N 21/44204 Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image

Abstract

Features of various implementations are used to enable identifying and presenting, in real time, Internet-accessible and/or Internet-based functionality, content and/or services associated with a particular television program concurrently playing on a television or display, as a part of an interactive television suite of technologies. In some implementations, the process of identifying Internet-accessible and/or Internet-based functionality, content and/or services associated with a particular television program is enabled to reduce communication between a primary set-top box and a supplemental set-top box and/or television integrated device, thereby reducing the need for one or more provider specific APIs. For example, one aspect of the disclosure is a computer-implemented method of identifying a media program by capturing display data from the media program, and extracting text from the display data in response to determining that the display data includes the text overlay, wherein the extracted text is associated with the media program.

Claims

1. A computer-implemented method, comprising:

at an electronic device that includes a processor and memory, automatically and without user interaction at the electronic device;

streaming a media program to a first client device for display on the first client device;

receiving a content search request associated with the media program displayed on the first client device, wherein the content search request is received from a second client device that is communicatively coupled to the first client device;

obtaining an image of what is being displayed on the first client device by capturing screen display data associated with the media program displayed on the first client device;

in response to the content search request;

after obtaining the image of what is being displayed on the first client device;

analyzing the obtained image for one or more predefined indicators of a program information overlay including information about the media program, wherein the program information overlay is distinct from the media program;

in response to the analysis, determining whether the one or more predefined indicators are present in the obtained image; and

in response to determining that the obtained image includes the one or more predefined indicators, extracting text displayed on the program information overlay in the obtained image, wherein the extracted text is associated with the media program;

generating search terms from the extracted text;

performing an Internet search based on at least some of the generated search terms to identify content associated with the media program; and

transmitting the results of the Internet search to the second screen client device for concurrent display thereon when the media program is displayed on the first client device.

2. The method of claim 1, wherein the Internet search is performed when the electronic device submits a search query based on the at least some of the generated search terms to a search server that is configured to provide information stored on one or more third party content servers according to heuristic data of prior searches of a plurality of users and content providers.

3. The method of claim 1, wherein capturing the display data includes performing a screen capture from a display device configured to display the media program.

4. The method of claim 1, wherein capturing the display data includes decoding media program data streamed by the electronic device to the display device.

5. The method of claim 1, further comprising comparing the extracted text to electronic program guide data to confirm identification of the playing broadcast media program.

6. The method of claim 1, further comprising:

transmitting a code to a set-top box to initiate the display of the text overlay; and

transmitting a code to the set-top box to end the display of the text overlay in response to capturing the display data.

7. The method of claim 6, wherein the code is an infrared code.

8. The method of claim 1, further comprising:

detecting a transmission directed at a set-top box;

determining whether the transmission includes a code that generally causes the set-top box to provide the text overlay in the display data, and

wherein capturing the display data occurs in response to determining that the transmission includes a code that generally causes the set-top box to provide the text overlay in the display data.

9. The method of claim 8, wherein the code is an infrared code.

10. The method of claim 1, wherein extracting text from the display data comprises applying an optical character recognition process to the captured display data.

11. The method of claim 1, wherein generating search terms comprises:

identifying noun phrases in the extracted text; and

selecting a threshold number of the noun phrases to be search terms.

12. The method of claim 11, wherein identifying noun phrases comprises:

removing program showing data from the extracted text to produce one or more descriptive phrases; and

removing at least one verb from the one or more descriptive phrases to produce one or more noun phrases.

13. The method of claim 12, wherein identifying noun phrases comprises comparing segments of the one or more descriptive phrases to known names of television shows, movies, sporting events and known proper names.

14. The method of claim 11, further comprising refining each of the selected threshold number of search terms by at least one of:

checking and correcting for typographical errors;

cross-referencing against known names of television shows, movies, sporting events and known proper names; and

cross-referencing against prior search data.

15. The method of claim 11, wherein performing the Internet search comprises performing a respective Internet search for each of the threshold number of noun phrases, and wherein each respective Internet search produces corresponding search results.

16. The method of claim 15, further comprising:

scoring the search results; and

displaying at least some of the search results based on the scoring.

17. The method of claim 16, wherein the scoring comprises:

categorizing the search results;

assigning a query score based on at least one of location data, search history and user preferences;

assigning a proximity score to each of the threshold number of the noun phrases selected as search terms based on the order the noun phrases appear the extracted text; and

multiplying each query score by a corresponding proximity score to produce a respective relevance score.

18. The method of claim 17, wherein the results of the search are displayed based at least on the respective relevance scores.

19. The method of claim 1, wherein the Internet search is selected from a group consisting of a general Internet search, a targeted search for associated news items, a targeted search for associated images, a targeted search for associated Internet-accessible media content, and a targeted search for associated social media content.

20. A non-transitory computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device cause the device to automatically, without user interaction at the computing device:

streaming a media program to a first client device for display on the first client device;

receive a content search request associated with a media program displayed on a first client device, wherein the content search request is received from a second client device that is communicatively coupled to the first client device;

obtain an image of what is being displayed on the first client device by capturing screen display data provided by a set-top box and associated with the media program;

in response to the content search request;

after obtaining the image of what is being displayed on the first client device;

analyzing the obtained image for one or more predefined indicators of a program information overlay including information about the media program, wherein the program information overlay is distinct from the media program;

in response to the analysis, determining whether the one or more predefined indicators are present in the obtained image; and

in response to determining that the obtained image includes the one or more predefined indicators, extracting text displayed on the program information overlay in the obtained image, wherein the extracted text is associated with the media program;

generate search terms from the extracted text;

perform an Internet search based on at least some of the generated search terms to identify content associated with the media program; and

transmit the results of the Internet search to the second screen client device for concurrent display thereon when the media program is displayed on the first client device.