Using OCR to detect currently playing television programs

• US 10,735,792 B2
• Filed: 09/19/2012
• Issued: 08/04/2020
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sandland, Nathan Leslie
• Primary Examiner(s): Flynn, Nathan J
• Assistant Examiner(s): Trinh, Tung T
• Application Number:
  US13/623,043
• Publication Number:
  US 20140082646A1
• Time in Patent Office: 2,876 Days
• Field of Search: 725 14
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Features of various implementations are used to enable determining what a particular consumer is watching on television in order to deliver matched functionality, content and/or services, as a part of an interactive television suite of technologies. In some implementations, the process of determining what a particular consumer is watching is enabled so that communication between the primary set-top box and a supplemental set-top box and/or television integrated device is reduced, thereby avoiding the need for one or more provider specific APIs. For example, one aspect of the disclosure is a computer-implemented method of identifying a media program by capturing display data from the media program, and extracting text from the display data in response to determining that the display data includes the text overlay, wherein the extracted text is associated with the media program.

Claims

1. A method of identifying a media program, comprising:

at a first electronic device coupled to a display device displaying the media program, the first electronic device including a processor and memory;

without issuing a command to the display;

capturing an image of display data from the media program;

evaluating the image to determine whether or not the display data includes a portion with a text overlay with information about the media program, including evaluating one or more of;

a color of the portion;

a percentage of the image covered by the portion;

an arrangement of text within the portion; and

a contrast between the portion and the media program;

in accordance with a determination that the display data does not include the text overlay, initiating a delay period;

at the end of the delay period and without being prompted by user interaction, repeating the capturing and the evaluating;

in accordance with a determination that the display data does include the text overlay, extracting text from the text overlay; and

obtaining the identity of the media program based on the extracted text.

2. The method of claim 1, wherein capturing the display data includes performing a screen capture from the display device.

3. The method of claim 1, wherein capturing the display data includes decoding data transmitted by a second electronic device to the display device.

4. The method of claim 1, further comprising comparing the extracted text to electronic program guide data to confirm identification of the playing media program.

5. The method of claim 1, further comprising:

detecting, without initiating, a remote control signal directed at a second electronic device that has access to the display data, wherein the remote control signal causes the display data to be displayed on a display device associated with the second electronic device; and

in response to detecting the remote control signal, capturing display data from the media program.

6. The method of claim 1, further comprising detecting a control signal directed at a second electronic device that has access to the display data;

wherein the control signal is generated by a mobile device configured to communicate with both the first electronic device and the second electronic device;

wherein the control signal causes the display data to be displayed on the display; and

wherein capturing display data from the media program comprises capturing the display data in response to detecting the control signal.

7. The method of claim 1, further comprising verifying the identity of the media program by comparing the extracted text with data from a database.

8. The method of claim 7, wherein the first electronic device includes the database.

9. The method of claim 1, wherein the first electronic device comprises a mobile device, and wherein capturing display data from the media program comprises capturing display data via a TV app on the first electronic device.

10. The method of claim 1, wherein capturing display data from the media program comprises capturing display data via a camera on the first electronic device.

11. A computer program product enabling identification of a playing broadcast media program by a first electronic device including a processor and a memory, the computer program product including instructions, stored in a non-transitory memory and executable by a processor, which when executed by a processor, cause the first electronic device to:

without issuing a command to a display device displaying display data;

capture an image of display data from the media program;

evaluate the image to determine whether or not the display data includes a portion with a text overlay with information about the media program, including evaluating one or more of;

a color of the portion;

a percentage of the image covered by the portion;

an arrangement of text within the portion; and

a contrast between the portion and the media program;

in accordance with a determination that the display data does not include the text overlay, initiate a delay period;

at the end of the delay period and without being prompted by user interaction, repeat the capturing and the evaluating;

in accordance with a determination that the display data does include the text overlay; and

obtain the identity of the media program based on the extracted text.

12. The computer program product of claim 11, wherein capturing the display data includes performing a screen capture from the display device.

13. The computer program product of claim 11, wherein capturing the display data includes decoding data transmitted by a second electronic device to the display device.

14. The computer program product of claim 11, wherein the one or more programs further include instructions to:

compare the extracted text to electronic program guide data to confirm identification of the playing broadcast media program.

15. The computer program product of claim 11, wherein the one or more programs comprise instructions, which when executed by the processor, cause the first electronic device to:

detect, without initiating, a remote control signal directed at a second electronic device that has access to the display data, wherein the remote control signal causes the display data to be displayed on a display device associated with the second electronic device; and

in response to detecting the remote control signal, capture display data from the media program.

16. A first electronic device, comprising:

a processor and

memory for storing one or more programs, wherein the one or more programs comprise instructions, which when executed by the processor, cause the first electronic device to;

without issuing a command to a display device displaying display data;

capture an image of display data from a media program;

evaluate the image to determine whether or not the display data includes a portion with a text overlay with information about the media program, including evaluating one or more of;

a color of the portion;

a percentage of the image covered by the portion;

an arrangement of text within the portion; and

a contrast between the portion and the media program;

in accordance with a determination that the display data does not include the text overlay, initiate a delay period;

at the end of the delay period and without being prompted by user interaction, repeat the capturing and the evaluating;

in accordance with a determination that the display data does include the text overlay; and

obtain the identity of the media program based on the extracted text.

17. The first electronic device of claim 16, wherein capturing the display data includes performing a screen capture from the display device.

18. The first electronic device of claim 16, wherein capturing the display data includes decoding data transmitted by a second electronic device to the display device.

19. The first electronic device of claim 16, wherein the one or more programs further comprise instructions to:

compare the extracted text to electronic program guide data to confirm identification of the playing media program.

20. The first electronic device of claim 16, wherein the one or more programs further comprise instructions to:

detect, without initiating, a remote control signal directed at a second electronic device that has access to the display data, wherein the remote control signal causes the display data to be displayed on a display device associated with the second electronic device; and

in response to detecting the remote control signal, capture display data from the media program.