Metadata from image recognition

• US 8,296,808 B2
• Filed: 04/16/2007
• Issued: 10/23/2012
• Est. Priority Date: 10/23/2006
• Status: Active Grant

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Hardacker, Robert L., Candelore, Brant L.
• Primary Examiner(s): Ingvoldstad, Bennett
• Application Number:
  US11/787,326
• Publication Number:
  US 20080098432A1
• Time in Patent Office: 2,017 Days
• Field of Search: 382/118
• US Class Current: 725/51
• CPC Class Codes: H04N 21/4355 involving reformatting operations of additional data, e.g. HTML pages on a television screen optimising the visualization of content for information retrieval from the Internet G06F16/9577; adaptation of message content in packet-switching networks H04L51/066; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A method of acquiring data associated with a television program consistent with certain embodiments involves acquiring information that identifies a currently playing television program; receiving a command from a user interface that selects an image forming a portion of a frame of video displayed on the television, wherein said frame of video is a portion of the television program; accessing a specified web site that contains a database of metadata associated with television programs via the Internet; querying the specified web site for metadata associated with the image by providing the image along with the information that identifies the currently playing television program; receiving a response from the specified web site that provides metadata associated with the image; and displaying at least a portion of the metadata. This abstract is not to be considered limiting, since other embodiments may deviate from the features described in this abstract.

Claims

1. A method of acquiring data associated with a television program, comprising:

acquiring information that identifies a currently playing television program playing on a television, the acquiring comprising;

the television sending one or more commands to an external access device to cause the access device to produce video signals representing an electronic program guide (EPG);

optical character reader processing one or more video signals representing video images of the electronic program guide provided by the access device, where the access device generates the EPG by direct access to EPG metadata and where the television does not have direct access to the EPG metadata used by the access device to generate the EPG except via video signals,

generating an alternate EPG for use by the television by reformatting the results of the OCR process, and

identifying the currently playing content by reference to the alternate EPG;

receiving a command from a user interface that selects an image forming a portion of a frame of video displayed on the television, where said frame of video is a portion of the television program;

accessing a specified web site that contains a database of metadata associated with television programs via the Internet;

querying the specified web site for metadata associated with the image by providing the image along with the information that identifies the currently playing television program to the web site;

receiving a response from the specified web site that provides metadata associated with the image; and

displaying at least a portion of the metadata.

2. The method according to claim 1, further comprising storing the metadata to a memory.

3. The method according to claim 1, where the displaying is carried out by display of the metadata in one of a Picture in Picture (PIP) image, Picture Outside Picture (POP) display, full screen display, and a display of a display enabled remote control.

4. The method according to claim 1, where the acquiring comprises identifying the currently playing content by reference to an electronic program guide.

5. The method according to claim 1, where the acquiring further comprises identifying the currently playing content by a combination of the current time, an approximate physical location, a selected channel and a service provider.

6. The method according to claim 1, where the query response is provided by use of an image recognition process to recognize the image.

7. The method according to claim 6, where the image comprises an image of a face and wherein the query response is provided by use of a facial recognition process to recognize the face.

8. The method according to claim 1, where the query response includes a link to a web site containing additional content.

9. A method of acquiring data associated with a television program, comprising:

acquiring information that identifies a currently playing television program playing on a television, the acquiring comprising;

the television sending one or more commands to an external access device to cause the access device to produce video signals representing an electronic program guide (EPG);

optical character reader processing one or more video signals representing video images of the electronic program guide provided by the access device, where the access device generates the EPG by direct access to EPG metadata and where the television does not have direct access to the EPG metadata used by the access device to generate the EPG except via video signals,

generating an alternate EPG for use by the television by reformatting the results of the OCR process, and

identifying the currently playing content by reference to the alternate EPG;

receiving a command from a user interface that selects an image forming a portion of a frame of video displayed on the television, where said frame of video is a portion of the television program;

accessing a specified web site that contains a database of metadata associated with television programs via the Internet;

querying the specified web site for metadata associated with the image by providing the image along with the information that identifies the currently playing television program to the web site;

receiving a response from the specified web site that provides metadata associated with the image, where the query response is provided by use of an image recognition process to recognize the image;

storing the metadata to a memory; and

displaying at least a portion of the metadata.

10. The method according to claim 9, where the displaying is carried out by display of the metadata in one of a Picture in Picture (PIP) image, Picture Outside Picture (POP) display, full screen display, and a display of a display enabled remote control.

11. The method according to claim 9, where the acquiring further comprises identifying the currently playing content by a combination of the current time, an approximate physical location, a selected channel and a service provider.

12. The method according to claim 9, where the image comprises an image of a face and wherein the query response is provided by use of a facial recognition process to recognize the face.

13. The method according to claim 9, where the query response includes a link to a web site containing additional content.

14. A digital television receiver device that provides access to content associated with a television program, comprising:

a video decoder that receives video content from a video source of an access device;

a frame memory that stores a frame of video representing an electronic program guide (EPG);

an optical character reader that processes one or more video signals representing video images of the electronic program guide provided by an external access device in response to the television receiver device sending one or more commands to the access device to cause the access device to produce video signals representing the EPG;

where the access device generates the EPG by direct access to EPG metadata and where the television does not have direct access to the EPG metadata used by the access device to generate the EPG except via video signals;

a programmed processor that generates an alternate EPG for use by the television by reformatting the results of the OCR process;

the programmed processor identifying the currently playing content by reference to the alternate EPG;

a user interface that receives a command that selects an image forming a portion of a frame of video displayed on the television, where said frame of video is a portion of the television program;

the programmed processor controlling an Internet connection that accesses a specified web site that contains a database of metadata associated with television programs via the Internet in order to query the specified web site for metadata associated with the image by providing the image along with the information that identifies the currently playing television program;

the programmed processor receiving a response from the specified web site that provides metadata associated with the image; and

a display that displays at least a portion of the metadata.

15. The digital television device according to claim 14, further comprising a storage device for storing the metadata.

16. The digital television device according to claim 14, where the display comprises one of a Picture in Picture (PIP) image, Picture Outside Picture (POP) display, full screen display, and a display of a display enabled remote control.

17. The digital television device according to claim 14, the currently playing content is further identified by a combination of the current time, an approximate physical location, a selected channel and a service provider.

18. The digital television device according to claim 14, wherein the query response is provided by use of an image recognition process to recognize the image.

19. The digital television device according to claim 18, where the image comprises an image of a face and wherein the query response is provided by use of a facial recognition process to recognize the face.

20. The digital television device according to claim 14, where the query response includes a link to a web site containing additional content.

21. The digital television device according to claim 14, further comprising a browser for navigating to the web site containing additional content.

22. A method of sourcing data associated with a television program, comprising:

at a television, acquiring information that identifies a currently playing television program playing on a television, the acquiring comprising;

the television sending one or more commands to an external access device to cause the access device to produce video signals representing an electronic program guide (EPG);

optical character reader processing one or more signals representing video images of the electronic program guide provided by the access device, where the access device generates the EPG by direct access to EPG metadata and where the television does not have direct access to the EPG metadata used by the access device to generate the EPG except via video signals,

generating an alternate EPG for use by the television by reformatting the results of the OCR process, and

identifying the currently playing content by reference to the alternate EPG;

at the television, receiving a command from a user interface that selects an image forming a portion of a frame of video displayed on the television, where said frame of video is a portion of the television program;

accessing a specified web site that contains a database of metadata associated with television programs via the Internet;

querying the specified web site for metadata associated with the image by providing the image along with the information that identifies the currently playing television program to the web site;

at the web site, receiving a search query that includes information that identifies a television program and an image appearing in a frame of the television program;

at the web site, applying the search query to a database of metadata associated with television programs using the image along with the information that identifies the currently playing television program as search criteria;

at the web site, receiving a response from the database, the response including metadata associated with the image;

sending the metadata to the television as a reply to the search query;

at the television, receiving a response from the specified web site that provides metadata associated with the image; and

displaying at least a portion of the metadata on the television.

23. The method according to claim 22, where the identifying information includes at least one of the play time of the content, an approximate physical location, a selected channel and a service provider.

24. The method according to claim 22, where the query response is provided by use of an image recognition process to recognize the image.

25. The method according to claim 24, where the image comprises an image of a face and wherein the query response is provided by use of a facial recognition process to recognize the face.

26. The method according to claim 22, where the query response includes a link to a web site containing additional content.