Techniques to consume content and metadata

• US 8,301,618 B2
• Filed: 02/26/2008
• Issued: 10/30/2012
• Est. Priority Date: 02/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Allard, James E.
• Primary Examiner(s): Nguyen, Phong
• Assistant Examiner(s): MYINT, DENNIS Y
• Application Number:
  US12/037,899
• Publication Number:
  US 20090216745A1
• Time in Patent Office: 1,708 Days
• Field of Search: 707/712, 707/732
• US Class Current: 707/712
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Content and metadata associated with the content may be provided to a number of users. The content may be displayed on a display device while the metadata may be transmitted to a remote device corresponding to a receiving user. The user may further request desired information or metadata pertaining to the content and the requested information or metadata may be transmitted to the user'"'"'s remote device. Different users may request different information on the same or different objects being displayed or presented on a display device. Each requesting user may receive requested information on the same or different objects via corresponding remote devices.

Claims

1. A method comprising at a client device:

receiving content in a broadcast;

identifying a request received from a remote device to capture a still image from the content in the broadcast;

capturing the still image from the content in the broadcast in response to the request from the remote device;

transmitting the still image for receipt and display by the remote device;

receiving from the remote device image data comprising a portion of the still image displayed at the remote device, the portion of the still image selected by a user via a user interface output at the remote device and including at least one object to be identified;

transmitting the portion of the still image for receipt by one or more servers to identify metadata that is relevant to the request by processing the portion of the still image using image recognition to identify the at least one object;

receiving the identified metadata from the one or more servers, the identified metadata associated with the at least one object in the portion of the still image; and

outputting the received metadata for the remote device to display the received metadata.

2. A method as described in claim 1, wherein the identifying is performed in real time as the content is received.

3. A method as described in claim 1, further comprising storing the content in storage at the client device, wherein the identifying and the outputting are performed as the content is stored.

4. A method as described in claim 1, further comprising storing the content in storage at the client device, wherein the identifying is performed as the content is output from the storage.

5. A method as described in claim 1, wherein the remote device is communicatively coupled to the client device to cause the client device to implement one or more functions that are local to the client device.

6. A method as described in claim 1, wherein the content is streamed to the client device from one or more servers of a head end of a network operator.

7. A method as described in claim 1, wherein the identifying is performed for a plurality of said remote devices such that each said remote device receives said metadata that is particular to respective preferences.

8. A method comprising at a remote device:

creating a persistent network connection between the remote device and a client device;

forming a request to cause a still image to be captured by the client device from content output in a broadcast at the client device and transmitted to the remote device for display;

displaying, at the remote device, the still image received from the client device;

receiving a user input, via a user interface output at the remote device, that selects a portion of the still image displayed at the remote device, the portion of the still image including one or more objects to be identified;

transmitting image data comprising the portion of the still image from the remote device for receipt by the client device to obtain metadata associated with the one or more objects in the portion of the still image from one or more servers configured to process the portion of the still image using image recognition to identify the one or more objects in the still image;

receiving the metadata from the one or more servers;

displaying the metadata that corresponds to the one or more objects in the still image; and

transmitting the metadata for display at another remote device that corresponds to another user and is configured to communicate with the client device.

9. A method as described in claim 8, wherein the network connection between the remote device and the client device is persistent such that the network connection between the remote device and the client device is automatically created without user intervention.

10. A method as described in claim 8, wherein:

the creating is performed for a plurality of said client devices; and

the network connection is persistent such that the network connection between the remote device and each said client device is automatically created.

11. A method as described in claim 8, wherein the remote device is configured to cause the client device to implement one or more functions that are local to the client device.

12. A method as described in claim 8, wherein:

the content and the metadata are streamed to the client device from a head end of a network operator; and

the remote device receives the metadata via the client device.

13. A method as described in claim 8, wherein:

the content is streamed to the client device from a head end of a network operator; and

the metadata is streamed directly to remote device without being streamed to the client device.

14. A method as described in claim 8, wherein the request identifies at least a portion of the content output by the client device.

15. A method comprising at a remote device:

displaying, at the remote device, an image captured by a client device from video data displayed on the client device;

receiving a user input that selects a portion of the image to be used with image recognition to obtain metadata associated with one or more objects in the portion of the image, the user input received in response to a user circling the one or more objects in the portion of the image via a user interface output by the remote device;

transmitting the portion of the image and not the still image as a whole from the remote device for receipt by the client device to identify the metadata associated with the one or more objects in the portion of the image;

displaying the metadata on a display screen integrated with the remote device to enable user interaction with the metadata.

16. The method as described in claim 15, wherein the metadata displayed by the remote device comprises metadata based on one of remote device preferences or user preferences.

17. The method as described in claim 15, further comprising causing the client device to capture a still image from the video data being displayed and transmit the still image to the remote device without interrupting a current broadcast of the video data at the client device.

18. The method as described in claim 15, wherein the remote device and the client device are located in a same room.

19. The method as described in claim 15, further comprising receiving a user selection of the portion of the image, wherein the one or more objects in the portion include a face of an actor and the image recognition includes facial recognition used to identify the actor for obtaining metadata associated with the actor.

20. The method as described in claim 15, further comprising:

causing the portion of the image to be transmitted for receipt by one or more servers for the image recognition and identification of the one or more objects; and

receiving the metadata associated with the one or more objects for display on the display screen.

21. A method comprising at a server:

receiving via a network a portion of a still image captured from a broadcast of content at a client device, the still image captured by the client device in response to a request from a remote device that is remote from the client device, the still image transmitted to the remote device for display at the remote device to enable user selection of the portion of the still image via a user interface output at the remote device, the portion of the still image, and not the still image as a whole, transmitted from the remote device to the client device for transmission to the server via the network;

processing the portion of the still image using image recognition to identify one or more objects in the portion of the still image;

locating metadata that corresponds to the identified one or more objects; and

transmitting the located metadata via the network for receipt by the client device.

22. The method as described in claim 21, wherein the transmitting is performed by streaming the located metadata to the client device.

23. The method as described in claim 21, wherein the transmitting is performed by streaming the located metadata directly to a remote device without streaming the metadata to the client device, the remote device being communicatively coupled to the client device and having caused the still image to be captured.

24. The method as described in claim 21, wherein the server is a component of a head end of a network operator.

25. The method as described in claim 21, wherein the located metadata comprises metadata based on one of user preferences or remote device preferences of a remote device communicatively coupled to the client device.

26. The method as described in claim 21, wherein the one or more objects include a face of an actor and the image recognition includes facial recognition used to identify the actor and obtain corresponding metadata that is associated with the actor.

27. The method as described in claim 21, wherein the still image is captured from a current broadcast of the content at the client device without interrupting the current broadcast.