Associating media using metadata and controlling multiple-device synchronization and rendering

• US 8,978,075 B1
• Filed: 01/15/2013
• Issued: 03/10/2015
• Est. Priority Date: 01/18/2012
• Status: Active Grant

• Current Assignee: Coincident TV Incorporated
• Original Assignee: Coincident TV Incorporated
• Inventors: Kaiser, David H., Schwartz, Bruce, Kalinin, Alex
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): ANDRAMUNO, FRANKLIN S
• Application Number:
  US13/742,341
• Time in Patent Office: 784 Days
• Field of Search: 725/62, 725/88, 725/105
• US Class Current: 725/62
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4825 using a list of items to be played back in a given order, e.g. playlists
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

A computer-implemented method comprises determining, at a companion device separate from a television, a current time point of a video program playing at the television, wherein the current time point of the video program is a time point currently being displayed at the television; identifying, based on said determination of the current time point of the video program and based on metadata defining a cue point associated with the determined current time point and an operation associated with the cue point, an operation to be performed; performing, at the companion device, the identified operation; wherein the method is performed by one or more computing devices.

Claims

1. A computer-implemented method comprising:

determining, at a companion device separate from a television, a current time point of a video program playing at the television, wherein the current time point of the video program is a time point currently being displayed at the television;

identifying, based on said determination of the current time point of the video program and based on metadata defining a cue point associated with the determined current time point and an operation associated with the cue point, a particular operation to be performed;

performing, at the companion device, the particular operation;

wherein the particular operation is to cause a display of supplemental content at the television;

determining that the supplemental content has concluded replaying;

in response to said determination that the supplemental content has concluded replaying, sending instructions to a set top box associated with the television, a video server containing the video program, or the television to resume replaying the video program at a resume time point of the video program;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the companion device is a tablet computer, mobile computing device, smartphone, personal computer, laptop computer, or netbook computer.

3. The method of claim 1, wherein said determination of the current time point of the video program comprises:

sending a request for content identification information to;

the set top box associated with the television, the video server containing the video program, or the television;

receiving content identification information identifying the video program being displayed and an initial time point of the video program, wherein the initial time point of the video program is a time point of the video program displaying at a time the content identification information was received;

determining, based on the initial time point of the video program, the current time point of the video program.

4. The method of claim 1, wherein said determination of the current time point of the video program comprises adding a calculated amount of time to a particular time point, wherein the particular time point is a time point specified in an instruction to display the video program and the calculated amount of time is an amount of time passed since a time at which the video program began displaying.

5. The method of claim 1, wherein the supplemental content determined based on the metadata.

6. The method of claim 1, wherein said performance of the particular operation comprises sending instructions to the set top box associated with the television, the video server containing the supplemental content, or the television to display supplemental content at the television.

7. The method of claim 6, wherein said instructions indicate that the supplemental content is to be displayed concurrently with the video program.

8. The method of claim 6, wherein said instructions indicate that supplemental content is to be overlaid upon the video program during display.

9. The method of claim 1, wherein the video program is live content and a portion of the video program is buffered in the set top box associated with the television.

10. The method of claim 1, wherein the video program is live content and the metadata is periodically updated.

11. The method of claim 1, wherein the associated operation is to display one or more graphics, a second video, or an online electronic document.

12. The method of claim 11, wherein the particular operation is to display a graphic image, and the graphic image represents an annotation, and the method further comprises:

receiving a selection of the annotation;

in response to determining that the annotation has been selected, performing a certain operation associated with the annotation.

13. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

determining, at a companion device separate from a television, a current time point of a video program playing at the television, wherein the current time point of the video program is a time point currently being displayed at the television;

identifying, based on said determination of the current time point of the video program and based on metadata defining a cue point associated with the determined current time point and an operation associated with the cue point, a particular operation to be performed;

performing, at the companion device, the particular operation;

wherein the particular operation is to cause a display of supplemental content at the television;

determining that the supplemental content has concluded replaying;

in response to said determination that the supplemental content has concluded replaying, sending instructions to a set top box associated with the television, a video server containing the video program, or the television to resume replaying the video program at a resume time point of the video program.

14. The non-transitory computer-readable data storage medium of claim 13, wherein the companion device is a tablet computer, mobile computing device, smartphone, personal computer, laptop computer, or netbook computer.

15. The non-transitory computer-readable data storage medium of claim 13, wherein said determination of the current time point of the video program comprises:

sending a request for content identification information to;

the set top box associated with the television, the video server containing the video program, or the television;

receiving content identification information identifying the video program being displayed and an initial time point of the video program, wherein the initial time point of the video program is a time point of the video program displaying at a time the content identification information was received;

determining, based on the initial time point of the video program, the current time point of the video program.

16. The non-transitory computer-readable data storage medium of claim 13, wherein said determination of the current time point of the video program comprises adding a calculated amount of time to a particular time point, wherein the particular time point is a time point specified in an instruction to display the video program and the calculated amount of time is an amount of time passed since a time at which the video program began displaying.

17. The non-transitory computer-readable data storage medium of claim 13, wherein the supplemental content is determined based on the metadata.

18. The non-transitory computer-readable data storage medium of claim 13, wherein said performance of the particular operation comprises sending instructions to the set top box associated with the television, the video server containing the supplemental content, or the television to display supplemental content at the television.

19. The non-transitory computer-readable data storage medium of claim 18, wherein said instructions indicate that the supplemental content is to be displayed concurrently with the video program.

20. The non-transitory computer-readable data storage medium of claim 18, wherein said instructions indicate that supplemental content is to be overlaid upon the video program during display.

21. The non-transitory computer-readable data storage medium of claim 13, wherein the video program is live content and a portion of the video program is buffered in the set top box associated with the television.

22. The non-transitory computer-readable data storage medium of claim 13, wherein the video program is live content and the metadata is periodically updated.

23. The non-transitory computer-readable data storage medium of claim 13, wherein the associated operation is to display one or more graphics, a second video, or an online electronic document.

24. The non-transitory computer-readable data storage medium of claim 23, wherein the particular operation is to display a graphic image and the graphic image represents an annotation, and the instructions which, when executed by one or more processors, further cause the one or more processors to perform:

receiving a selection of the annotation;

in response to determining that the annotation has been selected, performing a certain operation associated with the annotation.