Real time overlays on live streams

• US 8,832,741 B1
• Filed: 04/03/2012
• Issued: 09/09/2014
• Est. Priority Date: 04/03/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bota, Silviu, Milatinovici, Bianca, Falk, Christian Ingemar
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Baig, Sahar
• Application Number:
  US13/438,158
• Time in Patent Office: 889 Days
• Field of Search: 725/40, 725/43, 725/48, 725/51, 725/52
• US Class Current: 725/40
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/23106 involving caching operations prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/262 Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6408 Unicasting
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A system and method for generating and distributing overlay streams in real time over an overlay stream distribution channel that overlays events of a video stream distributed in a separate video channel. Overlay streams are distributed to subscriber recipient components via a unicast channel in order to enhance user viewing. A publisher subscriber component and caching techniques a low latency, high availability unicasting based real time overlay distribution system for a large numbers of concurrent overlay recipients.

Claims

1. A system, comprising:

a memory having stored thereon computer executable components; and

a processor configured to execute the following computer executable components stored in the memory;

a set of overlay distribution components that publishes a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals, and retrieve a subset of an overlay stream of the set of overlay streams based on a time interval request from an overlay recipient, and the local storage and the local cache having less than all of the overlay stream stored.

2. The system of claim 1, wherein the set of overlay distribution components respectively comprise an overlay communication component that communicates the set of overlay streams to a plurality of overlay recipients over the second channel being a different channel than the first channel to overlay and correspond with the events occurring live in the live video stream.

3. The system of claim 2, wherein the overlay communication component communicates the set of overlay streams concurrently as unicast streams over the second channel, the second channel being a unicast channel.

4. The system of claim 1, further comprising a set of overlay publisher components that generate an overlay stream having a set of overlays corresponding to the events within the live video stream in real time via the second channel, the set of overlays including text and data images generated in real time in response to the events.

5. The system of claim 4, wherein the set of overlay publisher components submit the set of overlay messages corresponding to the events within the live video stream to the set of overlay distribution components.

6. The system of claim 4, wherein the overlay recipient receives the live video stream of events and at least one overlay stream of the set of overlay streams from at least one overlay distribution component of the set of overlay distribution components based on a subscription with at least one overlay publisher component of the set of overlay publisher components.

7. The system of claim 1, wherein overlay streams of the set of overlay streams comprise data images, including annotations generated in real time corresponding to the events of the live video stream in real time, from a set of overlay publisher components, wherein the set of overlay distribution components are communicatively connected via a network including a cloud network or a wide area network.

8. The system of claim 1, further comprising a publisher subscriber component that coordinates the set of overlay messages from a set of overlay publisher components to be published in the set of overlay streams to the set of overlay distribution components by communicating the set of overlay messages to the set of overlay distribution components.

9. The system of claim 8, wherein the publisher subscriber component associates a set of overlay recipients to particular publisher components of the set of overlay publisher components, and communicates a subscription of the set of overlay recipients to the particular publisher components to the set of overlay distribution components.

10. The system of claim 8, wherein the publisher subscriber component and the set of overlay distribution components respectively generate an overlay stream of the set of overlay streams corresponding to the time interval of the overlay stream to the overlay recipient that subscribes to an overlay publisher component of the set of overlay publisher components based on the request from the overlay recipient being received.

11. The system of claim 1, wherein the set of overlay distribution components generate at least one overlay stream of the set of overlay streams from the local storage in response to the time interval request from the overlay recipient exceeding an available overlay time span of the local cache.

12. A method, comprising:

publishing, by a system including a processor via a set of overlay distribution components, a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals; and

retrieving, by the system via the set of overlay distribution components, a subset of an overlay stream of the set of overlay streams based on a time interval request from an overlay recipient, and the local storage and the local cache having less than all of the overlay stream stored.

13. The method of claim 12, further comprising communicating, by the system, the set of overlay streams to a plurality of overlay recipients over the second channel being a different channel than the first channel to overlay and correspond with the events occurring live in the live video stream.

14. A non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations comprising:

publishing, via a set of overlay distribution components, a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals; and

retrieving, via the set of overlay distribution components, a subset of an overlay stream of the set of overlay streams based on a time interval request from an overlay recipient, and the local storage and the local cache having less than all of the overlay stream stored.

15. A system, comprising:

a memory having stored thereon computer executable components; and

a processor configured to execute the following computer executable components stored in the memory;

a set of overlay distribution components that publishes a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals, and generate at least one overlay stream of the set of overlay streams from the local storage in response to the time interval request from the overlay recipient exceeding an available overlay time span of the local cache.

16. A method, comprising:

publishing, by a system including a processor via a set of overlay distribution components, a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals; and

generating, by the system via the set of overlay distribution components, at least one overlay stream of the set of overlay streams from the local storage in response to the time interval request from the overlay recipient exceeding an available overlay time span of the local cache.

17. A non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations comprising:

publishing, via a set of overlay distribution components, a set of overlay streams in real time via a second channel respectively, the set of overlay streams having a set of overlay messages that correspond to events of a live video stream of events occurring in real time distributed over a first channel, wherein the set of overlay messages enhance user viewing of the live video stream of events occurring in real time, and wherein the set of overlay distribution components include a local storage and a local cache respectively for retrieving the set of overlay streams corresponding to various time intervals; and

generating, via the set of overlay distribution components, at least one overlay stream of the set of overlay streams from the local storage in response to the time interval request from the overlay recipient exceeding an available overlay time span of the local cache.