Systems and methods of server based interactive content injection

• US 10,306,293 B2
• Filed: 07/18/2017
• Issued: 05/28/2019
• Est. Priority Date: 07/18/2017
• Status: Active Grant

• Current Assignee: Wowza Media Systems LLC
• Original Assignee: Wowza Media Systems LLC
• Inventors: Talvensaari, Michael Dean, Sherry, James S., Borgersen, Jan Martin, Kellicker, Scott
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US15/653,128
• Publication Number:
  US 20190028758A1
• Time in Patent Office: 679 Days
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/2542 for selling goods, e.g. TV shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4758 for providing answers, e.g. voting
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8541 involving branching, e.g. to different story endings
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

A method includes receiving, at a media server, a request for a media stream from a client device. The method further includes sending, from the media server, a combined media stream to the client device via a streaming session. The combined media stream includes the requested media stream and interactive content. The interactive content is configured to cause a media player of the client device to display the interactive content concurrently with the media stream.

Claims

1. A method comprising:

receiving, at a media server, a request for a media stream from a client device;

sending via a streaming session, from the media server, a combined media stream to the client device, the combined media stream including the requested media stream and interactive content, wherein the interactive content is configured to cause a media player of the client device to display the interactive content concurrently with the requested media stream during the streaming session, wherein the combined media stream comprises data packets that include video, audio, or both of the media stream, and wherein the interactive content is included in headers of the data packets;

receiving, during the display of the interactive content concurrently with the requested media stream during the streaming session, a user selection corresponding to the interactive content; and

providing, via the streaming session, second interactive content to the client device based on the user selection, the second interactive content included in the combined media stream, wherein the second interactive content comprises Hypertext Markup Language (HTML) code or Extensible Markup Language (XML) code, and wherein the second interactive content is included in headers of the data packets.

2. The method of claim 1, wherein the streaming session complies with a hypertext transfer protocol (HTTP) Live Streaming (HLS) protocol, an HTTP Dynamic Streaming (HDS) protocol, a Dynamic Adaptive Streaming over HTTP (DASH) streaming protocol, or a WOWZ streaming protocol.

3. The method of claim 1, wherein the interactive content is sent in a Hypertext Markup Language (HTML) format or an Extensible Markup Language (XML) format.

4. The method of claim 1, wherein the interactive content includes placement information, wherein the interactive content indicates a start time to display the interactive content, and wherein the interactive content includes at least one of a duration or an end time.

5. The method of claim 1, wherein the interactive content includes text, links to graphics, links to Cascading Style Sheets (CSS) files, links to media, script languages or a combination thereof.

6. The method of claim 1, wherein the interactive content includes games, puzzles, quiz questions, advertisements, a Hypertext Markup Language (HTML) based application, or a combination thereof.

7. The method of claim 1, wherein the interactive content is injected into the combined media stream as network abstraction layer (NAL) units, and wherein the second interactive content is injected into the combined media stream as NAL units.

8. The method of claim 1, further comprising:

sending a manifest to the client device responsive to receiving the request for the media stream, the manifest indicating a plurality of adaptive bitrate (ABR) renditions of the media stream;

receiving a response from the client device indicating a particular ABR rendition of the media stream of the plurality of ABR renditions of the media stream; and

sending, from the media server via the streaming session to the client device, the particular ABR rendition of the media stream based on receiving the response, wherein providing, via the streaming session, the second interactive content to the client device based on the user selection includes injecting the second interactive content into at least one ABR rendition of the media stream of the plurality of ABR renditions of the media stream indicated in the manifest.

9. The method of claim 1, wherein the headers of the data packets further include placement information associated with the interactive content.

10. The method of claim 1, wherein sending the combined media stream to the client device includes sending a plurality of adaptive bitrate (ABR) renditions of the media stream to the client device, each of the plurality of ABR renditions including the interactive content.

11. The method of claim 1, wherein sending the combined media stream to the client device includes sending a plurality of adaptive bitrate (ABR) renditions of the media stream to the client device, wherein at least one of the plurality of ABR renditions includes the interactive content, and wherein at least one of the plurality of ABR renditions does not include the interactive content.

12. An apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, cause the processor to perform operations comprising;

receiving a request for a media stream from a client device;

initiating sending via a streaming session, from the processor, a combined media stream to the client device, the combined media stream including the requested media stream and interactive content, wherein the interactive content is configured to cause a media player of the client device to display the interactive content concurrently with the requested media stream during the streaming session, wherein the combined media stream comprises data packets that include video, audio, or both of the media stream, and wherein the interactive content is included in headers of the data packets;

receiving, during the display of the interactive content concurrently with the requested media stream during the streaming session, a user selection corresponding to the interactive content; and

providing, via the streaming session, second interactive content to the client device based on the user selection, the second interactive content included in the combined media stream, wherein the second interactive content is included in headers of the data packets.

13. The apparatus of claim 12, further comprising an injector configured to inject the interactive content into the media stream to generate the combined media stream.

14. The apparatus of claim 12, further comprising a network interface configured to:

receive the request;

request the interactive content from an interactive content provider; and

send the combined media stream.

15. The apparatus of claim 12, wherein the operations further comprise selecting the interactive content based on content provider preferences, user preferences, interactions with received interactive content, or a combination thereof.

16. A method comprising:

requesting, at a client device, a media stream from a media server;

receiving the requested media stream from the media server via a streaming session;

receiving interactive content associated with the requested media stream during the streaming session, wherein the requested media stream comprises data packets that include video, audio, or both, and wherein the interactive content is included in headers of the data packets;

executing the interactive content to display, via a media player of the client device, the interactive content concurrently with at least a portion of the requested media stream during the streaming session;

responsive to receiving, during display of the interactive content concurrently with at least the portion of the requested media stream during the streaming session, user input corresponding to the interactive content, sending data indicating the user input to the media server;

receiving, via the streaming session, second interactive content based on the data indicating the user input, wherein the second interactive content is included in headers of the data packets; and

displaying at least a second portion of the requested media stream and the second interactive content concurrently.

17. The method of claim 16, wherein the media stream comprises a combined media stream that includes the requested media stream and the interactive content, and wherein the combined media stream is generated based on injecting the interactive content into the requested media stream as action message format (AMF) data packets.

18. The method of claim 17, wherein the interactive content is received via a side channel of the media stream.

19. The method of claim 16, wherein the interactive content comprises Hypertext Markup Language (HTML) code or Extensible Markup Language (XML) code, and wherein the interactive content is rendered in a first layer, the first layer overlying a second layer including the portion of the requested media stream.

20. The method of claim 16, further comprising installing the media player at the client device, the media player including an interactive content renderer, wherein the interactive content renderer executes the interactive content.