System and method for modifying media streams using metadata

• US 10,250,841 B2
• Filed: 06/22/2012
• Issued: 04/02/2019
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: TiVo Corporation (Adeia Inc.)
• Original Assignee: MobiTV Incorporated (Adeia Inc.)
• Inventors: Stiers, Todd
• Primary Examiner(s): Robinson, Kito R
• Assistant Examiner(s): Tibljas, Shacole C
• Application Number:
  US13/530,310
• Publication Number:
  US 20120263438A1
• Time in Patent Office: 2,475 Days
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal
• CPC Class Codes: H04N 9/8211 the additional signal being a sound signal H04N9/835 takes precedence
• CPC Class Codes: H04N 9/8233 the additional signal being a character code signal

Abstract

Real-time metadata tracks recorded to media streams allow search and analysis operations in a variety of contexts. Search queries can be performed using information in real-time metadata tracks such as closed captioning, sub-title, statistical tracks, miscellaneous data tracks. Media streams can also be augmented with additional tracks. The metadata tracks not only allow efficient searching and indexing, but also allow insertion of content specific advertising during appropriate portions of a media stream based on the content of the metadata tracks.

Claims

1. A method comprising:

selecting, via a controller, a content server among a plurality of content servers, the content server being geographically close to a mobile device, wherein the controller is configured to perform session management using RTSP (Real Time Streaming Protocol) protocol such that streaming functions, performed by the content server, and session management functions, performed by the controller, are separated, wherein the controller establishes the sessions directly with mobile devices, instead of the content server connecting directly with mobile devices, such that more mobile devices can operate simultaneously while being directly connected with the controller than if the mobile devices were directly connected to the content servers;

establishing, via the controller, a media streaming session between the content server and the mobile device;

obtaining device information from the mobile device by the content server;

streaming a media stream from the content server to the mobile device using RTP (Real Time Transport Protocol) protocol, the media stream including a video track, an audio track, and a metadata track;

analyzing metadata content of the media stream;

indexing metadata content in the metadata track to allow for later retrieval of the metadata content and corresponding video content in the video track, wherein the indexing is performed via entering the metadata content into a search engine;

selecting targeted advertising for the media stream using closed captioning contents in the metadata track, wherein the advertising is maintained in a database;

modifying, by the content server, the media stream using the device information to include selected advertising, wherein modifying includes;

maintaining a current sequence number for the media stream,

removing RTP packets with sequence numbers subsequent to the current sequence number from the media stream,

inserting new RTP packets with modified sequence numbers subsequent to the current sequence number, the new RTP packets corresponding to the selected advertising being inserted into the media stream;

updating sequence numbers of original RTP packets of the media stream that are transmitted to the device after transmitting the inserted new RTP packets, the modified sequence numbers of the original RTP packets being subsequent to the updated sequence numbers of the new RTP packets; and

transmitting the modified media stream to the mobile device.

2. The method of claim 1, wherein advertising is selected by using closed captioning contents and editorial contents in the metadata track.

3. The method of claim 1, wherein advertising is selected by using closed captioning contents and viewer comments in the metadata track.

4. The method of claim 1, wherein the media stream is modified in real time.

5. The method of claim 1, wherein the media stream is a media clip.

6. The method of claim 1, wherein the video track and the audio track are dynamically modified to include relevant and targeted advertising.

7. The method of claim 1, wherein analyzing metadata content includes analyzing the portion of the metadata track corresponding to only the most recent 10 minutes of the media stream being presented.

8. The method of claim 1, wherein selected advertising is ordered based on priority.

9. A system comprising:

a database configured to maintain advertising;

a controller configured to;

select a content server among a plurality of content servers, the content server being geographically close to a mobile device,

perform session management using RTSP (Real Time Streaming Protocol) protocol such that streaming functions, performed by the content server, and session management functions, performed by the controller, are separated, wherein the controller establishes the sessions directly with mobile devices, instead of the content server connecting directly with mobile devices, such that more mobile devices can operate simultaneously while being directly connected with the controller than if the mobile devices were directly connected to the content servers; and

establish a media streaming session between the content server and the mobile device;

the content server connected to the database and the controller, the content server configured to;

obtain device information from the mobile device;

stream a media stream to the mobile device using RTP (Real Time Transport Protocol) protocol, the media stream including a video track, an audio track, and a metadata track;

analyze metadata content of the media stream;

index metadata content in the metadata track to allow for later retrieval of the metadata content and corresponding video content in the video track, wherein the indexing is performed via entering the metadata content into a search engine;

select targeted advertising for the media stream using the metadata content in the metadata track, wherein the advertising is maintained in a database;

modify the media stream using the device information to include selected advertising, wherein modifying includes;

maintaining a current sequence number for the media stream,

removing RTP packets with sequence numbers subsequent to the current sequence number from the media stream,

inserting new RTP packets with modified sequence numbers subsequent to the current sequence number, the new RTP packets corresponding to the selected advertising being inserted into the media stream;

updating sequence numbers of original RTP packets of the media stream that are transmitted to the device after transmitting the inserted new RTP packets, the modified sequence numbers of the original RTP packets being subsequent to the updated sequence numbers of the new RTP packets; and

transmit the modified media stream to the mobile device.

10. The system of claim 9, wherein advertising is selected by using closed captioning contents and editorial contents in the metadata track.

11. The system of claim 9, wherein advertising is selected by using closed captioning contents and viewer comments in the metadata track.

12. The system of claim 9, wherein the media stream is recorded in real time.

13. The system of claim 9, wherein the media stream is a media clip.

14. The system of claim 9, wherein the video track and the audio track are dynamically modified to include relevant and targeted advertising.

15. The system of claim 9, wherein analyzing metadata content includes analyzing the portion of the metadata track corresponding to only the most recent 10 minutes of the media stream being presented.

16. The system of claim 9, wherein selected advertising is ordered based on priority.

17. A non-transitory computer readable medium comprising instructions to execute a method, the method comprising:

selecting, via a controller, a content server among a plurality of content servers, the content server being geographically close to a mobile device, wherein the controller is configured to perform session management using RTSP (Real Time Streaming Protocol) protocol such that streaming functions, performed by the content server, and session management functions, performed by the controller, are separated, wherein the controller establishes the sessions directly with mobile devices, instead of the content server connecting directly with mobile devices, such that more mobile devices can operate simultaneously while being directly connected with the controller than if the mobile devices were directly connected to the content servers;

establishing, via the controller, a media streaming session between the content server and the mobile device;

obtaining device information from the mobile device by the content server;

streaming a media stream from the content server to the mobile device using RTP protocol, the media stream including a video track, an audio track, and a metadata track;

analyzing metadata content of the media stream;

indexing metadata content in the metadata track to allow for later retrieval of the metadata content and corresponding video content in the video track, wherein the indexing is performed via entering the metadata content into a search engine;

selecting targeted advertising for the media stream using closed captioning contents in the metadata track, wherein the advertising is maintained in a database;

modifying, by the content server, the media stream using the device information to include selected advertising, wherein modifying includes;

maintaining a current sequence number for the media stream,

removing RTP (Real Time Transport Protocol) packets with sequence numbers subsequent to the current sequence number from the media stream,

inserting new RTP packets with modified sequence numbers subsequent to the current sequence number, the new RTP packets corresponding to the selected advertising being inserted into the media stream;

updating sequence numbers of original RTP packets of the media stream that are transmitted to the device after transmitting the inserted new RTP packets, the modified sequence numbers of the original RTP packets being subsequent to the updated sequence numbers of the new RTP packets; and

transmitting the modified media stream to the mobile device.

18. The non-transitory computer readable medium of claim 17, wherein advertising is selected by using closed captioning contents and editorial contents in the metadata track.

19. The non-transitory computer readable medium of claim 17, wherein advertising is selected by using closed captioning contents and viewer comments in the metadata track.

20. The non-transitory computer readable medium of claim 17, wherein the media stream is a live media stream.