Insertion of digital media

• US 7,992,165 B1
• Filed: 02/13/2007
• Issued: 08/02/2011
• Est. Priority Date: 04/06/2006
• Status: Active Grant

• Current Assignee: Mgage LLC (Kaleyra, Inc.)
• Original Assignee: Velti USA Incorporated (Velti PLC)
• Inventors: Ludewig, Carl, Huang, Alex Xueyuan
• Primary Examiner(s): Beliveau; Scott
• Assistant Examiner(s): Huerta; Alexander Q
• Application Number:
  US11/674,570
• Time in Patent Office: 1,631 Days
• Field of Search: 725/32, 725 34- 36, 725/135, 725/136, 709/219, 705/14, 37524001-24029
• US Class Current: 725/32
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

An example embodiment of the present invention provides a process for inserting a source media stream into a target media stream. The process opens and reads into memory structural information from a target media stream. The process then opens and reads into memory structural information from a source media stream. The process adjusts the structural information for both streams in order to effect the insertion of the source media stream into the target media stream. In the example embodiment, this adjustment avoids the use of temporary output storage. The process then writes onto an output media stream the adjusted structural information for the source and target media streams and then pipes onto the output media stream the media data for the source and target media streams. In particular embodiments, a container file format is used for the source media stream, target media stream, and output media stream.

Claims

1. A method, comprising:

receiving a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

receiving a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

reading the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

reading the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

writing onto an output media stream the adjusted first and second media streams; and

piping onto the output media stream media data of the first and second adjusted media streams, wherein piping onto the output media stream media data of the first and second media streams comprises utilizing file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

2. A method as in claim 1, further comprising prenormalizing one or more of the first media stream and the second media stream prior to the adjusting.

3. A method as in claim 2, wherein prenormalizing comprises transcoding one or more of the first and second media streams into common compatible format.

4. A method as in claim 2, wherein prenormalizing comprises, for one or more of the first and second media streams, resequencing media samples in the stream to generate a linear sequence of media samples.

5. A method as in claim 1, wherein the method is performed by an insertion server.

6. A method as in claim 1, wherein the first media stream comprises an advertisement and the second media stream comprises content.

7. A method as in claim 1, wherein the first container file format is an MP4 file format.

8. A method as in claim 1, wherein writing onto the output media stream is concurrent with one or more of receiving the first media stream and receiving the second media stream.

9. A method as in claim 1, wherein the output media stream is transmitted to a mobile device for consumption by the mobile device.

10. A method as in claim 1, wherein adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream comprises one or more of:

adjusting the duration in the second stream header to be a sum of an original duration of the first media stream and an original duration of the second media stream;

launching a nested for-loop that iterates over each first track in the first media stream;

launching a nested for-loop that iterates over each second track in the second media stream;

adjusting the duration in each of one or more of the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers to account for the new media length;

adjusting the sample count in the time-to-sample structure in one or more of the first and second sample tables;

adding additional samples to the sync-sample structure in one or more of the first and second sample tables;

adjusting the sample-to-chunk structure in one or more of the first and second sample tables to account for the additional samples;

adding the additional samples to the sample size structure in one or more of the first and second sample tables;

adjusting the chunk offset structure in one or more of the first and second sample tables to account for the additional samples and adding the new chunks to the structure;

recalculating the size of each of one or more of the first track headers, second track headers, first edit lists, second edit lists, first media headers, second media headers, first sample tables, and second sample tables before the first or second nested for-loops end;

recalculating a total size of all of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables; and

readjusting a chunk-to-offset table based on the recalculation of the total size.

11. The method of claim 1, wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

12. The method of claim 1, wherein piping onto the output media stream media data of the first and second adjusted media streams comprises:

concurrently reading the media data from the first adjusted media stream and the second adjusted media stream and writing the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.

13. An apparatus, comprising logic encoded in one or more persistent storage media for execution and when executed operable to:

receive a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

receive a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

read the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

read the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

adjust one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in both streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

write onto an output media stream the adjusted first and second streams; and

pipe onto the output media stream media data of the first and second adjusted media streams, wherein the logic operable to pipe onto the output media stream media data of the first and second media streams comprises logic operable to utilize file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

14. An apparatus as in claim 13, wherein the logic is further operable to prenormalize one or more of the first media stream and the second media stream.

15. An apparatus as in claim 13, wherein the encoded logic is executed by an insertion server.

16. An apparatus as in claim 13, wherein the first media stream comprises an advertisement and the second media stream comprises content.

17. An apparatus as in claim 13, wherein the first container file format is an MP4 file format.

18. An apparatus as in claim 13, wherein the logic operable to write onto the output media stream, receive the first media stream, and receive the second media stream is operable to write onto the output media stream concurrently with one or more of receiving the first media stream and receiving the second media stream.

19. An apparatus as in claim 13, wherein the logic operable to adjust one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream comprises logic operable to perform one or more of:

adjust the duration in the second stream header to be a sum of an original duration of the first media stream and an original duration of the second media stream;

launch a nested for-loop that iterates over each first track in the first media stream;

launch a nested for-loop that iterates over each second track in the second media stream;

adjust the duration in each of one or more of the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers to account for the new media length;

adjust the sample count in the time-to-sample structure in one or more of the first and second sample tables;

add additional samples to the sync-sample structure in one or more of the first and second sample tables;

adjust the sample-to-chunk structure in one or more of the first and second sample tables to account for the additional samples;

add the additional samples to the sample size structure in one or more of the first and second sample tables;

adjust the chunk offset structure in one or more of the first and second sample tables to account for the additional samples and adding the new chunks to the structure;

recalculate the size of each of one or more of the first track headers, second track headers, first edit lists, second edit lists, first media headers, second media headers, first sample tables, and second sample tables before the first or second nested for-loops end;

recalculate a total size of all of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables; and

readjust a chunk-to-offset table based on the recalculation of the total size.

20. The apparatus of claim 13, wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

21. The apparatus of claim 13, wherein pipe onto the output media stream media data of the first and second adjusted media streams comprises:

concurrently read the media data from the first adjusted media stream and the second adjusted media stream and write the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.

22. An apparatus, comprising:

means for receiving a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

means for receiving a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

means for reading the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

means for reading the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

means for adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in both streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

means for writing onto an output media stream the adjusted first and second media streams; and

means for piping onto the output media stream media data of the first and second adjusted media streams, wherein the means for piping onto the output media stream media data of the first and second media streams comprise means for utilizing file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

23. The apparatus of claim 22, wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

24. The apparatus of claim 22, wherein means for piping onto the output media stream media data of the first and second adjusted media streams comprises:

means for concurrently reading the media data from the first adjusted media stream and the second adjusted media stream and writing the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.