Efficient real-time stitching of multimedia files

• US 8,533,259 B2
• Filed: 01/27/2011
• Issued: 09/10/2013
• Est. Priority Date: 01/27/2011
• Status: Active Grant

• Current Assignee: RhythmOne LLC (Nexxen International Ltd.)
• Original Assignee: Rhythm Newmediainc
• Inventors: Wolf, Amir, Roy, Sumit
• Primary Examiner(s): Moorthy, Aravind
• Assistant Examiner(s): PLOTKIN, JASON R
• Application Number:
  US13/014,867
• Publication Number:
  US 20120197966A1
• Time in Patent Office: 957 Days
• Field of Search: None
• US Class Current: 709/203
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format

Abstract

A system and methods are disclosed for serving combined media using resource metadata referencing original media data. Combined resource metadata is created from combining the metadata of two or more files for a virtual media file representing the requested combined media. The combined data can be adjusted to synch audio and video portions as needed for combining the files. The virtual media file references the original media files and serves the content on-the-fly, rather than creating a new file with the combined media, at a savings of server response time and memory consumption, while maintaining high performance.

Claims

1. A method for serving combined media using resource metadata referencing original media data, comprising:

receiving at a server from a client a request for combined media comprising a first media file and second media file;

creating a virtual media file comprising resource metadata for the requested combined media, the resource metadata of the virtual media file combining resource metadata associated with the first media file and the second media file;

determining from resource metadata associated with the first media file and the second media file whether audio length matches video length for media portions of the first and second media files;

adjusting the resource metadata of the virtual media file, responsive to a determined mismatch between audio length and video length, such that a difference between audio length and video length for the media portion of the first media file is less than a threshold value, and making no adjustment to a determined mismatch between audio length and video length for the second media file; and

transmitting a response from the server to the client comprising the virtual media file.

2. The method of claim 1, wherein the virtual media file does not include the first and second media files.

3. The method of claim 1, wherein determining whether the audio length matches the video length for the first media file comprises:

parsing the first media file for audio length and video length information.

4. The method of claim 1, wherein creating the virtual media file comprises:

merging the resource metadata associated with the second media file at a terminal end of the resource metadata associated with the first media file.

5. The method of claim 1, wherein the virtual media file provides pointers to the media portions of the first media file and the second media file.

6. The method of claim 1, wherein adjusting the resource metadata of the virtual media file comprises:

adjusting the resource metadata of the virtual media file to drop one or more audio or video samples at the end of the first media file.

7. The method of claim 1, wherein the threshold value is determined to be within range of one media sample duration.

8. The method of claim 1, wherein adjusting the resource metadata of the virtual media file comprises:

adjusting the resource metadata of the virtual media file to repeat one or more audio or video samples at the end of the first media file.

9. The method of claim 1, further comprising:

responsive to the determined mismatch between audio length and video length, determining a difference between audio length and video length for the media portion of the first media file.

10. The method of claim 1, further comprising:

in response to the received request for combined media further comprising a third media file, appending the resource metadata associated with the third media file to the resource metadata of the virtual media file;

determining from resource metadata of the virtual media file whether audio length matches video length for a combined media portion of the first and second media files; and

in response to a mismatch between audio length and video length,

adjusting the resource metadata of the virtual media file such that a difference between audio length and video length for the combined media portion of the first and second media files is less than a threshold value.

11. A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed cause at least one processor to:

receive at a server from a client a request for combined media comprising a first media file and second media file;

create a virtual media file comprising resource metadata for the requested combined media, the resource metadata of the virtual media file combining resource metadata associated with the first media file and the second media file;

determine from resource metadata associated with the first media file and the second media file whether audio length matches video length for media portions of the first and second media files

responsive to a determined mismatch between audio length and video length, adjust the resource metadata of the virtual media file such that a difference between audio length and video length for the media portion of the first media file is less than a threshold value, and make no adjustment to a determined mismatch between audio length and video length for the second media file; and

transmit a response from the server to the client comprising the virtual media file.

12. The computer-readable storage medium of claim 11, wherein the instructions to determine whether the audio length matches the video length for the first media file comprises:

instructions to parse the first media file for audio length and video length information.

13. The computer-readable storage medium of claim 11, wherein the virtual media file provides pointers to the media portions of the first media file and the second media file.

14. The computer-readable storage medium of claim 11, wherein the instructions to adjust the resource metadata of the virtual media file comprises:

instructions to adjust the resource metadata of the virtual media file to drop one or more audio or video samples at the end of the first media file.

15. The computer-readable storage medium of claim 11, wherein the threshold value is determined to be within range of one media sample duration.

16. The computer-readable storage medium of claim 11, further comprising instructions that when executed cause the at least one processor to:

in response to the received request for combined media further comprising a third media file, appending the resource metadata associated with the third media file to the resource metadata of the virtual media file;

determine from resource metadata of the virtual media file whether audio length matches video length for a combined media portion of the first and second media files; and

;

in response to a mismatch between audio length and video length

adjust the resource metadata of the virtual media file such that a difference between audio length and video length for the combined media portion of the first and second media files is less than a threshold value.

17. A computer system for serving combined media using resource metadata referencing original media data, comprising:

a web server configured for receiving at a server from a client a request for combined media comprising a first media file and second media file; and

a file engine configured for creating a virtual media file comprising resource metadata for the requested combined media, the resource metadata of the virtual media file combining resource metadata associated with the first media file and the second media file,

the file engine further configured for determining from resource metadata associated with the first media file and the second media file whether audio length matches video length for media portions of the first and second media files,

in responsive to a determined mismatch between audio length and video length, the file engine further configured for adjusting the resource metadata of the virtual media file such that a difference between audio length and video length for the media portion of the first media file is less than a threshold value, and making no adjustment to a determined mismatch between audio length and video length for the second media file, and

the web server further configured for transmitting a response from the server to the client comprising the virtual media file.

18. The computer system of claim 17, wherein adjusting the resource metadata of the virtual media file comprises:

adjusting the resource metadata of the virtual media file to drop one or more audio or video samples at the end of the first media file.

19. The computer system of claim 17, wherein the virtual media file comprises pointers to the media portions of the first and second media files.

20. A method for serving combined media using resource metadata referencing original media data, the method comprising:

receiving at a server from a client a request for combined media comprising a plurality of media files in a sequence;

creating a virtual media file comprising pointers to the media portions of each of the plurality of media files in the sequence and resource metadata for the requested combined media, the resource metadata of the virtual media file combining resource metadata associated with each of the plurality of media files in the sequence;

determining from resource metadata associated with each of the plurality of media files in the sequence whether audio length matches video length for media portion of each of the plurality of media files;

responsive to a determined mismatch between audio length and video length in a respective media file, adjusting the resource metadata of the virtual media file such that a difference between audio length and video length for the respective media file is less than a threshold value, making no adjustment to a determined mismatch between audio length and video length for a last media file in the sequence of the plurality of media files for the combined media; and

transmitting a response from the server to the client comprising the virtual media file.