Syntax-aware manipulation of media files in a container format

• US 9,262,419 B2
• Filed: 04/05/2013
• Issued: 02/16/2016
• Est. Priority Date: 04/05/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Dalal, Firoz, Wu, Yongjun
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/857,975
• Publication Number:
  US 20140304303A1
• Time in Patent Office: 1,047 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 27/36 Monitoring, i.e. supervising the progress of recording or reproducing for digital recording G11B20/00 and s.gr.; for monitoring, testing or measuring of TV recorders of the type covered by H04N5/76 and subgroups, see H04N17/06

Abstract

A container format processing tool performs syntax-aware manipulation of hierarchically organized syntax elements defined according to a container format in a media file. For example, a container format verifier checks conformance of a media file to a container format, which can help ensure interoperability between diverse sources of media content and playback equipment. Conformance verification can include verification of individual syntax elements, cross-verification, verification that any mandatory syntax elements are present and/or verification of synchronization. Or, a container format “fuzzer” simulates corruption of a media file, which can help test the resilience of playback equipment to errors in the media files. The container format fuzzer can simulate random bit flipping errors, an audio recording failure or incorrect termination of recording. Or, a container format editor can otherwise edit the media file in the container format.

Claims

1. One or more computer-readable media storing computer-executable instructions for causing a computing system programmed thereby to perform operations comprising:

receiving a media file in a container format for a presentation that includes one or more of audio content, image content and video content, wherein the container format is tree-structured such that the media file includes hierarchically organized syntax elements defined according to the container format; and

performing syntax-aware manipulation of at least some of the hierarchically organized syntax elements defined according to the container format in the media file, including verifying conformance of the media file to the container format, wherein the verifying includes cross-verification for the at least some of the hierarchically organized syntax elements defined according to the container format, the cross-verification including, for a given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that a given one of duration, count of frames, sample size, file size, box size, or chunk offset is consistent between a value of the given syntax element and a value of each of one or more other syntax elements, respectively, of the hierarchically organized syntax elements defined according to the container format.

2. The one or more computer-readable media of claim 1 wherein the verifying further includes single-element verification for the at least some of the hierarchically organized syntax elements defined according to the container format, including, for the given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that the value of the given syntax element satisfies one or more constraints for the given syntax element.

3. The one or more computer-readable media of claim 1 wherein the verifying further includes checking synchronization between audio samples and video samples in the presentation, including checking that a timestamp for one of the audio samples is sufficiently close to a timestamp for a corresponding one of the video samples.

4. The one or more computer-readable media of claim 1 wherein the verifying further includes checking for presence of any mandatory syntax elements among the hierarchically organized syntax elements defined according to the container format.

5. The one or more computer-readable media of claim 1 wherein the performing syntax-aware manipulation further includes simulating corruption of the media file in the container format.

6. The one or more computer-readable media of claim 5 wherein the simulating corruption includes replacing valid values with other values for one or more of the at least some of the hierarchically organized syntax elements defined according to the container format, while preserving file format structure for the media file in the container format.

7. The one or more computer-readable media of claim 5 wherein the simulating corruption includes, for the given syntax element of the hierarchically organized syntax elements defined according to the container format, simulating a random bit flip by altering the value of the given syntax element.

8. The one or more computer-readable media of claim 5 wherein the simulating corruption includes simulating an audio recording failure by removing audio data from an audio track of the media file in the container format.

9. The one or more computer-readable media of claim 5 wherein the simulating corruption includes simulating incorrect termination of recording by adding extraneous syntax elements defined according to the container format to an object of the media file in the container format.

10. The one or more computer-readable media of claim 1 wherein the performing syntax-aware manipulation further includes:

detecting one or more corrupted values among the hierarchically organized syntax elements defined according to the container format; and

correcting the one or more corrupted values.

11. The one or more computer-readable media of claim 1 wherein the performing syntax-aware manipulation further includes inserting new hierarchically organized syntax elements defined according to the container format into the media file in the container format to model a problem with another media file in the container format.

12. The one or more computer-readable media of claim 1 wherein the hierarchically organized syntax elements defined according to the container format are objects and attributes of objects that are nested in a hierarchy according to the container format.

13. In a computing system, a method comprising:

receiving a media file in a container format for a presentation that includes one or more of audio content, image content and video content, wherein the container format is tree-structured such that the media file includes hierarchically organized syntax elements defined according to the container format; and

performing syntax-aware manipulation of at least some of the hierarchically organized syntax elements defined according to the container format in the media file, wherein the syntax-aware manipulation includes verifying conformance of the media file to the container format by;

checking for presence of any mandatory syntax elements among the hierarchically organized syntax elements defined according to the container format;

performing single-element verification for each of the at least some of the hierarchically organized syntax elements defined according to the container format; and

performing cross-verification for some of the at least some of the hierarchically organized syntax elements defined according to the container format, the cross-verification including, for a given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that a given one of duration, count of frames, sample size, file size, box size, or chunk offset is consistent between a value of the given syntax element and a value of each of one or more other syntax elements of the hierarchically organized syntax elements, respectively, defined according to the container format.

14. In a computing system, a method comprising:

receiving a media file in a container format for a presentation that includes one or more of audio content, image content and video content, wherein the container format is tree-structured such that the media file includes hierarchically organized syntax elements defined according to the container format; and

performing syntax-aware manipulation of at least some of the hierarchically organized syntax elements defined according to the container format in the media file, including verifying conformance of the media file to the container format, wherein the verifying includes cross-verification for the at least some of the hierarchically organized syntax elements defined according to the container format, the cross-verification including, for a given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that a given one of duration, count of frames, sample size, file size, box size, or chunk offset is consistent between a value of the given syntax element and a value of each of one or more other syntax elements, respectively, of the hierarchically organized syntax elements defined according to the container format.

15. The method of claim 14 wherein the verifying further includes single-element verification for the at least some of the hierarchically organized syntax elements defined according to the container format, including, for the given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that the value of the given syntax element satisfies one or more constraints for the given syntax element.

16. The method of claim 14 wherein the verifying further includes checking synchronization between audio samples and video samples in the presentation, including checking that a timestamp for one of the audio samples is sufficiently close to a timestamp for a corresponding one of the video samples.

17. The method of claim 14 wherein the performing syntax-aware manipulation further includes simulating corruption of the media file in the container format.

18. A computing system comprising:

a buffer for storing a media file in a container format for a presentation that includes one or more of audio content, image content and video content, wherein the container format is tree-structured such that the media file includes hierarchically organized syntax elements defined according to the container format; and

a syntax manipulation tool configured to perform syntax-aware manipulation of at least some of the hierarchically organized syntax elements defined according to the container format in the media file, wherein the syntax manipulation tool is configured to perform conformance verification of the media file to the container format, wherein the conformance verification includes cross-verification for the at least some of the hierarchically organized syntax elements defined according to the container format, the cross-verification including, for a given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that a given one of duration, count of frames, sample size, file size, box size, or chunk offset is consistent between a value of the given syntax element and a value of each of one or more other syntax elements, respectively, of the hierarchically organized syntax elements defined according to the container format.

19. The computing system of claim 18 wherein the conformance verification further includes single-element verification for the at least some of the hierarchically organized syntax elements defined according to the container format, including, for the given syntax element of the hierarchically organized syntax elements defined according to the container format, checking that the value of the given syntax element satisfies one or more constraints for the given syntax element.

20. The computing system of claim 18 wherein the conformance verification further includes checking synchronization between audio samples and video samples in the presentation, including checking that a timestamp for one of the audio samples is sufficiently close to a timestamp for a corresponding one of the video samples.

21. The computing system of claim 18 wherein the syntax-aware manipulation further includes simulating corruption of the media file in the container format.

22. The computing system of claim 18 wherein the hierarchically organized syntax elements defined according to the container format are objects and attributes of objects that are nested in a hierarchy according to the container format.