Method and apparatus for supporting advanced coding formats in media files

• US 7,613,727 B2
• Filed: 02/21/2003
• Issued: 11/03/2009
• Est. Priority Date: 02/25/2002
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Tabatabai, Ali, Visharam, Mohammed Zubair, Walker, Toby
• Primary Examiner(s): Truong; Cam Y T
• Application Number:
  US10/371,434
• Publication Number:
  US 20030163781A1
• Time in Patent Office: 2,447 Days
• Field of Search: 707/1, 707/3, 707/100, 707/104.1, 707/101, 369/53.2, 370/465, 386/104, 386/126, 386/46
• US Class Current: 1/1
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8451 using Advanced Video Coding [AVC]
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Switch sample metadata defining switch sample sets associated with multimedia data is created. Each switch sample set includes samples that have identical decoding values. Further, a file associated with the multimedia data is formed. This file includes the switch sample metadata, as well as other information pertaining to the multimedia data.

Claims

1. A computerized method for creating metadata for multimedia data, the method comprising:

receiving a file with encoded multimedia data;

examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data;

identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks;

creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and

forming a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

2. The method of claim 1 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples.

3. The method of claim 1, wherein the predefined data structure is a table box containing a set of nested tables.

4. The method of claim 1 further comprising:

sending a file associated with the encoded multimedia data to a decoding system;

receiving the file associated with the encoded multimedia data at the decoding system; and

extracting, at the decoding system, the switch sample metadata from the file associated with the encoded multimedia data, the extracted switch sample metadata being subsequently used to find a replacement for a specific sample.

5. A computerized method for creating metadata for multimedia data, the method comprising:

receiving a file with encoded multimedia data;

examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data;

identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks;

creating sub-sample metadata defining a plurality of sub-samples within each sample of the encoded multimedia data;

creating parameter set metadata identifying one or more parameter sets for a plurality of portions of the encoded multimedia data;

creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and

forming a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata, the parameter set metadata, and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

6. The method of claim 5 wherein the creating sub-sample metadata comprises:

organizing the sub-sample metadata into a set of predefined data structures comprising a first data structure containing information about sub-sample sizes, a second data structure containing information about a number of sub-samples in each sample, and a third data structure containing information describing each sub-sample.

7. The method of claim 5 wherein each portion of the plurality of portions of multimedia data is any one of a sample and a sub-sample within the multimedia data.

8. The method of claim 5 wherein the creating parameter set metadata comprises:

organizing the parameter set metadata into a set of predefined data structures comprising a first data structure containing descriptive information about the one or more parameter sets and a second data structure containing information that defines associations between the one or more parameter sets and the plurality of portions of multimedia data.

9. The method of claim 5 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples.

10. The method of claim 5 wherein the predefined data structure is a table box containing a set of nested tables.

11. The method of claim 5 further comprising:

sending a file associated with the encoded multimedia data to a decoding system;

receiving the file associated with the encoded multimedia data at the decoding system; and

extracting, at the decoding system, the sub-sample metadata, the parameter set metadata, and the switch sample metadata from the file associated with the encoded multimedia data.

12. A computerized method for creating metadata for multimedia data, the method comprising:

receiving a file with encoded multimedia data;

examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data;

identifying samples in the switch sample sets that have identical decoded values while using reference samples from different tracks;

creating sub-sample metadata defining a plurality of sub-samples within each sample of multimedia data;

creating, by a computer including a processor, switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences; and

forming a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs a decoding system on finding a replacement for a specific sample.

13. The method of claim 12 wherein the creating sub-sample metadata comprises:

organizing the sub-sample metadata into a set of predefined data structures comprising a first data structure containing information about sub-sample sizes, a second data structure containing information about a number of sub-samples in each sample, and a third data structure containing information describing each sub-sample.

14. The method of claim 12 wherein the samples in each switch sample set of the plurality of switch sample sets uses different reference samples.

15. The method of claim 12 wherein the creating switch sample metadata comprises:

organizing the switch sample metadata into a predefined data structure represented as a table box containing a set of nested tables.

16. The method of claim 12 further comprising:

sending a file associated with the encoded multimedia data to a decoding system;

receiving the file associated with the encoded multimedia data at the decoding system; and

extracting, at the decoding system, the sub-sample metadata and the switch sample metadata from the file associated with the encoded multimedia data.

17. An apparatus for creating metadata for multimedia data, the apparatus comprising:

a memory to store the multimedia data;

a processor coupled to the memory;

a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

a metadata generator to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and

the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

18. The apparatus of claim 17 wherein the samples in each switch sample set of the plurality of switch sample sets use different reference samples.

19. The apparatus of claim 17 further comprising:

a metadata extractor to receive the file associated with the encoded multimedia data at a decoding system and to extract the switch sample metadata from the file associated with the encoded multimedia data; and

a media data stream processor to use the extracted switch sample metadata finding a replacement for a specific sample.

20. An apparatus for creating metadata for multimedia data, the apparatus comprising:

a memory to store the multimedia data;

a processor coupled to the memory;

a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

a metadata generator to create sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data, to create parameter set metadata identifying one or more parameter sets for a plurality of portions of the encoded multimedia data, and to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and

the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata, the parameter set metadata, and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

21. An apparatus for creating metadata for multimedia data, the apparatus comprising:

a memory to store the multimedia data;

a processor coupled to the memory;

a file creator to receive a file with encoded multimedia data, examine the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

a metadata generator to create sub-sample metadata defining a plurality of sub-samples within each sample of multimedia data, and to create switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and

the file creator further to form a file associated with the encoded multimedia data, the formed file comprising the sub-sample metadata and the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

22. An apparatus for creating metadata for multimedia data, the apparatus comprising:

means for receiving a file with encoded multimedia data, examining the encoded multimedia data to identify switch sample sets associated with the encoded multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

a memory to store instructions;

a processor to execute the instructions for performing steps;

creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, and

forming a file associated with the encoded multimedia data, the formed file comprising the switch sample metadata, wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

23. An apparatus for creating metadata for multimedia data, the apparatus comprising:

means for creating sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data;

means for creating parameter set metadata identifying one or more parameter sets for a plurality of portions of the multimedia data;

a memory to store instructions;

a processor to execute the instructions for performing steps;

forming a file associated with the multimedia data, the forming is to receive a file with encoded multimedia data, examine the multimedia data to identify switch sample sets associated with the multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing an identified sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure, and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, wherein the formed file comprises the sub-sample metadata, the parameter set metadata, and the switch sample metadata, and wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.

24. An apparatus for creating metadata for multimedia data, the apparatus comprising:

means for creating sub-sample metadata defining a plurality of sub-samples within each sample of the multimedia data;

a memory to store instructions;

a processor to execute the instructions for performing steps;

forming a file associated with multimedia data, wherein the forming is to receive a file with encoded multimedia data, examine the multimedia data to identify switch sample sets associated with the multimedia data, and to identify samples in the switch sample sets that have identical decoded values while using reference samples from different tracks; and

creating switch sample metadata defining a plurality of switch sample sets, each switch sample set of the plurality of switch sample sets containing sample, wherein the creating switch sample metadata comprises organizing the switch sample metadata into a predefined data structure and converting each repeated sequence of data within the predefined data structure into a reference to a sequence occurrence and a number of occurrences, wherein the formed file comprises the sub-sample metadata and the switch sample metadata, and wherein when the formed file is received by a decoding system, the switch sample metadata instructs the decoding system on finding a replacement for a specific sample.