Segmented metadata and indexes for streamed multimedia data

• US 9,852,219 B2
• Filed: 08/19/2008
• Issued: 12/26/2017
• Est. Priority Date: 08/20/2007
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Hannuksela, Miska M.
• Primary Examiner(s): Uddin, Mohammed R
• Application Number:
  US12/194,496
• Publication Number:
  US 20090055417A1
• Time in Patent Office: 3,416 Days
• Field of Search: 707802, 707999107, 707693, 707700, 707754
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/908 using metadata automatically derived from the content
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format

Abstract

A method of organizing streamed data includes storing streamed data in a file, identifying metadata applicable to a subset of the streamed data, and forming at least one group of one or more samples of the streamed data, each sample in a group having identical metadata content for a metadata type. The file may be in accordance with ISO base media file format. The storing streamed data in a file may include storing in a reception hint track. The at least one group may be indicated in a sample group description box. The metadata type may be indicated by a grouping type and grouping instance data, the grouping type specifying semantics of the grouping instance data and the metadata content. The metadata content may comprise a metadata payload and zero or more metadata payload extensions, where the metadata payload is included in a first structure and the zero or more metadata payload extensions are included in a second structure.

Claims

1. A method comprising:

causing streamed data to be stored in a file, wherein the file consists of media data and metadata enclosed separately, wherein causing the streamed data to be stored in the file includes storing in a reception hint track;

identifying metadata applicable to two or more samples of the streamed data;

causing at least one timed metadata track to be created based on the identified metadata, the at least one timed metadata track describing a referred media track and the reception hint track, wherein the hint track refers to samples comprising instructions for constructing packets for transmission over an indicated communication protocol, wherein the media track refers to samples formatted according to a media compression format;

forming at least one group from the two or more samples of the streamed data, each sample in a group having identical metadata content for a metadata type;

selecting each sample to group box associated with the reception hint track and the media track; and

finding a sample group description index of a particular reception hint sample or a media sample.

2. The method of claim 1, further comprising causing the at least one group to be identified in a file.

3. The method of claim 1, wherein the file is in accordance with International Organization for Standardization (ISO) base media file format.

4. The method of claim 3, wherein the metadata type is indicated by a grouping type and grouping instance data, the grouping type specifying semantics of the grouping instance data and the metadata content.

5. The method of claim 4, wherein the metadata content comprises a metadata payload and zero or more metadata payload extensions, wherein the metadata payload is included in a first structure, and wherein the zero or more metadata payload extensions are included in a second structure.

6. The method of claim 4, wherein the at least one group is indicated in a sample group description box.

7. The method of claim 3, wherein the at least one group is indicated in a sample group description box.

8. The method of claim 7, wherein the at least one group of one or more samples of the streamed data is indicated in a sample group description box for the timed metadata track through associating the at least one group of one or more samples of the streamed data with respective timed metadata samples in the timed metadata track.

9. The method of claim 1, wherein the metadata type includes titles of segments of the streamed data.

10. The method of claim 1, wherein a group includes samples from two or more tracks.

11. The method of claim 10, further comprising:

identifying the group including samples from two or more tracks with metadata associated with all related tracks.

12. The method of claim 10, further comprising:

identifying the group including samples from two or more tracks with metadata associated with a master track.

13. The method of claim 1, wherein timed metadata track associates timing metadata with one or more timelines.

14. The method of claim 13, wherein timed metadata track associates timing metadata with two or more timelines.

15. The method of claim 1, wherein causing the at least one timed metadata track to be created comprises causing at least two timed metadata tracks to be created.

16. The method of claim 15, wherein causing at least two timed metadata tracks to be created comprises causing a media-specific timed metadata track to be created and causing a program-specific metadata track to be created.

17. A computer program product comprising a non-transitory computer-readable medium storing program code portions therein, the program code portions being configured to, upon execution, cause an apparatus to at least:

cause streamed data to be stored in a file, wherein the file consists of media data and metadata enclosed separately, wherein causing the streamed data to be stored in a file includes storing in a reception hint track;

identifying metadata applicable to two or more samples of the streamed data;

cause at least one timed metadata track to be created based on the identified metadata, the at least one timed metadata track describing a referred media track and the reception hint track, wherein the hint track refers to samples comprising instructions for constructing packets for transmission over an indicated communication protocol, wherein the media track refers to samples formatted according to a media compression format; and

form at least one group from the two or more samples of the streamed data, each sample in a group having identical metadata content for a metadata type;

select each sample to group box associated with the reception hint track and the media track; and

find a sample group description index of a particular reception hint sample or media sample.

18. The computer program product of claim 17, wherein the program code portions are further configured to, upon execution, cause the apparatus to cause the at least one group to be identified in a file.

19. The computer program product of claim 17, wherein the file is in accordance with International Organization for Standardization (ISO) base media file format.

20. The computer program product of claim 19, wherein the metadata type is indicated by a grouping type and grouping instance data, the grouping type specifying semantics of the grouping instance data and the metadata content.

21. The computer program product of claim 20, wherein the metadata content comprises a metadata payload and zero or more metadata payload extensions, wherein the metadata payload is included in a first structure, and wherein the zero or more metadata payload extensions are included in a second structure.

22. The computer program product of claim 19, wherein the at least one group is indicated in a sample group description box.

23. The computer program product of claim 22, wherein the at least one group of one or more samples of the streamed data is indicated in a sample group description box for the timed metadata track through associating the at least one group of one or more samples of the streamed data with respective timed metadata samples in the timed metadata track.

24. The computer program product of claim 17, wherein the metadata type includes titles of segments of the streamed data.

25. The computer program product of claim 17, wherein a group includes samples from two or more tracks.

26. The computer program product of claim 25, wherein the program code portions are further configured to, upon execution, cause the apparatus to:

identify the group including samples from two or more tracks with metadata associated with all related tracks.

27. The computer program product of claim 25, wherein the program code portions are further configured to, upon execution, cause the apparatus to:

identify the group including samples from two or more tracks with metadata associated with a master track.

28. The computer program product of claim 17, wherein timed metadata track associates timing metadata with one or more timelines.

29. The computer program product of claim 28, wherein timed metadata track associates timing metadata with two or more timelines.

30. The computer program product of claim 17, wherein the program code portions are further configured to, upon execution, cause the apparatus to cause the at least one timed metadata track to be created by causing at least two timed metadata tracks to be created.

31. The computer program product of claim 30, wherein the program code portions are further configured to, upon execution, cause the apparatus to cause the at least two timed metadata tracks to be created by causing a media-specific timed metadata track to be created and causing a program-specific metadata track to be created.

32. An apparatus comprising a processor and a memory storing program code, the memory and program code being configured to, with the processor, cause the apparatus to at least:

cause streamed data to be stored in a file, wherein the file consists of media data and metadata enclosed separately, wherein causing the streamed data to be stored in a file includes storing in a reception hint track;

identifying metadata applicable to two or more samples of the streamed data;

cause at least one timed metadata track to be created based on the identified metadata, the at least one timed metadata track describing a referred media track and the reception hint track, wherein the hint track refers to samples comprising instructions for constructing packets for transmission over an indicated communication protocol, wherein the media track refers to samples formatted according to a media compression format;

form at least one group from the two or more samples of the streamed data, each sample in a group having identical metadata content for a metadata type;

select each sample to group box associated with the reception hint track and the media track; and

find a sample group description index of a particular reception hint sample or media sample.

33. The apparatus of claim 32, wherein the apparatus is further caused to identify the at least one group in a file stored in the memory.

34. The apparatus of claim 32, wherein the file is in accordance with International Organization for Standardization (ISO) base media file format.

35. The apparatus of claim 34, wherein the metadata type is indicated by a grouping type and grouping instance data, the grouping type specifying semantics of the grouping instance data and the metadata content.

36. The apparatus of claim 35, wherein the metadata content comprises a metadata payload and zero or more metadata payload extensions, wherein the metadata payload is included in a first structure, and wherein the zero or more metadata payload extensions are included in a second structure.

37. The apparatus of claim 34, wherein the at least one group is indicated in a sample group description box.

38. The apparatus of claim 37, wherein the at least one group of one or more samples of the streamed data is indicated in a sample group description box for the timed metadata track through associating the at least one group of one or more samples of the streamed data with respective timed metadata samples in the timed metadata track.

39. The apparatus of claim 32, wherein the metadata type includes titles of segments of the streamed data.

40. The apparatus of claim 32, wherein a group includes samples from two or more tracks.

41. The apparatus of claim 40, wherein the apparatus is further caused to identify the group including samples from two or more tracks with metadata associated with a master track.

42. The apparatus of claim 32, wherein the apparatus is further caused to identify the group including samples from two or more tracks with metadata associated with all related tracks.

43. The apparatus of claim 32, wherein timed metadata track associates timing metadata with one or more timelines.

44. The apparatus of claim 43, wherein timed metadata track associates timing metadata with two or more timelines.

45. The apparatus of claim 32, wherein the apparatus is caused to cause the at least one timed metadata track to be created by causing at least two timed metadata tracks to be created.

46. The apparatus of claim 45, wherein the apparatus is caused to cause at least two timed metadata tracks to be created by causing a media-specific timed metadata track to be created and causing a program-specific metadata track to be created.

47. An apparatus, comprising:

means for causing streamed data to be stored in a file, wherein the file consists of media data and metadata enclosed separately, wherein causing the streamed data to be stored a file includes storing a reception hint track;

means for identifying metadata applicable to two or more samples of the streamed data;

means for causing at least one timed metadata track to be created based on the identified metadata, the at least one timed metadata track describing a referred media track and the reception hint track, wherein the hint track refers to samples comprising instructions for constructing packets for transmission over an indicated communication protocol, wherein the media track refers to samples formatted according to a media compression format;

means for forming at least one group from the two or more samples of the streamed data, each sample in a group having identical metadata content for a metadata type;

means for selecting each sample to group box associated with the reception hint track and the media track; and

means for finding a sample group description index of a particular reception hint sample or media sample.

48. The apparatus of claim 47, wherein the file is in accordance with International Organization for Standardization (ISO) base media file format.

49. The apparatus of claim 48, wherein the metadata type is indicated by a grouping type and grouping instance data, the grouping type specifying semantics of the grouping instance data and the metadata content.

50. The apparatus of claim 49, wherein the metadata content comprises a metadata payload and zero or more metadata payload extensions, wherein the metadata payload is included in a first structure, and wherein the zero or more metadata payload extensions are included in a second structure.

51. The apparatus of claim 48, wherein the at least one group is indicated in a sample group description box.