NETWORK STREAMING OF MEDIA DATA

• US 20130091251A1
• Filed: 10/04/2012
• Published: 04/11/2013
• Est. Priority Date: 10/05/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: WALKER, Gordon Kent, LUBY, Michael G.
• Granted Patent: US 9,843,844 B2
• Time in Patent Office: Days
• US Class Current: 709/219
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

In one example, a device includes one or more processors configured to receive a first segment of media data, wherein the media data of the first segment comprises a stream access point, receive a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and decode at least a portion of the media data of the second segment relative to at least a portion of data for the first segment. In this manner, the techniques of this disclosure may be used to achieve a Low Latency Live profile for, e.g., dynamic adaptive streaming over HTTP (DASH).

Claims

1. A method of receiving media data, the method comprising:

receiving a first segment of media data, wherein the media data of the first segment comprises a first stream access point;

receiving a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment; and

before a second stream access point, subsequent to the first stream access point, is available, decoding at least a portion of the media data of the second segment relative to at least a portion of the media data of the first segment.

2. The method of claim 1, wherein the second segment lacks a stream access point.

3. The method of claim 1, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

4. The method of claim 1, further comprising decoding at least a portion of the media data of the first segment prior to receiving the second segment.

5. The method of claim 1, wherein the coded data of the first segment corresponds to a playback duration, and wherein the coded data of the second segment corresponds to the same playback duration.

6. The method of claim 5, wherein a representation comprises a plurality of segments including the first segment and the second segment, and wherein each of the plurality of segments comprises the same playback duration.

7. The method of claim 1, wherein the coded data of the first segment corresponds to a first duration of playback time, and wherein the coded data of the second segment corresponds to a second playback duration different from the first playback duration.

8. The method of claim 1, wherein the stream access point comprises an instantaneous decoder refresh (IDR) picture of the first segment.

9. The method of claim 1, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

10. The method of claim 1, wherein the first segment comprises a first media file, and wherein the second segment comprises a second media file separate from the first media file.

11. A device for receiving media data, the device comprising one or more processors configured to receive a first segment of media data, wherein the media data of the first segment comprises a first stream access point, receive a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and before a second stream access point, subsequent to the first stream access point, is available, decode at least a portion of the media data of the second segment relative to at least a portion of data of the first segment.

12. The device of claim 11, wherein the second segment lacks a stream access point.

13. The device of claim 11, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

14. The device of claim 11, wherein the one or more processors are configured to decode at least a portion of the media data of the first segment prior to receiving the second segment.

15. The device of claim 11, wherein the coded data of the first segment corresponds to a playback duration, and wherein the coded data of the second segment corresponds to the same playback duration.

16. The device of claim 15, wherein a Representation comprises a plurality of segments comprising the first segment and the second segment, and wherein each of the plurality of segments comprises the same playback duration.

17. The device of claim 11, wherein the media data of the first segment corresponds to media data having a first playback time, and wherein the media data of the second segment corresponds to media data having a second playback time different from the first playback time.

18. The device of claim 11, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

19. A device for receiving media data, the device comprising:

means for receiving a first segment of media data, wherein the first segment comprises a first stream access point;

means for receiving a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment; and

means for decoding, before a second stream access point, subsequent to the first stream access point, is available, at least a portion of the media data of the second segment relative to at least a portion of the media data of the first segment.

20. The device of claim 19, wherein the second segment lacks a stream access point.

21. The device of claim 19, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

22. The device of claim 19, further comprising means for decoding at least a portion of the media data of the first segment prior to receiving the second segment.

23. The device of claim 19, wherein the coded data of the first segment corresponds to a playback duration, and wherein the coded data of the second segment corresponds to the same playback duration.

24. The device of claim 19, wherein the media data of the first segment corresponds to media data having a first playback time, and wherein the media data of the second segment corresponds to media data having a second playback time different from the first playback time.

25. The device of claim 19, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

26. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

receive a first segment of media data, wherein the media data of the first segment comprises a first stream access point;

receive a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment; and

before a second stream access point, subsequent to the first stream access point, is available, decode at least a portion of the media data of the second segment relative to at least a portion of the media data of the first segment.

27. The computer-readable storage medium of claim 26, wherein the second segment lacks a stream access point.

28. The computer-readable storage medium of claim 26, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

29. The computer-readable storage medium of claim 26, further comprising instructions that cause the processor to decode at least a portion of the media data of the first segment prior to receiving the second segment.

30. The computer-readable storage medium of claim 26, wherein the coded data of the first segment corresponds to a playback duration, and wherein the coded data of the second segment corresponds to the same playback duration.

31. The computer-readable storage medium of claim 26, wherein the media data of the first segment corresponds to media data having a first playback time, and wherein the media data of the second segment corresponds to media data having a second playback time different from the first playback time.

32. The computer-readable storage medium of claim 26, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

33. A method of sending media data, the method comprising:

obtaining a first segment of media data, wherein the first segment comprises a first stream access point;

sending the first segment to a client device;

obtaining a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and wherein at least a portion of the media data of the second segment is coded relative to at least a portion of the media data of the first segment; and

before a second stream access point, subsequent to the first stream access point, is available, sending the second segment to the client device.

34. The method of claim 33, wherein the second segment lacks a stream access point.

35. The method of claim 33, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

36. The method of claim 35, further comprising determining that the second segment corresponds to a temporal position at which a stream access point is required, and treating a scene change instantaneous decoder refresh (IDR) picture, corresponding to the stream access point at the position other than the beginning of the second segment, as the required stream access point.

37. The method of claim 35, further comprising forming an ensemble of services, each of the services comprising a respective plurality of segments, wherein one of the services comprises a plurality of segments comprising the first and second segments, wherein forming the ensemble of services comprises:

forming one of the services to comprise a first service, wherein a first SAP of a first segment of the first service has a first presentation time, wherein a second SAP of a second segment of the first service has a second presentation time, and wherein the first segment and the second segment are consecutive within the first service; and

forming one or more of the services to comprise respective plurality of services, separate from the first service, such that, for presentation times between the first presentation time and the second presentation time, the plurality of services include SAPs having presentation times that are between the first presentation time and the second presentation time, different from the first presentation time, and different from presentation times of other SAPs of other services of the plurality of services.

38. The method of claim 37, wherein when segments of the plurality of services including the SAPs having the presentation times that are between the first presentation time and the second presentation time do not include scene change instantaneous decoder refresh (IDR) pictures, forming first service comprises forming the first segment to have a length that is a multiple of lengths of the segments of the plurality of services, wherein the multiple is greater than one.

39. The method of claim 38, wherein the multiple is greater than or equal to the number of the plurality of services.

40. The method of claim 37, wherein segments of the plurality of services have a first length, and wherein the first segment of the first service has a length equal to the number of plurality of services times the firth length, plus the first length.

41. The method of claim 37, wherein the ensemble of services comprises a first subset of a full ensemble of services, and wherein a second subset of the full ensemble of services includes plurality of services including SAPs at presentation times of at least some of the presentation times of the plurality of services of the first subset.

42. The method of claim 37, further comprising:

determining whether the ensemble of services includes a number of services that exceeds a threshold number of services; and

when the number of services exceeds the threshold number of services, splitting the ensemble into two or more separate ensembles of services.

43. The method of claim 37, wherein forming the plurality of services comprises inserting the SAPs in a manner substantially conforming to a round-robin insertion, and when a first one of the plurality of services includes a scene change instantaneous decoder refresh (IDR) picture having a presentation time other than a presentation time corresponding to the round-robin insertion, inserting a SAP of a second one of the plurality of services at the presentation time corresponding to the round robin insertion for the first one of the plurality of services.

44. The method of claim 33, wherein sending the first segment comprises sending the first segment before obtaining the second segment.

45. The method of claim 33, further comprising forming a Representation conforming to a low latency live profile of dynamic adaptive streaming over HTTP (DASH) such that the representation includes the first segment and the second segment.

46. The method of claim 33, wherein the coded data for the first segment corresponds to a playback duration, and wherein the coded data for the second segment corresponds to the same playback duration.

47. The method of claim 33, wherein a Representation comprises a plurality of segments comprising the first segment and the second segment.

48. The method of claim 33, wherein obtaining the first segment comprises receiving the first segment from a content preparation device, and wherein obtaining the second segment comprises receiving the second segment from the content preparation device.

49. The method of claim 33, wherein obtaining the first segment comprises receiving media data for the first segment in real time during a live recording.

50. The method of claim 33, wherein obtaining the first segment comprises encoding media data for the first segment, and wherein obtaining the second segment comprises encoding media data for the second segment.

51. The method of claim 50, wherein encoding the media data for the second segment comprises encoding at least a portion of the media data for the second segment relative to data for the first segment.

52. The method of claim 33, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

53. The method of claim 33, wherein the first segment comprises a first media file, and wherein the second segment comprises a second media file separate from the first media file.

54. A device for sending media data, the device comprising one or more processors configured to obtain a first segment of media data, wherein the media data of the first segment comprises a first stream access point, send the first segment to a client device, obtain a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and wherein at least a portion of the media data of the second segment is coded relative to at least a portion of the media data of the first segment, and before a second stream access point, subsequent to the first stream access point, is available, send the second segment to the client device.

55. The device of claim 54, wherein the second segment lacks a stream access point.

56. The device of claim 54, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

57. The device of claim 54, wherein the one or more processors are configured to send the first segment before obtaining the second segment.

58. The device of claim 54, wherein the coded data for the first segment corresponds to a playback duration, and wherein the coded data for the second segment corresponds to the same playback duration.

59. The device of claim 54, wherein a Representation comprises a plurality of segments comprising the first segment and the second segment.

60. The device of claim 54, wherein the one or more processors are configured to encode media data for the first segment and to encode media data for the second segment relative to data for the first segment.

61. The device of claim 54, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

62. A device for sending media data, the device comprising:

means for obtaining a first segment of media data, wherein the media data of the first segment comprises a first stream access point;

means for sending the first segment to a client device;

means for obtaining a second segment of the media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and wherein at least a portion of the media data of the second segment is coded relative to at least a portion of the media data of the first segment; and

means for sending, before a second stream access point, subsequent to the first stream access point, is available, the second segment to the client device.

63. The device of claim 62, wherein the second segment lacks a stream access point.

64. The device of claim 62, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

65. The device of claim 62, wherein the means for sending the first segment comprises means for sending the first segment before obtaining the second segment.

66. The device of claim 62, wherein a Representation comprises a plurality of segments comprising the first segment and the second segment.

67. The device of claim 62, wherein the means for obtaining the first segment comprises means for encoding media data for the first segment, and wherein the means for obtaining the second segment comprises means for encoding media data for the second segment relative to data for the first segment.

68. The device of claim 62, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.

69. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

obtain a first segment of media data, wherein the media data of the first segment comprises a stream access point;

send the first segment to a client device;

obtain a second segment of media data, wherein the media data of the second segment lacks a stream access point at the beginning of the second segment, and wherein at least a portion of the media data of the second segment is coded relative to at least a portion of the media data of the first segment; and

before a second stream access point, subsequent to the first stream access point, is available, send the second segment to the client device.

70. The computer-readable storage medium of claim 69, wherein the second segment lacks a stream access point.

71. The computer-readable storage medium of claim 69, wherein the second segment includes a third stream access point at a position other than the beginning of the second segment, and wherein the second stream access point is subsequent to the third stream access point.

72. The computer-readable storage medium of claim 69, wherein the instructions that cause the processor to send the first segment comprise instructions that cause the processor to send the first segment before obtaining the second segment.

73. The computer-readable storage medium of claim 69, wherein a Representation, comprises a plurality of segments comprising the first segment and the second segment.

74. The computer-readable storage medium of claim 69, wherein the instructions that cause the processor to obtain the first segment comprise instructions that cause the processor to encode media data for the first segment, and wherein the instructions that cause the processor to obtain the second segment comprise instructions that cause the processor to encode media data for the second segment relative to data for the first segment.

75. The computer-readable storage medium of claim 69, wherein the first segment comprises a first dynamic adaptive streaming over HTTP (DASH) segment, and wherein the second segment comprises a second DASH segment different from the first DASH segment.