METHOD AND SYSTEM FOR EFFICIENT STREAMING VIDEO DYNAMIC RATE ADAPTATION

• US 20120004960A1
• Filed: 09/15/2011
• Published: 01/05/2012
• Est. Priority Date: 03/23/2009
• Status: Abandoned Application

• Current Assignee: Telefonaktiebolaget LM Ericsson
• Original Assignee: Azuki Systems, Inc (Telefonaktiebolaget LM Ericsson)
• Inventors: Ma, Kevin J., Xu, Jianguo, Ng, Tung, Nair, Raj, Lin, Ichang, Wu, Chin-Cheng
• Application Number:
  US13/233,713
• Publication Number:
  US 20120004960A1
• Time in Patent Office: Days
• US Class Current: 705/14.4
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/0882 Utilisation of link capacity
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]

Abstract

A streaming media system employs dynamic rate adaptation. The method includes a file format compatible with legacy HTTP infrastructure to deliver media over a persistent connection. The method further includes the ability for legacy client media players to dynamically change the encoded delivery rate of the media over a persistent connection. The method provided works transparently with standard HTTP servers, requiring no modification and leverages standard media players embedded in mobile devices for seamless media delivery over wireless networks with high bandwidth fluctuations. A system is also specified for implementing a client and server in accordance with the method.

Claims

1. A method for streaming data to a client, comprising:

encoding the data into a plurality of encodings;

combining the plurality of encodings into a single file;

indexing the single file to determine respective locations of each individual encoding; and

advertising index information to a client device to enable the client to select an encoding to be retrieved and rendered at the client.

2. The method of claim 1, wherein the data is an audio/video file including a plurality of audio/video advertisements.

3. The method of claim 1, wherein the data is transcoded into different formats for different target devices.

4. The method of claim 3, further comprising transcoding each format into different bit rates while maintaining a single frame rate and resolution for all encodings.

5. The method of claim 4, wherein all encodings of a given format are combined into a single file by concatenating the different encodings in a manner appropriate for that format.

6. The method of claim 5, further comprising concatenating advertisements to the end of the file, after all of the individual encodings of the original data have been concatenated.

7. The method of claim 5, further comprising inserting padding between the concatenated encodings such that individual encodings begin on integer time boundaries.

8. The method of claim 7, further comprising using black frames and silent video for padding between concatenated encodings.

9. The method of claim 7, further comprising using interstitial advertisements for padding between concatenated encodings.

10. The method of claim 1, wherein concatenated file byte offsets are generated for the start of each encoding and advertisement.

11. The method of claim 10, further comprising generating the concatenated file byte offsets for the start of each frame within the encoding and storing the concatenated file byte offsets at fixed rate index file byte offsets within a rate map index file.

12. The method of claim 1, further comprising monitoring a TCP window occupancy to gather bandwidth availability information via a TCP window data drain rate.

13. The method of claim 12, further comprising keeping a limited trailing history of bandwidth measurements to smooth averaging.

14. The method of claim 13, further comprising using a weighted history to allow current fluctuations to take precedence, and using different weighted histories for transitioning to higher bit rates than for transitioning to lower bit rates.

15. The method of claim 12, further comprising using the average bandwidth to select new encodings when bandwidth changes by comparing the average bandwidth to set high and low bandwidth thresholds

16. The method of claim 15, further comprising filling a next request from the client, after a rate switch is determined necessary, with a different data range than what was specified by the client or would be sequential for the client, but which corresponds to the same playout time for a different encoding, as determined by the rate switch.

17. The method of claim 16, further comprising sending data using HTTP chunked transfer encoding.

18. The method of claim 17, further comprising inserting metadata information into an HTTP chunk ahead of the concatenated file data.

19. The method of claim 18, wherein the metadata information includes information about the encoding, the time position, and the concatenated file byte offset corresponding to the beginning the accompanying concatenated file data.

20. The method of claim 17, further comprising pacing the sending of HTTP chunks to limit bandwidth usage, and selecting a paced delivery rate which exceeds the bit rate of the new encoding.

21. The method of claim 1, being adapted to also rotate ad stitched media by further including:

creating a first stitched media file;

dividing the first stitched media file into static and dynamic particles;

creating a first particle index file for mapping the particles of the first stitched media file;

uploading all particles to a remote storage device;

uploading the first particle index file to the remote storage;

creating a second stitched media file;

dividing the second stitched media file into particles;

creating a second particle index file for mapping the particles of the second stitched media file;

uploading only the dynamic particles to the remote storage device; and

uploading the second particle index file to the remote storage.

22. The method of claim 21, wherein the stitched media file contains audio/video data.

23. The method of claim 22, further comprising concatenating a plurality of audio/video segments to create the stitched media file.

24. The method of claim 23, further comprising concatenating a plurality of audio/video clips to create each audio/video segment.

25. The method of claim 24, further comprising concatenating a plurality of transcodings to create each audio/video clip, wherein each transcoding contains the same audio/video content, transcoded to different bit rates.

26. The method of claim 25, further comprising classifying segments as either static content or dynamic content.

27. The method of claim 26, further comprising ordering segments such that static content precedes dynamic content.

28. The method of claim 27, further comprising adding a silence segment between the last static segment and the first dynamic segment.

29. The method of claim 21, wherein a static particle is a feature particle containing static segments plus the silence segment, wherein the static segments include feature content data.

30. The method of claim 21, wherein a single dynamic particle is an ad particle containing the dynamic segments, wherein the dynamic segments include ad content data.

31. The method of claim 30, wherein another dynamic particle is a header particle containing the stitched media file audio/video container headers, the stitched media file audio/video container headers including container header information for both the feature content data and the ad content data.

32. The method of claim 21, wherein the particle index file contains file name and version information for each particle.

33. The method of claim 21, wherein the particle index file contains stitched media file byte offset information for each clip.

34. The method of claim 21, wherein particles are uploaded to remote storage using standard protocols such as HTTP, FTP, SCP, or SFTP.

35. A server for streaming delivery of content, comprising:

a processor;

a memory; and

instructions executable by the processor for causing the server to perform a method including;

encoding the content into a plurality of encodings;

combining the plurality of encodings into a single file;

indexing the single file to determine respective locations of each individual encoding; and

advertising index information to a client to enable the client to select an encoding to be retrieved and rendered at the client.

36. The server of claim 35, wherein the data is an audio/video file including a plurality of audio/video advertisements.

37. The server of claim 35, wherein the data is transcoded into different formats for different target devices.

38. The server of claim 37, wherein the method performed by the server further includes transcoding each format into different bit rates while maintaining a single frame rate and resolution for all encodings.

39. The server of claim 38, wherein all encodings of a given format are combined into a single file by concatenating the different encodings in a manner appropriate for that format.

40. The server of claim 39, wherein the method performed by the server further includes concatenating advertisements to the end of the file, after all of the individual encodings of the original data have been concatenated.

41. The server of claim 39, wherein the method performed by the server further includes inserting padding between the concatenated encodings such that individual encodings begin on integer time boundaries.

42. The server of claim 41, wherein the method performed by the server further includes using black frames and silent video for padding between concatenated encodings.

43. The server of claim 41, wherein the method performed by the server further includes using interstitial advertisements for padding between concatenated encodings.

44. The system of claim 35, wherein concatenated file byte offsets are generated for the start of each encoding and advertisement.

45. The server of claim 44, wherein the method performed by the server further includes generating the concatenated file byte offsets for the start of each frame within the encoding and store the concatenated file byte offsets at fixed rate index file byte offsets within a rate map index file.

46. The server of claim 35, wherein the method performed by the server further includes monitoring TCP window occupancy to gather bandwidth availability information via the TCP window data drain rate.

47. The server of claim 46, wherein the method performed by the server further includes keeping a limited trailing history of bandwidth measurements to smooth averaging.

48. The server of claim 47, wherein the method performed by the server further includes using a weighted history to allow current fluctuations to take precedence.

49. The server of claim 48, wherein the method performed by the server further includes using different weighted histories for transitioning to higher bit rates than for transitioning to lower bit rates.

50. The server of claim 46, wherein the method performed by the server further includes using the average bandwidth to select new encodings when bandwidth changes by comparing the average bandwidth to set high and low bandwidth thresholds

51. The server of claim 50, wherein the method performed by the server further includes filling a next request of the client, after a rate switch is determined necessary, with a different data range than what was specified by the client or would be sequential for the client, but which corresponds to the same playout time for a different encoding, as determined by the rate switch.

52. The server of claim 51, wherein the method performed by the server further includes sending data using HTTP chunked transfer encoding.

53. The server of claim 52, wherein the method performed by the server further includes inserting metadata information into an HTTP chunk ahead of the concatenated file data.

54. The server of claim 53, wherein the metadata information includes information about the encoding, the time position, and the concatenated file byte offset corresponding to the beginning the accompanying concatenated file data.

55. The server of claim 52, wherein the method performed by the server further includes pacing the sending of HTTP chunks to limit bandwidth usage.

56. The server of claim 55, wherein the method performed by the server further includes selecting a paced delivery rate which exceeds the bit rate of the new encoding.

57. The server of claim 35, wherein the concatenated data file is encrypted.

58. The server of claim 35, wherein the instructions further include ad-rotating instructions which cause the server to perform an ad-rotating method of rotating ad stitched media, the ad-rotating method including:

creating a first stitched media file;

dividing the first stitched media file into static and dynamic particles;

creating a first particle index file for mapping the particles of the first stitched media file;

uploading all particles to a remote storage device;

uploading the first particle index file to the remote storage;

creating a second stitched media file;

dividing the second stitched media file into particles;

creating a second particle index file for mapping the particles of the second stitched media file;

uploading the dynamic particles to the remote storage device; and

uploading the second particle index file to the remote storage.

59. The server of claim 58, wherein the stitched media file contains audio/video data.

60. The server of claim 59, wherein the ad-rotating method performed by the server further includes concatenating a plurality of audio/video segments to create the stitched media file.

61. The server of claim 60, wherein concatenating a plurality of audio/video clips to create each audio/video segment.

62. The server of claim 61, wherein the ad-rotating method performed by the server further includes concatenating a plurality of transcodings to create each audio/video clip, wherein each transcoding contains the same audio/video content, transcoded to different bit rates.

63. The server of claim 61, wherein the ad-rotating method performed by the server further includes classifying segments as either static content or dynamic content.

64. The server of claim 63, wherein the ad-rotating method performed by the server further includes ordering segments such that static content precedes dynamic content.

65. The server of claim 64, wherein the ad-rotating method performed by the server further includes adding a silence segment between the last static segment and the first dynamic segment.

66. The server of claim 58, wherein a static particle is a feature particle containing the static segments plus the silence segment, wherein the static segments contain feature content data.

67. The server of claim 58, wherein a single dynamic particle is an ad particle containing the dynamic segments, wherein the dynamic segments contain ad content data.

68. The server of claim 67, wherein another dynamic particle is a header particle containing the stitched media file audio/video container headers, wherein the stitched media file audio/video container headers contain container header information for both the feature content data and the ad content data.

69. The server of claim 58, wherein the particle index file contains file name and version information for each particle.

70. The server of claim 58, wherein the particle index file contains stitched media file byte offset information for each clip.

71. The server of claim 58, wherein particles are uploaded to remote storage using standard protocols such as HTTP, FTP, SCP, or SFTP.

72. Apparatus, comprising:

one or more a processors for executing a file encoder and an HTTP streaming server, wherein the HTTP streaming server accepts requests for source media, wherein the file encoder accepts a source media file and a configuration file;

a memory for storing output of the file encoder;

first instructions executable by the processors for causing the apparatus to perform a file encoder method including;

transcoding the source file into a plurality of encodings as defined by the configuration;

encrypting the plurality of resultant encodings as defined by the configuration;

concatenating the plurality of resultant encrypted encodings into a single file as defined by the configuration;

storing the concatenated file to the memory;

creating an rate map index file for the concatenated file; and

storing the rate map index file to the memory; and

second instructions executable by the processor for causing the apparatus to perform an HTTP streaming server method including;

selecting a concatenated media file whose first encoding matches that of the request;

creating a header containing information about the current encoding and the current index into the concatenated media file;

packing the header into an HTTP chunk;

packing a segment of concatenated media file data into the HTTP chunk;

sending the HTTP chunk to a client;

estimating available bandwidth between the server and the client; and

changing encodings when bandwidth falls below or rises above a given set of thresholds.