Peer-based cloud storage for media broadcasts

• US 10,523,755 B1
• Filed: 02/17/2016
• Issued: 12/31/2019
• Est. Priority Date: 02/17/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Quach, Anderson Kaailodge, Chaboud, Matthew Liang, Jamal, Shah Pavel, Darling, Jr., Donald John, Halbach, Kyke Daniel
• Primary Examiner(s): Chea, Philip J
• Assistant Examiner(s): Ma, Wing
• Application Number:
  US15/046,342
• Time in Patent Office: 1,413 Days
• Field of Search: 709217
• CPC Class Codes: H04L 65/1063 Application servers providing network services systems providing special services to telephonic subscribers H04M3/42
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The recording of broadcast media can be performed by a network storage service even where a client is unable to provide a full copy of the media file. For example, a client might want to record a television broadcast by transmitting the content to a remote storage service. For a high definition format, however, the upload limitations of the data connection may prevent the full content from being uploaded from that client. Accordingly, clients recording the same content and using the same service can upload respective segments of the content, with each segment being provided by at least one of the clients and available for subsequent download. At least some of those clients can also store some of the segments such that those clients can act as peer sources for obtaining those segments.

Claims

1. A computer-implemented method, comprising:

receiving, at a content storage system, a respective request from each of a plurality of client devices to record a media broadcast stream;

determining a series of chunking offsets for the media broadcast stream, each chunking offset defining a respective segment of the media broadcast stream;

providing the series of chunking offsets to the plurality of clients for purposes of generating a series of chunks each containing the respective segment of the media broadcast stream, each chunk further containing a hash generated using data for the respective segment;

determining, for each chunk of the series of chunks and based at least in part upon respective capabilities of the plurality of client devices, one or more client devices of the plurality of client devices to upload the chunk to the content storage system;

storing at least one copy of each chunk received from the one or more client devices;

causing a set of manifests to be stored for the plurality of client devices, each manifest indicating information for at least one chunk of the series of chunks, the information including the respective hash value for the chunk;

receiving, from a determined client device of the plurality of client devices, a download request to download the media broadcast stream for playback on the determined client device;

causing a first chunk of the media broadcast stream, determined using the at least one manifest, to be transcoded into a more space efficient format;

causing the first chunk to be downloaded to the determined client device from the content storage system if the first chunk is not already locally stored by the determined client device;

determining, for each additional chunk of the series of chunks for the media broadcast stream that is not already locally stored by the determined client device, whether to cause the additional chunk to be provided to the determined client device from the content storage service or from another client device of the plurality of client devices storing a copy of the additional chunk, at least a subset of the client devices in the plurality functioning as peers; and

causing, for each additional chunk of the media broadcast stream, the additional chunk to be pulled from local storage or to be provided to the determined client device from the content storage service or another client device storing a copy of the additional chunk.

2. The computer-implemented method of claim 1, further comprising:

causing the series of chunks to be transcoded into a more space efficient format before causing the series of chunks to be uploaded from the determined client device.

3. The computer-implemented method of claim 1, further comprising:

receiving, at the content storage system, an initial chunk for the media broadcast stream; and

determining the series of chunking offsets based at least in part upon data from the initial chunk.

4. The computer-implemented method of claim 1, further comprising:

instructing a client device to not upload a specified chunk if a valid copy of the chunk, having a corresponding hash value, is already stored by the content storage system.

5. The computer-implemented method of claim 1, further comprising:

determining that an additional chunk of the media broadcast stream is not available to be provided to the determined client device; and

causing an interstitial page to be displayed in place of the additional chunk on the determined client device.

6. A computer-implemented method, comprising:

receiving a first request from a first client device to record a broadcast stream;

determining a set of chunks corresponding to the broadcast stream, each chunk including a respective segment of the broadcast stream and a respective hash value;

determining a first subset of the set of chunks for the first client device to upload based at least in part upon capabilities of the first client device;

causing the first client device to upload at most the first subset of the set of chunks corresponding to the broadcast stream, a second subset of the set of chunks being provided from other client devices of a plurality of client devices for which the broadcast stream is being recorded;

causing a plurality of manifests to be stored for the broadcast stream, the manifests including information for the set of chunks for the broadcast stream;

receiving, from the first client device, a second request to download the broadcast stream;

causing the set of chunks to be transcoded into a more space efficient format; and

causing, according to at least a subset of the plurality of the manifests, chunks of the set of chunks to be downloaded to the first client device where those chunks are not already stored locally by the first client device.

7. The computer-implemented method of claim 6, further comprising:

receiving, from a second client device of the plurality of client devices, a third request to download the broadcast stream;

determining, for each chunk of the set of chunks corresponding to the broadcast stream, whether the chunk is stored locally by the second client device; and

causing, according to the plurality of manifests, chunks of the set of chunks that are not stored locally to be downloaded to the second client device.

8. The computer-implemented method of claim 6, further comprising:

determining a set of chunking offsets for the set of chunks; and

providing the set of chunking offsets to the plurality of client devices, the plurality of client devices including the first client device.

9. The computer-implemented method of claim 6, further comprising:

causing each chunk of the set of chunks to be transcoded to a more space efficient format before causing the set of chunks to be uploaded from, or downloaded to, the plurality of client devices.

10. The computer-implemented method of claim 6 further comprising:

maintaining at least two versions of a specified chunk for the broadcast stream, each version having a different hash value based on a respective segment of the broadcast stream having different recorded data for each version.

11. The computer-implemented method of claim 6, wherein the capabilities of the first client device include at least one of a capacity, a bandwidth, or a throughput associated with the first client device.

12. The computer-implemented method of claim 6, further comprising:

determining, for each additional chunk of the broadcast stream that is not stored locally by the first client device, whether to cause the additional chunk to be downloaded to the first client device from a network storage service or whether to cause the additional chunk to be provided from another client device, of the plurality of client devices, storing a copy of the additional chunk; and

causing, for each additional chunk of the broadcast stream that is not stored locally by the first client device, the additional chunk to be provided to the first client device.

13. The computer-implemented method of claim 12, further comprising:

maintaining a respective manifest for each chunk of the set of chunks, each respective manifest including a respective hash value and metadata for the chunk.

14. The computer-implemented method of claim 6, wherein the broadcast stream includes at least one of a television broadcast, a multimedia broadcast, an audio broadcast, or a data broadcast.

15. The computer-implemented method of claim 6, further comprising:

comparing at least one quality factor for a stored copy of a specific chunk and an uploaded copy of the specific chunk; and

causing the uploaded copy to become a new stored copy of the specific chunk if the at least one quality factor for the uploaded copy is higher than for the stored copy.

16. A computer system, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the computer system to;

receive a first request from a first client device to record a broadcast stream;

determine a set of chunks corresponding to the broadcast stream, each chunk including a respective segment of the broadcast stream and a respective hash value;

determine a first subset of the set of chunks for the first client device to upload based at least in part upon capabilities of the first client device;

cause the first client device to upload at most the first subset of the set of chunks corresponding to the broadcast stream, a second subset of the set of chunks being provided from other client devices of a plurality of client devices for which the broadcast stream is being recorded;

cause a plurality of manifests to be stored for the broadcast stream, the manifests including information for the set of chunks for the broadcast stream;

receive, from the first client device, a second request to download the broadcast stream;

cause the set of chunks to be transcoded into a more space efficient format; and

cause, according to at least a subset of the plurality of the manifests, chunks of the set of chunks to be downloaded to the first client device where those chunks are not already stored locally by the first client device.

17. The computer system of claim 16, wherein the instructions when executed further cause the computing system to:

determine a set of chunking offsets for the set of chunks; and

provide the set of chunking offsets to the plurality of client devices, the plurality of client devices including the first client device.

18. The computer system of claim 16, wherein the capabilities of the first client device include at least one of a capacity, bandwidth, or throughput associated with the first client device.

19. The computer system of claim 16, wherein the instructions when executed further cause the computing system to:

determine, for each additional chunk of the broadcast stream not stored locally by the first client device, whether to cause the additional chunk to be downloaded to the first client device from a network storage service or whether to cause the additional chunk to be provided from another client device, of the plurality of client devices, storing a copy of the additional chunk; and

cause, for each additional chunk of the broadcast stream not stored locally by the first client device, the additional chunk to be provided to the determined client device.

20. The computer system of claim 16, wherein the instructions when executed further cause the computing system to:

determine that an additional chunk of the broadcast stream is not available to be provided to the first client device; and

cause an interstitial page to be displayed in place of the additional chunk on the first client device.