SYSTEM AND METHOD FOR LOAD BALANCING BASED ON EXPECTED LATENCY FOR USE IN MEDIA CONTENT OR OTHER ENVIRONMENTS

• US 20170230295A1
• Filed: 02/05/2016
• Published: 08/10/2017
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Spotify AB (Spotify Technology SA)
• Original Assignee: Spotify AB (Spotify Technology SA)
• Inventors: Polacek, Lukas
• Granted Patent: US 10,089,309 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/00 Network arrangements or protocols for supporting network services or applications user-to-user messaging H04L51/00; network arrangements, protocols or services for supporting real-time applications in data packet communications networks H04L65/00
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

In accordance with an embodiment, described herein is a system and method for load balancing of requests to servers based on expected latency, for use in the providing of media content or other types of data. The system comprises an access point, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers. Each of the servers can be associated with an endpoint. An expected latency selection component can be used by the access point to determine how requests are distributed or load balanced to servers, based on a determination of expected latency at those servers. Upon the access point receiving a particular request, the determination of expected latency can be used to determine which endpoint or server to direct that particular request.

Claims

1. A system for selection of servers based on expected latency, comprising:

a computer including a processor and memory;

an access point, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers;

an expected latency selection component, that is used by the access point, or other process or component, to determine how requests are distributed or load balanced to servers, based on a determination of expected latency at those servers;

wherein each of a plurality of servers is associated with an endpoint at which that server receives requests from the access point; and

whereupon the access point receiving a particular request for data, the determination of expected latency is used to determine which endpoint to direct that particular request.

2. The system of claim 1, comprising a media server, wherein the access point, or other process or component is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should stream or otherwise provide media content, in response to processing the media-access request.

3. The system of claim 2, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in streaming or otherwise providing the media content to the destination client device.

4. The system of claim 1, wherein the expected latency selection component measures, for each of the plurality of servers able to receive requests from the access point,

a data indicative of success latency and success rate for that server,

an indication of an amount of outstanding requests sent from the access point and that server, and

a data indicative of failure latency for that server,

which are then used in the determination of expected latency.

5. The system of claim 4, wherein a data indicative of failure roundtrip time is additionally used by the expected latency selection component in the determination of expected latency.

6. A method for selection of servers based on expected latency, comprising:

providing, at a computer including a processor and memory,

an access point, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers, and

an expected latency selection component, that is used by the access point, or other process or component, to determine how requests are distributed or load balanced to servers, based on a determination of expected latency at those servers;

associating each of a plurality of servers with an endpoint at which that server receives requests from the access point; and

upon receiving, at the access point, a particular request for data, determining according to the determination of expected latency, which endpoint to direct that particular request.

7. The method of claim 6, further comprising providing a media server, wherein the access point, or other process or component is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should stream or otherwise provide media content, in response to processing the media-access request.

8. The method of claim 7, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in streaming or otherwise providing the media content to the destination client device.

9. The method of claim 6, wherein the expected latency selection component measures, for each of the plurality of servers able to receive requests from the access point,

a success latency and success rate for that server,

an amount of outstanding requests sent from the access point and that server, and

a failure latency for that server,

which are then used in the determination of expected latency.

10. The method of claim 6, wherein an indication of failure roundtrip time is additionally used by the expected latency selection component in the determination of expected latency.

11. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:

providing, at a computer including a processor and memory,

an access point, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers, and

an expected latency selection component, that is used by the access point, or other process or component, to determine how requests are distributed or load balanced to servers, based on a determination of expected latency at those servers;

associating each of a plurality of servers with an endpoint at which that server receives requests from the access point; and

upon receiving, at the access point, a particular request for data, determining according to the determination of expected latency, which endpoint to direct that particular request.

12. The non-transitory computer readable storage medium of claim 11, further comprising providing a media server, wherein the access point, or other process or component is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should stream or otherwise provide media content, in response to processing the media-access request.

13. The non-transitory computer readable storage medium of claim 12, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in streaming or otherwise providing the media content to the destination client device.

14. The non-transitory computer readable storage medium of claim 11, wherein the expected latency selection component measures, for each of the plurality of servers able to receive requests from the access point,

a success latency and success rate for that server,

an amount of outstanding requests sent from the access point and that server, and

a failure latency for that server,

which are then used in the determination of expected latency.

15. The non-transitory computer readable storage medium of claim 11, wherein an indication of failure roundtrip time is additionally used by the expected latency selection component in the determination of expected latency.