System and method for load balancing based on expected latency for use in media content or other environments

• US 10,089,309 B2
• Filed: 02/05/2016
• Issued: 10/02/2018
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Spotify AB (Spotify Technology SA)
• Original Assignee: Spotify AB (Spotify Technology SA)
• Inventors: Polacek, Lukas
• Primary Examiner(s): Bui, Jonathan A
• Application Number:
  US15/017,392
• Publication Number:
  US 20170230295A1
• Time in Patent Office: 970 Days
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/00 Network arrangements or protocols for supporting network services or applications user-to-user messaging H04L51/00; network arrangements, protocols or services for supporting real-time applications in data packet communications networks H04L65/00
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

In accordance with an embodiment, described herein is a system and method for load balancing of requests to servers based on expected latency, for use in the providing of media content or other types of data. The system comprises an access point, or other process or component, which operates as a load balancer in providing access to one or more servers, for use in processing requests at those servers. Each of the servers can be associated with an endpoint. An expected latency selection component can be used by the access point to determine how requests are distributed or load balanced to servers, based on a determination of expected latency at those servers. Upon the access point receiving a particular request, the determination of expected latency can be used to determine which endpoint or server to direct that particular request.

Claims

1. A system for selection of servers based on expected latency, comprising:

a computer including a processor and memory;

an access point that operates on the computer as a load balancer to provide access to a plurality of servers, for use in processing requests at the plurality of servers;

an expected latency selection component, that is used by the access point to determine a distribution of the requests to the plurality of servers, based on a determination of expected latency at the plurality of servers, wherein the expected latency is determined based on measurement, for each server of the plurality of servers able to receive requests from the access point, data indicative of success latency, success rate, amount of outstanding requests, and failure latency for that server;

wherein each server of the plurality of servers is associated with an endpoint at which that server receives requests from the access point; and

whereupon the access point receiving a particular request for data, the determination of expected latency at the plurality of servers is used to determine a particular endpoint associated with a particular server to direct the particular request.

2. The system of claim 1, comprising a media server, wherein the access point is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should provide media content, in response to processing the media-access request.

3. The system of claim 2, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in providing the media content to the destination client device.

4. The system of claim 1, wherein a data indicative of failure roundtrip time is additionally used by the expected latency selection component in the determination of expected latency at the plurality of servers.

5. The system of claim 1, wherein the expected latency selection component determines a weight associated with each of the plurality of servers able to receive requests, for use in directing traffic to the servers in accordance with their weights.

6. The system of claim 1, wherein the expected latency selection component removes a server from rotation if its expected latency is below a certain threshold.

7. A method for selection of servers based on expected latency, comprising:

providing, at a computer including a processor and memory,

an access point that operates on the computer as a load balancer in providing access to a plurality of servers, for use in processing requests at the plurality of servers, and

an expected latency selection component, that is used by the access point to determine a distribution of the requests to the plurality of servers, based on a determination of expected latency at the plurality of servers, wherein the expected latency is determined based on measuring, for each server of the plurality of servers able to receive requests from the access point, data indicative of success latency, success rate, amount of outstanding requests, and failure latency for that server;

associating each server of the plurality of servers with an endpoint at which that server receives requests from the access point; and

upon receiving, at the access point, a particular request for data, determining according to the determination of expected latency at the plurality of servers, a particular endpoint associated with a particular server to direct the particular request.

8. The method of claim 7, further comprising providing a media server, wherein the access point is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should provide media content, in response to processing the media-access request.

9. The method of claim 8, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in providing the media content to the destination client device.

10. The method of claim 7, wherein an indication of failure roundtrip time is additionally used by the expected latency selection component in the determination of expected latency at the plurality of servers.

11. The method of claim 7, wherein the expected latency selection component determines a weight associated with each of the plurality of servers able to receive requests, for use in directing traffic to the servers in accordance with their weights.

12. The method of claim 7, wherein the expected latency selection component removes a server from rotation if its expected latency is below a certain threshold.

13. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform the steps comprising:

performing, at a computer including a processor and memory, a process which

operates as a load balancer in providing access to a plurality of servers, for use in processing requests at the plurality of servers, and

determines a distribution of the requests to the plurality of servers, based on a determination of expected latency at the plurality of servers, wherein the expected latency is determined based on measuring, for each server of the plurality of servers able to receive requests, data indicative of success latency, success rate, amount of outstanding requests, and failure latency for that server;

associating each server of the plurality of servers with an endpoint at which that server receives requests; and

upon receiving a particular request for data, determining according to the determination of expected latency at the plurality of servers, a particular endpoint associated with a particular server to direct the particular request.

14. The non-transitory computer readable storage medium of claim 13, further comprising providing a media server, wherein the process is configured to receive media-access requests from a client device, which identifies a destination client device to which the media server should provide media content, in response to processing the media-access request.

15. The non-transitory computer readable storage medium of claim 14, wherein each of the plurality of servers is one of a media content metadata server, or a proxy media content metadata server, for use in providing the media content to the destination client device.

16. The non-transitory computer readable storage medium of claim 13, wherein an indication of failure roundtrip time is additionally used by the process in the determination of expected latency at the plurality of servers.

17. The non-transitory computer readable storage medium of claim 13, wherein the process determines a weight associated with each of the plurality of servers able to receive requests, for use in directing traffic to the servers in accordance with their weights.