System and method for early media buffering using detection of user behavior

• US 10,425,665 B2
• Filed: 10/27/2016
• Issued: 09/24/2019
• Est. Priority Date: 06/17/2013
• Status: Active Grant

• Current Assignee: Spotify AB (Spotify Technology SA)
• Original Assignee: Spotify AB (Spotify Technology SA)
• Inventors: Oman, Andreas
• Primary Examiner(s): Flynn, Nathan J
• Assistant Examiner(s): Somera, Caroline
• Application Number:
  US15/336,665
• Publication Number:
  US 20170048563A1
• Time in Patent Office: 1,062 Days
• Field of Search: 725110
• CPC Class Codes: H04L 47/722 at the destination endpoint, e.g. reservation of terminal resources or buffer space
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04N 21/2183 Cache memory caches in web servers or browsers G06F16/9574; intermediate storage and caching in data networks H04L67/568
• CPC Class Codes: H04N 21/2225 Local VOD servers
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/437 Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server flow control in data networks H04L47/10; streaming protocols, e.g. RTP or RTCP, H04L65/65; scheduling or organising the servicing of application requests in data packet switching networks H04L67/60
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/8545 for generating interactive applications

Abstract

A system and method for early media buffering using prediction of user behavior. In accordance with an embodiment, a client device user interface displays a plurality of media options, each of which is associated with a corresponding media content, for example a media channel or video stream, that can be received from a media server system for playing at the device, or at a controlled device. While the user interacts with the user interface, media content associated with regions of the interface can be buffered in preparation for selection and subsequent playing. The client device can include a user interaction sensor, for example a motion, hand/proximity, or heat sensor, which detects when a user is about to interact with the device. The media server system can be triggered to buffer media content associated with one or more neighboring media options on the user interface, in preparation for selection and playing.

Claims

1. A system for early media buffering using prediction of user behavior, comprising:

a client device, including a processor;

an application user interface provided at the client device, which displays a plurality of media options as a visual array of media tiles, each of which media tiles is associated with a corresponding media stream that can be received from a media server system for playing at one or more of the client device, or a controlled device;

a user interaction sensor provided at the client device, which detects when a user is about to interact with the client device; and

a user prediction logic provided at the client device, which receives an indication from the user interaction sensor when the user is about to interact with the client device;

wherein the client device operates to

determine one or more neighboring media tiles within the visual array of media tiles as one or more media tiles having a likelihood to be selected next, and communicate with the media server system to enable the client device to buffer the corresponding media streams associated with a first set of media tiles within the visual array, including the one or more media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing, and

in response to receiving an input indicative of panning the visual array of media tiles,

display one or more additional media options at the user interface as one or more additional media tiles,

determine one or more of the additional media tiles as having a likelihood to be selected next, and

communicate with the media server system to enable the client device to buffer the corresponding media streams associated with a second set of media tiles within the visual array, including at least one of the one or more additional media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing.

2. The system of claim 1, wherein each of the media tiles is associated with a corresponding media channel or video stream that can be selected and received from the media server system for playing at one or both of the client device and the controlled device.

3. The system of claim 1, wherein the user interaction sensor is one or more of a motion, hand/proximity, or heat sensor, which detects when the user picks up the client device and provides the indication when the user is about to interact with the client device.

4. The system of claim 1, wherein the controlled device includes an output display that displays media content, and wherein the client device operates as a remote control, and is adapted to receive a selection from the user of a selected media tile, for subsequent receipt from the media server system and playing at the output display of the controlled device.

5. The system of claim 4, wherein the controlled device displays media channels or video streams on the output display as received from the media server system, and wherein selection from the user of a selected media tile is used as a media-change request to change a media channel at the controlled device.

16. The system of claim 1, wherein determining the one or more media tiles having a likelihood to be selected next includes predicting a direction of a subsequent swipe input.

19. The system of claim 1, further comprising a buffering logic, which enables a portion of each media content item associated with one or more neighboring media tiles displayed to the left, right, above, or below a selected media tile as the one or more media tiles having a likelihood to be selected next, to be pre-buffered as determined by the user prediction logic.

6. A method for early media buffering using prediction of user behavior, comprising:

providing, at a client device, an application user interface which displays a plurality of media options as a visual array of media tiles, each of which media tiles is associated with a corresponding media stream that can be received from a media server system for playing at one or more of the client device, or a controlled device;

detecting, using a user interaction sensor provided at the client device, when a user is about to interact with the client device;

receiving an indication from the user interaction sensor when the user is about to interact with the client device;

determining one or more neighboring media tiles within the visual array of media tiles as one or more media tiles having a likelihood to be selected next, and communicating with the media server system to enable the client device to buffer the corresponding media streams associated with a first set of media tiles within the visual array, including the one or more media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing; and

in response to receiving an input indicative of panning the visual array of media tiles

displaying one or more additional media options at the user interface as one or more additional media tiles,

determining one or more of the additional media tiles as having a likelihood to be selected next, and

communicating with the media server system to enable the client device to buffer the corresponding media streams associated with a second set of media tiles within the visual array, including at least one of the one or more additional media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing.

7. The method of claim 6, wherein each of the media tiles is associated with a corresponding media channel or video stream that can be selected and received from the media server system for playing at one or both of the client device and the controlled device.

8. The method of claim 6, wherein the user interaction sensor is one or more of a motion, hand/proximity, or heat sensor, which detects when the user picks up the client device and provides the indication when the user is about to interact with the client device.

9. The method of claim 6, wherein the controlled device includes an output display that displays media content, and wherein the client device operates as a remote control, and is adapted to receive a selection from the user of a selected media tile, for subsequent receipt from the media server system and playing at the output display of the controlled device.

10. The method of claim 9, wherein the controlled device displays media channels or video streams on the output display as received from the media server system, and wherein selection from the user of a selected media tile is used as a media-change request to change a media channel at the controlled device.

17. The method of claim 6, wherein determining the one or more media tiles having a likelihood to be selected next includes predicting a direction of a subsequent swipe input.

20. The method of claim 6, further comprising pre-buffering a portion of each media content item associated with one or more neighboring media tiles displayed to the left, right, above, or below a selected media tile as the one or more media tiles having a likelihood to be selected next.

11. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform steps comprising:

providing, at a client device, an application user interface which displays a plurality of media options as a visual array of media tiles, each of which media tiles is associated with a corresponding media stream that can be received from a media server system for playing at one or more of the client device, or a controlled device;

detecting, using a user interaction sensor provided at the client device, when a user is about to interact with the client device; and

receiving an indication from the user interaction sensor when the user is about to interact with the client device;

determining one or more neighboring media tiles within the visual array of media tiles as one or more media tiles having a likelihood to be selected next, and communicating with the media server system to enable the client device to buffer the corresponding media streams associated with a first set of media tiles within the visual array, including the one or more media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing; and

in response to receiving an input indicative of panning the visual array of media tiles,

displaying one or more additional media options at the user interface as one or more additional media tiles,

determining one or more of the additional media tiles as having a likelihood to be selected next, and

communicating with the media server system to enable the client device to buffer the corresponding media streams associated with a second set of media options within the visual array, including at least one of the one or more additional media tiles having a likelihood to be selected next, in preparation for selection and subsequent playing.

12. The non-transitory computer readable storage medium of claim 11, wherein each of the media tiles is associated with a corresponding media channel or video stream that can be selected and received from the media server system for playing at one or both of the client device and the controlled device.

13. The non-transitory computer readable storage medium of claim 11, wherein the user interaction sensor is one or more of a motion, hand/proximity, or heat sensor, which detects when the user picks up the client device and provides the indication when the user is about to interact with the client device.

14. The non-transitory computer readable storage medium of claim 11, wherein the controlled device includes an output display that displays media content, and wherein the client device operates as a remote control, and is adapted to receive a selection from the user of a selected media tile, for subsequent receipt from the media server system and playing at the output display of the controlled device.

15. The non-transitory computer readable storage medium of claim 14, wherein the controlled device displays media channels or video streams on the output display as received from the media server system, and wherein selection from the user of a selected media tile is used as a media-change request to change a media channel at the controlled device.

18. The non-transitory computer readable storage medium of claim 11, wherein determining the one or more media tiles having a likelihood to be selected next includes predicting a direction of a subsequent swipe input.

21. The non-transitory computer readable storage medium of claim 11, wherein the instructions further cause the one or more computers to pre-buffer a portion of each media content item associated with one or more neighboring media tiles displayed to the left, right, above, or below a selected media tile as the one or more media tiles having a likelihood to be selected next.