Temporary decoder apparatus and method

• US 9,962,610 B2
• Filed: 06/22/2015
• Issued: 05/08/2018
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger, Cotter, Timothy, Furman, Scott, McCool, Robert, Buckley, Ian
• Primary Examiner(s): Taylor, Joshua D
• Application Number:
  US14/746,428
• Publication Number:
  US 20150289028A1
• Time in Patent Office: 1,051 Days
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/535 for monitoring, e.g. of user parameters, terminal parameters, application parameters, network parameters
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/25833 : involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055
• CPC Class Codes: H04N 21/25858 : involving client software characteristics, e.g. OS identifier
• CPC Class Codes: H04N 21/4263 : involving specific tuning arrangements, e.g. two tuners
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6336 : directed to decoder
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6379 : directed to encoder , e.g. for requesting a lower encoding rate
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/6547 : comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A computer-implemented system and method for streaming video from a server to a client are described. For example, a method according to one embodiment comprises: receiving at the server a request for video content from the client; in response to the request, determining the hardware/software configuration of the client; generating and/or selecting a temporary decoder based on the hardware/software configuration of the client; transmitting the temporary decoder to the client, the client installing the temporary decoder; encoding and streaming the requested video content from the server to the client, the video content being encoded based on the capabilities of the temporary decoder, the video content being decoded by the temporary decoder and rendered on the client; detecting that the client has ended the session with the server; and in response to detecting that the client has ended the session, temporarily disabling and/or removing the temporary decoder from the client.

Claims

1. A computer-implemented method for streaming video from a server to a client, the server being part of a hosting service and the client being one of a plurality of clients that are provided with access to the hosting service, comprising:

receiving at the server a request for video content from the client during a session, wherein the video content changes interactively based on user input received at the server from the client;

in response to the request, determining, by the server, a configuration of the client based on information received from the client;

selecting, by the server, a temporary decoder based on the configuration of the client;

transmitting, by the server, the temporary decoder to the client for use by the client, the temporary decoder configured to decode both I-frames and P-frames, the server configured to receive verification of receipt of the temporary decoder by the client;

encoding and streaming, after the verification, the requested video content from the server to the client, the video content being encoded based on capabilities of the temporary decoder, the encoded video content configured to be decoded by the temporary decoder and rendered on the client, wherein encoding of the requested video content is performed to achieve a data communication latency of less than or equal to a specified maximum value, the data communication latency measured from initiation of transmission of the user input from the client to the server and rendering of the video content corresponding to the user input on the client, wherein a next frame of the requested video content is encoded as an I-frame when the data communication latency is less than a latency threshold value, wherein the next frame of the requested video content is encoded as a P-frame when the data communication latency is greater than or equal to the latency threshold value, wherein the latency threshold value is less than the specified maximum value;

detecting that the client has ended the session with the server; and

in response to detecting that the client has ended the session, sending instructions to the client to disable the temporary decoder at the client.

2. The method as in claim 1, wherein the encoding is associated to an image resolution, a frame rate, a bitrate, and/or a compression ratio.

3. The method as in claim 1, wherein the configuration of the client includes characteristics of a communication channel between the client and the server, the communication channel is associated with a round-trip latency.

4. The method as in claim 3, wherein one of the characteristics of the communication channel comprises a data rate of the communication channel;

or

wherein one of the characteristics of the communication channel comprises a reliability of the communication channel.

5. The method as in claim 1, wherein the user input drives changes in types of content rendered on the server and transmitted to the client through a communication channel.

6. The method as in claim 5, wherein the reliability of the communication channel is characterized based on a percentage of lost packets or packets received with errors.

7. The method as in claim 6, further comprising:

in response to detecting a percentage of lost packets or packets received with errors below a first specified threshold value, encoding the requested video content without error correction; and

in response to detecting a percentage of lost packets or packets received with errors above the first specified threshold value, encoding the requested video content with error correction.

8. The method as in claim 7, further comprising:

in response to detecting a percentage of lost packets or packets received with errors above a second specified threshold value, attempting to establish a different communication channel with the client.

9. The method as in claim 8, wherein the different communication channel is between the client and a different server.

10. The method as in claim 2, wherein one or more of the image resolution, the frame rate, the bitrate, and/or the compression ratio is adjusted during streaming of the requested video content from the server to the client in order to maintain the data communication latency of less than or equal to the specified maximum value.

11. A hosting service configured for streaming video from a server to a client, the server being part of the hosting service and the client being one of a plurality of clients that are provided with access to the hosting service, comprising:

a hosting service downloader application of the hosting service configured to receive a request for video content during a session, wherein the video content changes interactively based on user input received at the hosting service from the client, the hosting service downloader application configured to receive a configuration of the client from a client downloader application of the client,

the hosting service downloader application configured to select a temporary decoder based on the configuration of the client, the temporary decoder configured to decode both I-frames and P-frames,

the hosting service downloader application configured to direct transmission of the temporary decoder to the client downloader application for installation on the client,

the hosting service downloader application configured to verify installation of the temporary decoder on the client; and

an encoder of the hosting service configured to receive information regarding the temporary decoder installed on the client, the encoder configured to encode the requested video content based on capabilities of the temporary decoder to generate encoded video content, the encoded video content configured for decoding by the temporary decoder to enable rendering of the requested video content on the client, the encoder configured to encode the requested video content to achieve a data communication latency of less than or equal to a specified maximum value, the data communication latency measured from initiation of transmission of the user input from the client to the hosting service and rendering of the requested video content corresponding to the user input on the client, wherein the encoder is configured to encode a next frame of the requested video content as an I-frame when the data communication latency is less than a latency threshold value, wherein the encoder is configured to encode the next frame of the requested video content as a P-frame when the data communication latency is greater than or equal to the latency threshold value, wherein the latency threshold value is less than the specified maximum value,

the encoder configured to transmit the encoded video content from the server to the client after verifying installation of the temporary decoder on the client,

wherein the hosting service downloader application is configured to communicate with the client downloader application to uninstall the temporary decoder at the client upon detecting termination of the session.

12. The hosting service as in claim 11, wherein the encoding is associated to an image resolution, a frame rate, a bitrate, and/or a compression ratio.

13. The hosting service as in claim 11, wherein the configuration of the client includes characteristics of a communication channel between the client and the server, the communication channel is associated with a round-trip latency.

14. The hosting service as in claim 13, wherein one of the characteristics of the communication channel comprises a data rate of the communication channel;

or

wherein one of the characteristics of the communication channel comprises a reliability of the communication channel.

15. The hosting service as in claim 13, wherein the user input drives changes in types of content rendered on the server and transmitted to the client through the communication channel.

16. The hosting service as in claim 15, wherein the reliability of the communication channel is characterized based on a percentage of lost packets or packets received with errors.

17. The hosting service as in claim 16, wherein in response to detecting a percentage of lost packets or packets received with errors below a first specified threshold value, the encoder is configured to encode the requested video content without error correction, and wherein in response to detecting a percentage of lost packets or packets received with errors above the first specified threshold value, the encoder is configured to encode the requested video content with error correction.

18. The hosting service as in claim 17, wherein in response to detecting a percentage of lost packets or packets received with errors above a second specified threshold value, the hosting service is configured to attempt to establish a different communication channel with the client.

19. The hosting service as in claim 18, wherein the different communication channel is between the client and a different server.

20. The hosting service as in claim 12, wherein the hosting service is configured to adjust one or more of the image resolution, the frame rate, the bitrate, and/or the compression ratio is during streaming of the requested video content from the server to the client in order to maintain the data communication latency of less than or equal to the specified maximum value.