Qualified video delivery

• US 9,584,575 B2
• Filed: 06/01/2010
• Issued: 02/28/2017
• Est. Priority Date: 06/01/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perry, David, Gault, Andrew Buchanan, Pereira, Rui Filipe Andrade
• Primary Examiner(s): Dollinger, Tonia L
• Assistant Examiner(s): NGUYEN, STEVEN C
• Application Number:
  US12/791,819
• Publication Number:
  US 20100306813A1
• Time in Patent Office: 2,464 Days
• Field of Search: 709/228, 709/232
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/71 using secure communication between game devices and game servers, e.g. by encrypting game data or authenticating players
• CPC Class Codes: A63F 13/77 involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
• CPC Class Codes: A63F 13/795 for finding other players; for building a team; for providing a buddy list
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/51 Server architecture
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: G06F 2009/45579 : I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45595 : Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45537 : Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 65/1101 : Session protocols
• CPC Class Codes: H04L 65/61 : for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 : Media network packetisation
• CPC Class Codes: H04L 65/80 : Responding to QoS
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 : specially adapted for the location of the user terminal
• CPC Class Codes: H04N 19/136 : Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/42 : characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/44 : Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 : Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

A video server is configured to provide streaming video to players of computer games over a computing network. The video server can provided video of different games to different players simultaneously. This is accomplished by rendering several video streams in parallel using a single GPU. The output of the GPU is provided to graphics processing pipelines that are each associated with a specific client/player and are dynamically allocated as needed. A client qualifier may be used to assure that only clients capable of presenting the streaming video to a player at a minimum level of quality receive the video stream.

Claims

1. A video server system having a processor for executing program instructions stored in a memory to provide, comprising:

a client qualifier configured to remotely determine if a client is qualified to receive a video stream configured for display on the client in response to a received request to access a selected video game, the qualification being based on abilities of the client to receive and display the video stream at least at a minimum level of quality;

a monitor configured to assign clients to particular ones of a plurality of video sources, the assignment being made in part based on test results gathered by the client qualifier that confirms the minimum level of quality for the video stream and whether the video source is provisioned with game logic for the selected video game, wherein each video source includes,

virtual machines for the provisioned game logic;

virtual drivers associated with the virtual machines;

a non-virtual driver executed on a hardware graphical processing unit (GPU);

a shared memory;

the non-virtual driver is configured for receiving one or more video streams in parallel from the virtual drivers, the hardware GPU is configured for producing video frames, wherein each virtual driver is configured to manage transfer of a respective video stream to the non-virtual driver and manage transfer of video frames for the respective video stream to the shared memory;

an encoder for encoding the video frames, the encoder is defined by one or more dynamic encoding pipelines (DEPs), and each DEP performs the encoding using an H.264 protocol, wherein video frames produced for each virtual driver are processed by a respective DEP having access to the shared memory; and

an I/O device configured for communicating encoded video streams to the client via a communication network.

2. The system of claim 1, wherein the client qualifier is further configured to provide a software agent to the client, the software agent being configured to determine the abilities of the client to receive and display the video stream.

3. The system of claim 2, wherein the abilities of the client to receive and display the video stream include an ability of the client to decode the encoded video stream and a bandwidth and a reliability of a communication channel between the client and the video source.

4. The system of claim 1, wherein the client qualifier is further configured to control display of an object on the client, dependent on whether the client is qualified to receive the encoded video stream.

5. The system of claim 4, wherein the object is only displayed if the client is qualified.

6. The system of claim 4, wherein the object includes a banner advertisement, a link, an image, text, or a video.

7. The system of claim 1, wherein the video source is configured to provide additional encoded video streams, the additional encoded video streams using one or more additional encoding protocols, each protocol having one or more of a different encodings, or frame size, or color depth or frame rate.

8. The system of claim 7, wherein the I/O device is configured for further communicating the additional encoded video streams to different IP addresses.

9. The system of claim 7, wherein one IP addresses is an IP address of a social networking site.

10. The system of claim 1, wherein the video source includes an audio source.

11. A video server system having a processor for executing program instructions stored in a memory to provide, comprising:

a client qualifier configured to remotely determine if a client is qualified to receive a video stream configured for display on the client in response to a received request to access a selected video game, the qualification being based on abilities of the client to receive and display the video stream at least a minimum level of quality;

a monitor configured to assign clients to particular ones of a plurality of video sources, assignments being made based on test results gathered by the client qualifier that confirm the minimum level of quality for the video stream and based on selection of a geographic location that is more proximate that other geographic locations of the client that produced the request to access the selected video game, wherein each video source of the video server system includes,

(i) virtual machines for provisioning game logic for video games;

(ii) virtual drivers associated with the virtual machines;

(iii) a non-virtual driver executed on a hardware graphical processing unit (GPU), the non-virtual driver configured for receiving one or more video streams in parallel from the virtual drivers, the hardware GPU configured for producing the video streams;

(iv) a shared memory;

(v) an encoder for encoding the video frames, and wherein each virtual driver is configured to direct transfer of a respective video stream to the non-virtual driver and direct transfer of video frames for the respective video stream to the shared memory, the encoder is defined by one or more dynamic encoding pipelines (DEPs), and a respective one of the DEP is configured to access the shared memory to encode video frames for respective ones of the virtual drivers;

an I/O device for communicating the encoded video frames as encoded video streams to one of the client or other clients, wherein the minimum level of quality identifies when the client is able to decode the encoded video stream that is encoded by the encoder.

12. The system of claim 11, wherein the client qualifier performs tests over a single page load, or multiple web page loads, the multiple web page loads produce data that is gathered to produce statistics to estimate reliably of the client.

13. The system of claim 11, wherein the client qualifier produces test results that define whether particular video games will not be made available to the client.

14. The system of claim 13, wherein the client qualifier produces test results that define whether particular video games will not be displayed to the client as available.

15. The system of claim 13, wherein the I/O device is configured for communicating to different IP addresses and one IP addresses is an IP address of a social networking site, such that the video stream enables playing a game with a friend via the video stream sent to the social networking site.

16. The system of claim 11, wherein the monitor is configured to detect changes in sizes of a browser window, and automatic adjustments of frame size for the video streaming via a size of the browser window.

17. The system of claim 11, wherein the video stream includes overlays of content that include a command interface, or log in instructions, or messages to/from game players, or images of other game players, or video feeds of other game players, or audio sounds of other game players, or combinations thereof.

18. The system of claim 11, wherein the encoder encodes one or more video streams using one of Adobe Flash®

standard, or .wav, or .mpg, or H.264, or H.263, or On2, or VP6, or VC-1, or WMA, or Huffyuv, or Lagarith, or MPG-x., or Xvid., or FFmpeg, or x264, or VP6-8, or realvideo, or mp3.

19. A cloud gaming system defined by one or more servers connected to the Internet and having a processor for executing program instructions stored in a memory, comprising:

a client qualifier configured to remotely determine if clients are qualified to receive a video stream in response to requests to access select video games, the qualification being based on abilities of the client to receive and display the video stream at least at a minimum level of quality, the select video games being provided via streaming to the clients;

a plurality of video sources, each video source being assignable to one or more clients, each video source is defined by at least one virtual machine, wherein select virtual machines are provisioned with access to video game logic for executing ones of select video games, wherein one of the virtual machines is associated with a virtual driver that communicates with a non-virtual driver, the non-virtual driver handling exchange of execution by a graphical processing unit (GPU), the GPU is configured to produce video frames that are encoded by an encoder, the encoder producing the video stream in an encoded format, wherein video frames produced by the GPU are directed to a shared memory by the virtual driver, and the encoder is defined by a dynamic encoding pipeline (DEP) that is configured to access the shared memory for encoding the video frames produced by the GPU; and

an I/O device for communicating the video stream that is encoded by the DEP to the client, the client being qualified to decode the encoded format of the video stream.

20. The cloud gaming system of claim 19, wherein the encoder format is one of Adobe Flash®

standard, or .wav, or .mpg, or H.264, or H.263, or On2, or VP6, or VC-1, or WMA, or Huffyuv, or Lagarith, or MPG-x., or Xvid., or FFmpeg, or x264, or VP6-8, or realvideo, or mp3.