Server system for processing graphic output and responsively blocking select input commands

• US 10,226,700 B2
• Filed: 05/23/2016
• Issued: 03/12/2019
• Est. Priority Date: 08/03/2006
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Gault, Andrew Buchanan, Pereira, Rui Filipe Andrade, Perry, David, Lau, Brian K. F., Yong, Kelvin Michael, Rickeby, Claes Christian, Breed, Ryan Hamilton, Galano, III, Eleazar T., English, Austin
• Primary Examiner(s): McDowell, Jr., Maurice L.
• Application Number:
  US15/162,548
• Publication Number:
  US 20160339341A1
• Time in Patent Office: 1,023 Days
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video

Abstract

A video server is configured to provide streaming video to players of computer games over a computing network. The video server can provided video of different games to different players simultaneously. This is accomplished by rendering several video streams in parallel using a single GPU (Graphics Processing Unit). The output of the GPU is provided to graphics processing pipelines that are each associated with a specific client/player and are dynamically allocated as needed. A client qualifier may be used to assure that only clients capable of presenting the streaming video to a player at a minimum level of quality receive the video stream. Video frames provided by the video server optionally include overlays added to the output of the GPU. These overlays can include voice data received from another game player. These overlays may be used to prevent presentation of non-allowed input controls.

Claims

1. A video server system comprising:

an application engine for executing a first computer application;

a graphics processing unit for generating a graphic output in response to graphics data received from the first computer application and a point of view;

command processing logic for blocking communication of an input command to the first computer application by examining input commands received by one or more input/output devices before forwarding the input commands to the first computer application, so as to affect the graphic output, wherein video frames of the graphic output is adjusted to not include an input control for the input command that is blocked;

an encoding pipeline for generating an encoded video stream using the graphic output, wherein the encoded video stream includes video frames that do not include the input control for the input command that is blocked by the command processing logic;

an output logic for communicating the encoded video stream to a remote client via a communication network, so that the video frames rendered on the remote client do not graphically display the input control for the input command that is blocked; and

a microprocessor for executing the command processing logic at the video server system.

2. The system of claim 1, wherein the command processing logic selects specific input commands to block depending on a state of the first computer application.

3. The system of claim 1, wherein the command processing logic determines specific input commands to block based on characteristics of a game player'"'"'s account.

4. The system of claim 1, wherein the command processing logic includes a second input control associated with a non-blocked input command in place of the input control associated with the blocked input command, interaction at the second input control re-directs the first computer application to load a pre-approved add-on on to the video server system.

5. The system of claim 1, wherein the command processing logic processes the input command that is blocked by saving a state of the first computer application in static storage.

6. The system of claim 1, wherein the command processing logic processes the input command that is blocked so as to activate an alternate application.

7. The system of claim 1, wherein the command processing logic processes the input command that is blocked by engaging a client control logic to open or close a window or tab on the remote client, wherein the window or tab is a browser window or browser tab.

8. The system of claim 7, wherein the command processing logic initiates generation of a video of a virtual file system using information provided by the video server system, wherein the initiation includes engaging the client control logic to open a second browser window for rendering images of the first computer application.

9. The system of claim 7, wherein the client control logic provides a uniform resource locator (URL) to a video output of the video server system when the browser window is opened, the video output providing a video of a virtual file system of the video server system, wherein navigation and selection of a file from the virtual file system is enabled using virtual controls provided within the video.

10. The system of claim 7, wherein the client control logic loads a website into the browser window or browser tab on the remote client, when the browser window or browser tab is opened.

11. The system of claim 10, wherein the website is a social networking website.

12. The system of claim 1, wherein the command processing logic causes processing of the input command that is blocked by executing a second computer application on the remote client.

13. The system of claim 1, wherein the command processing logic processes the input command that is blocked by passing a different input command to the first computer application.

14. The system of claim 1, further comprising state logic for changing operation of the command processing logic responsive to a state of the first computer application, such that different sets of input commands are blocked depending on the state.

15. The system of claim 1, wherein the first computer application is a video game.

16. A method comprising:

executing a first computer application using an application engine running on a video server system;

generating, by a graphics processor unit of the video server system, a graphic output in response to graphics data received from the first computer application and a point of view;

examining, by a command processing logic of the video server system, input commands received by one or more input/output devices before forwarding the input commands to the first computer application to affect the graphic output, wherein the examining includes blocking communication of an input command to the first computer application so as to not be included in the graphic output, wherein the blocking causes adjustment to video frames of the graphic output so as to not include input control for the input command that is blocked;

generating, by an encoding pipeline of the video server system, an encoded video stream using the graphic output, wherein the encoded video stream includes video frames that do not include the input control for the input command that is blocked; and

communicating, by an output logic of the video server system, the encoded video stream to a remote client via a communication network, so that the video frames rendered on the remote client do not graphically display the input control for the input command that is blocked.

17. The method of claim 16, wherein the examining of input commands further includes identifying specific input commands that are blocked, based on a state of the first computer application.

18. The method of claim 16, further includes directing the first computer application to load a pre-approved add-on on to the video server system, upon detecting the input command that is blocked.

19. The method of claim 16, further includes saving a state of the first computer application in static storage, upon detecting the input command that is blocked.

20. The method of claim 16, further includes processing the input command that is blocked by activating a client control logic, wherein the client control logic is configured to open or close a window or tab on the remote client, wherein the window or tab is a browser window or browser tab.

21. A non-transitory computer-readable medium having program instructions, which when executed by a microprocessor in a video server system performs a method, the program instructions include:

program instructions for executing a first computer application;

program instructions for generating a graphic output in response to graphics data received from the first computer application and a point of view;

program instructions for examining input commands received by one or more input/output devices before forwarding the input commands to the first computer application to affect the graphic output, wherein the program instructions for examining includes program instructions for blocking communication of an input command that is blocked to the first computer application so as to not be included in the graphic output, wherein the program instructions for blocking causes adjustment to video frames of the graphic output so as to not include input control for the input command that is blocked;

program instructions for generating an encoded video stream using the graphic output, wherein the encoded video stream includes video frames that do not include the input control for the input command that is blocked; and

program instructions for communicating the encoded video stream to a remote client via a communication network, so that the video frames rendered on the remote client do not graphically display the input control for the input command that is blocked.