Video game overlay

• US 10,173,134 B2
• Filed: 12/30/2016
• Issued: 01/08/2019
• Est. Priority Date: 06/01/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Gault, Andrew Buchanan, Perry, David, Pereira, Rui Filipe Andrade
• Primary Examiner(s): Rowland, Steve
• Application Number:
  US15/396,169
• Publication Number:
  US 20170106281A1
• Time in Patent Office: 739 Days
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/32 using local area network [LAN] connections
• CPC Class Codes: A63F 13/327 using wireless networks, e.g. Wi-Fi® or piconet
• CPC Class Codes: A63F 13/33 using wide area network [WAN] connections
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: G06F 9/45537 Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence

Abstract

A video server is configured to provide streaming video to players of computer games over a computing network. The video server can provided video of different games to different players simultaneously. This is accomplished by rendering several video streams in parallel using a single GPU (Graphics Processing Unit). The output of the GPU is provided to graphics processing pipelines that are each associated with a specific client/player and are dynamically allocated as needed. A client qualifier may be used to assure that only clients capable of presenting the streaming video to a player at a minimum level of quality receive the video stream. Video frames provided by the video server optionally include overlays added to the output of the GPU. These overlays can include voice data received from another game player.

Claims

1. A method for generating video for a video game, comprising:

transmitting information about a decoder capability of a client computing system to a server computing system;

receiving encoded video data from the server computing system at the client computing system, the encoded video data including an encoded version of video data for the video game representing a game environment rendered based on game rules and one or more input signals, the encoded video data generated by processing video data for the video game through a dynamically provisioned encoding pipeline that includes an encoder compatible with the decoder capability of the client computing system;

decoding the received encoded video data at the client computing system to obtain the video data for the video game;

applying a video overlay to at least a portion of the video data for the video game at the client computing system to generate modified video data for the video game locally at the client computing system, wherein applying the video overlay includes mapping pixels of a video overlay image to specific pixels in a video frame defined by the video data for the video game; and

providing the modified video data for the video game for display at the client computing system.

2. The method for generating video for the video game as recited in claim 1, wherein the video overlay is defined by overlay data sourced at the client computing system.

3. The method for generating video for the video game as recited in claim 2, wherein the overlay data provides one or more graphical controls for a computer program executing on the client computing system.

4. The method for generating video for the video game as recited in claim 3, wherein the computer program executing on the client computing system enables audio communication between a player of the video game at the client computing system and one or more other players of the video game.

5. The method for generating video for the video game as recited in claim 4, wherein the one or more other players of the video game are respectively associated with one or more other client computer systems.

6. The method for generating video for the video game as recited in claim 5, wherein the audio communication between the player of the video game at the client computing system and one or more other players of the video game at the one or more other client computer systems is transmitted through the server computing system.

7. The method for generating video for the video game as recited in claim 1, further comprising:

receiving overlay data through a network connection, the overlay data defining the video overlay.

8. The method for generating video for the video game as recited in claim 7, wherein the overlay data provides one or more graphical controls for a computer program executing on a remote computing system.

9. The method for generating video for the video game as recited in claim 8, wherein the computer program executing on the remote computing system enables audio communication between a player of the video game at the client computing system and one or more other players of the video game.

10. The method for generating video for the video game as recited in claim 1, further comprising:

receiving encoded audio data at the client computing system;

decoding the encoded audio data to generate decoded audio data;

detecting activation of a graphical control presented by the video overlay, the graphical control defined to control playback of the decoded audio data; and

upon detecting activation of the graphical control, playing the decoded audio data at the client computing system.

11. The method for generating video for the video game as recited in claim 10, further comprising:

receiving overlay data through a network connection, the overlay data defining the video overlay, wherein the encoded audio data is included in the overlay data.

12. The method for generating video for the video game as recited in claim 1, further comprising:

detecting activation of a graphical control presented by the video overlay, the graphical control defined to establish an audio communication channel between the client computing system and one or more other client computing systems; and

upon detecting activation of the graphical control, establishing the audio communication channel between the client computing system and at least one other client computing system.

13. A video game system, comprising:

an agent configured to determine information about a decoder capability of the video game system and transmit the information about the decoder capability of the video game system to a server computing system;

a network interface configured to receive encoded video data from the server computing system, the encoded video data including an encoded version of video data for a video game representing a game environment rendered based on game rules and one or more user input signals, the encoded video data generated by processing video data for the video game through a dynamically provisioned encoding pipeline that includes an encoder compatible with the decoder capability of the video game system as determined by the agent;

a decoder corresponding to the decoder capability of the video game system as determined by the agent, the decoder configured to decode the received encoded video data to obtain the video data for the video game;

overlay logic configured to apply a video overlay to at least a portion of the video data for the video game to generate modified video data for the video game locally at the video game system, the video overlay sourced independently from the video game, wherein the overlay logic is configured to map pixels of a video overlay image to specific pixels in a video frame defined by the video data for the video game; and

a video output interface configured to provide the modified video data for display on a display device.

14. The video game system as recited in claim 13, wherein the overlay data provides one or more graphical controls for a computer program executing locally on the video game system.

15. The video game system as recited in claim 14, wherein the computer program executing on the video game system enables audio communication between a player of the video game at the video game system and one or more other players of the video game.

16. The video game system as recited in claim 15, wherein the one or more other players of the video game are respectively associated with one or more other video game systems.

17. The video game system as recited in claim 16, wherein the audio communication between the player of the video game at the video game system and one or more other players of the video game at the one or more other video game systems is transmitted through the server computing system.

18. The video game system as recited in claim 13, wherein the video overlay is defined by overlay data sourced at the video game system.

19. The video game system as recited in claim 13, wherein the video overlay is defined by overlay data received through the network interface.