System for combining recorded application state with application streaming interactive video output

• US 8,832,772 B2
• Filed: 12/05/2007
• Issued: 09/09/2014
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: OL2 Incorporated (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): Pendleton, Brian T
• Assistant Examiner(s): Saint Cyr, Jean D
• Application Number:
  US11/999,472
• Publication Number:
  US 20090125968A1
• Time in Patent Office: 2,470 Days
• Field of Search: 725133-134
• US Class Current: 725/134
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/21815 comprising local storage units
• CPC Class Codes: H04N 21/231 Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/47217 : for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04N 9/79 : Processing of colour television signals in connection with recording
• CPC Class Codes: H04N 9/87 : Regeneration of colour television signals H04N9/80 takes precedence

Abstract

An apparatus for recording application state information over time while simultaneous recording application streaming interactive video output. A playback means in which the recorded application video may be played back or the application may be re-executed with the application state information.

Claims

1. A computer-implemented method comprising:

running a full twitch video game on one or more servers of a hosting service center entirely, wherein none of the processes is executed by any of one or more client devices;

the full twitch video game being played or used by one or more users on one or more client devices remote to the hosting service center via compressed streaming interactive video transmitted over the Internet, the compressed streaming interactive video being compressed with a worst case, roundtrip latency of 90 ms, from a user control input to display of a response to the user control input on a client device of a user, over a transmission distance of;

up to and including 1500 miles;

continually recording, in a memory, game state information of the full twitch video game along with the compressed streaming interactive video as the full twitch video game is being played.

2. The computer-implemented method of claim 1 further comprising using the recorded game state information to reproduce video of the full twitch video game.

3. The computer-implemented method of claim 2 further comprising providing the reproduced video to a user with Digital Video Recorder (DVR) control functionality.

4. The computer-implemented method of claim 3 wherein the DVR control functionality includes pause, rewind, and fast forward functions.

5. The computer-implemented method of claim 1 further comprising running a user interface (UI) application on another server of the hosting service center, the UI application allowing another user to control visual presentation aspects of the full twitch video game while the recorded game state is being played.

6. The computer-implemented method of claim 5 wherein the visual presentation aspects include a camera view.

7. The computer-Implemented method of claim 6 wherein different video is generated when the game state information is being played and the camera view changes.

8. The computer-implemented method of claim 1 further comprising allowing the user to specify a game state segment to record and retain.

9. The computer-implemented method of claim 8 further comprising making the game state segment available on a user page.

10. The computer-implemented method of claim 9 further comprising allowing additional users to control aspects of the full twitch video game while simultaneously viewing the game state segment on the user page.

11. The computer-implemented method of claim 1 wherein the game state information includes video output by the full twitch video games streamed to the users of the full twitch video games.

12. The computer implemented method of claim 1 wherein the game state information includes different video output by the full twitch video games than that streamed to the users of the full twitch video games.

13. The computer-implemented method of claim 2 wherein the reproduced video is uncompressed.

14. The computer-implemented method of claim 2 wherein the reproduced video is compressed.

15. A computer implemented method comprising:

running a full twitch video game on one or more servers of a hosting service center entirely, wherein none of the processes is executed by any of one or more client devices, the full twitch game being played or used by one or more users on one or more client devices remote to the hosting service center via compressed streaming interactive video transmitted over the Internet;

providing an additional application that allows other users to participate in the twitch video game as spectators; and

wherein the compressed streaming interactive video is compressed with a worst-case, roundtrip latency of 90 ms, from a user control input to display of a response to the user control input on a client device of a user, over a transmission distance of up to and including 1500 miles.

16. The computer-implemented method of claim 15 wherein the additional application allows the other users to view video of game play.

17. The computer implemented method of claim 15 wherein the additional application allows the other users to control aspects of video of the full twitch video game.

18. The computer-implemented method of claim 17 wherein the aspects includes a camera view.

19. A nontransitory computer-readable storage medium encoded with a computer program, which, when executed by at least one processor, is operable to:

run a full twitch video game on one or more servers in a hosting service center entirely, wherein none of the processes is executed by any of one or more client devices;

the full twitch video game being played by one or more users on one or more client devices remote to the hosting service center via compressed streaming interactive video transmitted over the Internet, wherein the compressed streaming interactive video is compressed with;

a worst-case, round-trip latency of 90 ms, from a user control input to display of a response to the user control input on a client device of a user, over a transmission distance of up to and including 1500 miles; and

provide an additional application that allows other users to participate in the full twitch video game as spectators.

20. The non-transitory computer-readable storage medium of claim 19 wherein the additional application allows the other users to vies video of game play.

21. The nontransitory computer-readable storage medium of claim 19 wherein the full twitch video game allows the other users to control aspects of video of the full twitch video game.

22. The non-transitory computer-readable storage medium of claim 21 wherein the aspects includes a camera view.