Enhancing user experience by presenting past application usage

• US 9,409,090 B1
• Filed: 12/30/2013
• Issued: 08/09/2016
• Est. Priority Date: 12/09/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: McKenzie, Bruce J., Overton, Adam J., Fisher, Brian D., Shepard, Isaac J., JnBaptiste, Eden Ashley, Zhang, Eric M., Jenks, Jason C., McCuller, Patrick G.
• Primary Examiner(s): Myhr, Justin
• Application Number:
  US14/143,795
• Time in Patent Office: 953 Days
• Field of Search: 463 7- 9, 463/42
• US Class Current: 1/1
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/69 by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/0489 using dedicated keyboard keys or combinations thereof
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066

Abstract

Past usage of an application is presented in order to enhance the experience of a user with an application. An application is executed in a hosted environment in one or more computing devices. Input commands may be obtained from a client and provided to the application. A real-time state of the application may be recorded as the application is executed in the hosted environment. The real-time state of the application that has been recorded may be replayed.

Claims

1. A non-transitory computer-readable medium embodying a program that, when executed by at least one computing device, causes the at least one computing device to at least:

receive a request from a client to launch an application;

execute the application in a hosted environment in the at least one computing device;

determine that a user of the client is experiencing difficulty in a session of the application;

select a recorded application state of the application from a plurality of past session application states based at least in part on a state of the application; and

replay the recorded application state.

2. The non-transitory computer-readable medium of claim 1, wherein the state of the application comprises a plurality of input commands.

3. The non-transitory computer-readable medium of claim 1, wherein the state of the application comprises a video signal obtained from a wrapper application and generated by the application.

4. The non-transitory computer-readable medium of claim 3, wherein the program further causes the at least one computing device to:

determine the wrapper application, the wrapper application configured to provide a virtualized environment for execution of the application;

determine a video encoder based at least in part on the request from the client;

obtain the video signal from the wrapper application;

generate an encoded video signal by encoding the video signal with the video encoder; and

send the encoded video signal to the client as the application is executed.

5. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to send a recorded video signal to another client.

6. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to select the state of the application to be recorded based at least in part on a completed checkpoint in the application.

7. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to obtain a plurality of input commands from the client and provide the plurality of input commands to the application as the application is executed within the hosted environment.

8. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to send a combined media stream to the client, the combined media stream comprises the recorded application state and a current application state.

9. A system, comprising:

a data store comprising a plurality of applications and a plurality of wrapper applications; and

at least one computing device in communication with the data store, the at least one computing device being configured to at least;

execute an application in a hosted environment in the at least one computing device;

determine that a user is experiencing difficulty in a session of the application;

select a recorded application state corresponding to a past session of the application based at least in part on a state of the application; and

replay the recorded application state based at least in part on the state of the application.

10. The system of claim 9, wherein the recorded application state is selected based at least in part on an achievement in the application reflected by the recorded application state.

11. The system of claim 9, wherein the at least one computing device is further configured to determine that the user is experiencing difficulty in the session of the application based at least in part on an input command history of the user associated with the session.

12. The system of claim 9, wherein the recorded application state comprises a recorded media stream.

13. The system of claim 12, wherein the at least one computing device is further configured to send a combined media stream to a client, the combined media stream including the recorded media stream and a real-time media stream corresponding to a current state of the application.

14. A method, comprising:

sending, by a first computing device, a plurality of input commands over a network to a program executing on a second computing device, wherein the plurality of input commands are configured to be provided by the program to an application executed in a hosted environment in the second computing device;

receiving, by the first computing device, first application state data from the program executing on the second computing device, the first application state data presenting current usage of the application;

determining, by the first computing device, that a user is experiencing difficulty in a session of the application;

receiving, by the first computing device, second application state data from the program executing on the second computing device, the second application state data presenting past usage of the application; and

rendering, upon a display of the first computing device, a video signal presenting both the current usage of the application and the past usage of the application based at least in part on the first application state data and the second application state data.

15. The method of claim 14, further comprising requesting, by the first computing device, the second application state data based at least in part on at least one achievement completed in the past usage of the application.

16. The method of claim 14, further comprising requesting, by the first computing device, the second application state data based at least in part on the first application state data.

17. The method of claim 14, wherein the application comprises a game application.

18. The method of claim 14, wherein the first application state data and the second application state data comprise respective media streams.

19. The method of claim 14, wherein the first application state data and the second application state data comprise a single media stream.

20. The method of claim 14, further comprising:

receiving, by the first computing device, a user-specified selection criterion from the user;

sending, by the first computing device, the user-specified selection criterion to the program executing on the second computing device; and

wherein the program executing on the second computing device is configured to select the second application state data from a library of recorded application states based at least in part on the user-specified selection criterion.