System and method for providing interactive content to non-native application environments

• US 8,903,897 B2
• Filed: 07/06/2012
• Issued: 12/02/2014
• Est. Priority Date: 07/08/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Relan, Rohan, Gupta, Rajat, Ramavajjala, Chaitanya, Chang, Chuo-Ling, Wu, Ping-Hao, Tan, Peter
• Primary Examiner(s): Dalencourt, Yves
• Application Number:
  US13/543,720
• Publication Number:
  US 20130013671A1
• Time in Patent Office: 879 Days
• Field of Search: 709/203
• US Class Current: 709/203
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

An automated electronic application delivery system for delivery software applications to client devices for which they were not originally intended for or developed. The applications are run on the particular application environment at the server side, and the content output by the application is capture, encoded, and streamed to the client devices in a way that minimizes latency. User interacts with the content and such interaction data is either mapped to a specific command catered to the application either at the client side or the server side. The mapped command or raw interaction data is then provided to the application over a control channel.

Claims

1. In a system including a plurality of host devices, a client device, and a server coupled to the plurality of host devices and the client device over a data communications network, each host device hosting a software application configured to run on an application environment provided by the host device, and the client device receiving interactive content generated by a particular software application configured to run on an application environment different than an application environment provided by the client device, the server comprising:

a processor; and

a memory coupled to the processor and storing program instructions therein, the processor being configured to execute the program instructions, the program instructions comprising;

receiving from the client device identification of the particular software application;

identifying one of the plurality of host devices hosting the particular software application;

invoking the particular software application on the application environment provided by the identified host device;

periodically capturing and encoding a screen display output generated by the particular software application into a plurality of video frames;

streaming the plurality of video frames over the data communications network during a streaming session;

receiving interaction data from the client device in response to the streamed video frames;

converting the interaction data to a corresponding input catered to the particular software application;

providing the input to the particular software application, wherein the software application is configured to generate a second screen output in response;

identifying a configuration file associated with the particular software application,

wherein a user input is mapped to a particular command based on the configuration file; and

receiving the user input or the particular command from the client device as the interaction data.

2. The server of claim 1, wherein the client device is a mobile phone or electronic tablet.

3. The server of claim 2, wherein the software application is a Flash-based application, Windows application, Java-based application, or Linux application.

4. The server of claim 1, wherein the second screen is output in real-time with the providing of the interaction data by a user of the client device.

5. The server of claim 1, wherein the program instructions further include:

monitoring condition of the data communications network during the streaming session; and

dynamically adjusting a visual quality of one or more of the plurality of video frames during the streaming session.

6. The server of claim 5, wherein the program instructions for dynamically adjusting the visual quality includes dynamically adjusting a frame rate associated with the streaming session.

7. The server of claim 5, wherein the program instructions for dynamically adjusting the visual quality includes dynamically adjusting a video bit rate associated with the streaming session.

8. The server of claim 5, wherein the program instructions for dynamically adjusting the visual quality includes dynamically adjusting a maximum frame size of a video frame to be streamed during the streaming session.

9. The server of claim 5, wherein a maximum frame size is calculated based on a dynamically determined frame rate and video bit rate.

10. The server of claim 5, wherein the monitoring of the condition of the data communications network includes monitoring congestion of the network.

11. The server of claim 1, wherein the application environment provided by the host device provides an application programming interface not available to the application environment provided by the client device.

12. In a system including a plurality of host devices, a client device, and a server coupled to the plurality of host devices and the client device over a data communications network, each host device hosting a software application configured to run on an application environment provided by the host device, and the client device receiving interactive content generated by a particular software application configured to run on an application environment different than an application environment provided by the client device, a method for providing interactive content to the client device, the method comprising:

receiving from the client device identification of the particular software application;

identifying one of the plurality of host devices hosting the particular software application;

invoking the particular software application on the application environment provided by the identified host device;

periodically capturing and encoding a screen display output generated by the particular software application into a plurality of video frames;

streaming the plurality of video frames over the data communications network during a streaming session;

receiving interaction data from the client device in response to the streamed video frames;

converting the interaction data to a corresponding input catered to the particular software application;

providing the input to the particular software application, wherein the software application is configured to generate a second screen output in response;

identifying a configuration file associated with the particular software application,

wherein a user input is mapped to a particular command based on the configuration file; and

receiving the user input or the particular command from the client device as the interaction data.

13. The method of claim 12, wherein the client device is a mobile phone or electronic tablet.

14. The method of claim 13, wherein the software application is a Flash-based application, Windows application, Java-based application, or Linux application.

15. The method of claim 12, wherein the second screen is output in real-time with the providing of the interaction data by a user of the client device.

16. The method of claim 13 further comprising:

monitoring condition of the data communications network during the streaming session; and

dynamically adjusting a visual quality of one or more of the plurality of video frames during the streaming session.

17. The method of claim 16, wherein the dynamically adjusting the visual quality includes dynamically adjusting a frame rate associated with the streaming session.

18. The method of claim 16, wherein the dynamically adjusting the visual quality includes dynamically adjusting a video bit rate associated with the streaming session.