INSTANT REMOTE RENDERING

• US 20120086717A1
• Filed: 09/27/2011
• Published: 04/12/2012
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: IMERJ LLC (Flex, Ltd.)
• Inventors: Liu, Wuke
• Granted Patent: US 9,047,102 B2
• Time in Patent Office: Days
• US Class Current: 345/564
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G09G 5/395 Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen G09G5/399 takes precedence
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers

Abstract

Cross-environment rendering and user interaction support provide a seamless computing experience in a multi-operating system computing environment. The multi-operating system computing environment may include a mobile operating system and a desktop operating system running concurrently and independently on a mobile computing device. Real-time or instant display of an application running in the mobile operating system within an environment of the desktop operating system is provided by rendering application graphics for the application within the desktop operating system. A console application of the desktop operating system may access surface information for the application from shared memory and render the application within a console window of the computing environment associated with the desktop operating system. The mobile computing device may be a smartphone running the Android mobile operating system and a full desktop Linux distribution on a modified Android kernel.

Claims

1. A method comprising:

maintaining display of application graphics for a first application and a second application, both compiled for and in active concurrent execution within a first operating system, by iteratively performing steps comprising;

calculating updates to surfaces of the first application using the first operating system;

rendering the updated surfaces of the first application using the first operating system to generate a first graphics frame in a first memory location;

calculating updates to surfaces of the second application using the first operating system;

storing the updated surfaces of the second application in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel; and

rendering the updated surfaces of the second application using a console application of the second operating system to generate a second graphics frame in a third memory location.

2. The method of claim 1, further comprising:

displaying the first graphics frame from the first memory location to a display of a first computing environment associated with the first operating system; and

displaying the second graphics frame from the third memory location to a display of a second computing environment associated with the second operating system.

3. The method of claim 2, wherein the second graphics frame is displayed from the third memory location to a display of a second computing environment associated with the second operating system via the console application.

4. The method of claim 1, further comprising receiving, by the console application, a draw notification indicating that updated surfaces are available at the second memory location.

5. The method of claim 4, wherein the receiving of the draw notification includes receiving a file descriptor to the second memory location.

6. The method of claim 4, wherein the draw notification is received through an inter-process communication channel.

7. The method of claim 4, further comprising receiving a reference to the second memory location by mapping the file descriptor to the process space of the console application.

8. The method of claim 7, wherein rendering the updated surfaces of the second application includes reading the second memory location through the reference.

9. The method of claim 1, further comprising:

instantiating a renderer object in the console application; and

registering, in a graphics server of the first operating system, a remotable interface of the renderer object to receive draw notifications for the surfaces of the first application.

10. The method of claim 1, wherein the first operating system comprises a mobile operating system and the second operating system comprises a desktop operating system.

11. The method of claim 1, wherein the second application is incompatible with a graphics server of the second operating system.

12. The method of claim 1, wherein the second application comprises an Android application and a graphics server of the second operating system comprises an X-windows graphics server.

13. The method of claim 1, wherein rendering the updated surfaces of the second application includes compositing the updated surfaces by a layer order.

14. A mobile computing device comprising:

a first application and a second application in active concurrent execution within a first operating system;

a first graphics surface associated with the first application;

a graphics server of the first operating system operable to render the first graphics surface to generate a first graphics frame in a first memory location;

a second graphics surface associated with the second application, the second graphics surface stored in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel;

a console application running within the second operating system, the console application operable to render the second graphics surface to generate a second graphics frame in a third memory location.

15. The mobile computing device of claim 14, further comprising a renderer object instantiated by the console application that includes a remotable method for rendering the second graphics surface to generate the second graphics frame.

16. The mobile computing device of claim 14, wherein the console application receives a draw notification from the graphics server of the first operating system indicating that the second graphics surface has been updated.

17. The mobile computing device of claim 14, wherein a graphics server of the second operating system is an X-windows type graphics server and wherein the second application uses a graphics library of the first operating system that is incompatible with the X-windows type graphics server of the second operating system.

18. A computing device including a computer-readable medium storing instructions for a physical processor, the instructions, when executed, causing the processor to perform steps comprising:

maintaining display of application graphics for a first application and a second application, both compiled for and in active concurrent execution within a first operating system, by iteratively performing steps comprising;

calculating updates to surfaces of the first application using the first operating system;

rendering the updated surfaces of the first application using the first operating system to generate a first graphics frame in a first memory location;

calculating updates to surfaces of the second application using the first operating system;

storing the updated surfaces of the second application in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel; and

rendering the updated surfaces of the second application using a console application of the second operating system to generate a second graphics frame in a third memory location.