APPLICATION MIRRORING USING MULTIPLE GRAPHICS CONTEXTS

• US 20120081353A1
• Filed: 09/27/2011
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: IMERJ LLC (Flex, Ltd.)
• Inventors: Yusupov, Alisher, Reeves, Paul E., Chincisan, Octavian, Liu, Wuke
• Granted Patent: US 9,060,006 B2
• Time in Patent Office: Days
• US Class Current: 345/418
• CPC Class Codes: G06F 2209/545 Gui
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45537 Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Cross-environment rendering and user interaction support provide a seamless computing experience in a multi-operating system computing environment. Cross-environment rendering provides real-time display of applications running in a mobile operating system to be displayed within an environment of a desktop operating system. The mobile operating system and the desktop operating system may be running concurrently and independently on a shared kernel of a mobile computing device. A graphics server of the mobile operating system tears down and rebuilds the rendering context for each application as it composites the surface information. The rendering context may be established to match the resolution of the associated display, so that graphics will be appropriately rendered for that resolution. The mobile computing device may be a smartphone running the Android mobile operating system and a full desktop Linux distribution on a modified Android kernel.

Claims

1. A method comprising:

maintaining display of application graphics for a first application and a second application, both compiled for and in active concurrent execution within a first operating system, by iteratively performing steps comprising;

establishing a first rendering context of the first operating system;

calculating updates to surfaces of the first application using the first operating system;

rendering the updated surfaces of the first application in the first rendering context of the first operating system to generate a first graphics frame in a first memory location;

disestablishing the first rendering context;

establishing a second rendering context of the first operating system;

calculating updates to surfaces of the second application using the first operating system;

rendering the updated surfaces of the second application in the second rendering context of the first operating system to generate a second graphics frame in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel; and

disestablishing the second rendering context.

2. The method of claim 1, wherein the maintaining display of application graphics for the first application and the second application is by iteratively performing steps further comprising:

displaying the first graphics frame from the first memory location to a display of a first computing environment associated with the first operating system; and

displaying the second graphics frame from the second memory location to a display of a second computing environment associated with the second operating system.

3. The method of claim 1, wherein the maintaining display of application graphics for the first application and the second application further comprises receiving, by the console application, a draw notification indicating that the second graphics frame is available at the second memory location.

4. The method of claim 3, wherein the receiving of the draw notification includes receiving a file descriptor to the second memory location.

5. The method of claim 3, wherein the draw notification is received through an inter-process communication channel.

6. The method of claim 3, further comprising receiving a reference to the second memory location by mapping the file descriptor to the process space of the console application.

7. The method of claim 1, wherein the second application is incompatible with a graphics server of the second operating system.

8. The method of claim 1, wherein the second application comprises an Android application and the graphics server of the second operating system comprises an X-windows graphics server.

9. The method of claim 1, wherein the second rendering context is established according to characteristics of a display of a second computing environment associated with the second operating system.

10. A mobile computing device, comprising:

a first application running within a first operating system, the first application calculating updates for a first graphics surface using the first operating system;

a second application running within the first operating system, the second application calculating updates for a second graphics surface using the first operating system; and

a graphics server of the first operating system, wherein the graphics server establishes a first rendering context of the first operating system, renders updated surfaces of the first application through the first rendering context to generate a first graphics frame in a first memory location, disestablishes the first rendering context, establishes a second rendering context of the first operating system, renders the updated surfaces of the second application through the second rendering context to generate a second graphics frame in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel, and disestablishes the second rendering context.

11. The mobile computing device of claim 10, further comprising a first computing environment associated with the first operating system, the first graphics frame displayed to a display of the first computing environment.

12. A computing environment including the mobile computing device of claim 11, further comprising a second computing environment associated with the second operating system, the second graphics frame displayed to a display of the second computing environment.

13. The mobile computing device of claim 10, further comprising a mobile processor, the first and second operating systems running concurrently on the mobile processor.

14. The mobile computing device of claim 10, further comprising a console application running in the second operating system, wherein the graphics server of the first operating system passes a draw notification to the console application indicating that the second graphics frame is available at the second memory location before disestablishing the second rendering context.

15. The mobile computing device of claim 14, wherein the draw notification includes a file descriptor to the second memory location.

16. The mobile computing device of claim 10, wherein a graphics server of the second operating system is an X-windows type graphics server and wherein the second application uses a graphics library of the first operating system that is incompatible with the X-windows type graphics server of the second operating system.

17. A computing device including a computer-readable medium storing instructions for a physical processor, the instructions, when executed, causing the processor to perform steps comprising:

maintaining display of application graphics for a first application and a second application, both compiled for and in active concurrent execution within a first operating system, by iteratively performing steps comprising;

establishing a first rendering context of the first operating system;

calculating updates to surfaces of the first application using the first operating system;

rendering the updated surfaces of the first application in the first rendering context of the first operating system to generate a first graphics frame in a first memory location;

disestablishing the first rendering context;

establishing a second rendering context of the first operating system;

calculating updates to surfaces of the second application using the first operating system;

rendering the updated surfaces of the second application in the second rendering context of the first operating system to generate a second graphics frame in a second memory location, the second memory location being a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel; and

disestablishing the second rendering context.