EXTENDED GRAPHICS CONTEXT WITH COMMON COMPOSITING

• US 20120081396A1
• Filed: 09/27/2011
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: IMERJ LLC (Flex, Ltd.)
• Inventors: Yusupov, Alisher, Reeves, Paul E., Chincisan, Octavian, Liu, Wuke
• Granted Patent: US 9,077,731 B2
• Time in Patent Office: Days
• US Class Current: 345/635
• CPC Class Codes: G06F 2209/545 Gui
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45537 Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Cross-environment rendering and user interaction support provide a seamless computing experience in a multi-operating system computing environment. The multi-operating system computing environment may include a mobile operating system and a desktop operating system running concurrently and independently on a mobile computing device. The mobile operating system may be associated with a first user environment and the desktop operating system may be associated with a second user environment. Real-time display of applications running in the mobile operating system within the second user environment is provided by rendering the application through an extended graphics context of the mobile operating system. Application graphics for multiple applications are rendered into portions of a single graphics frame. The mobile computing device may be a smartphone running the Android mobile operating system and a full desktop Linux distribution on a modified Android kernel.

Claims

1. A method for maintaining display of application graphics for a first application and a second application, both compiled for and in active concurrent execution within a first operating system, the method comprising:

establishing an extended rendering context of the first operating system having a first context space of the extended rendering context associated with the first application and a second context space of the extended rendering context associated with the second application, the first and second context spaces being non-overlapping;

calculating updates to surfaces of the first application and the second application using the first operating system; and

rendering the updated surfaces using the first operating system to generate an extended graphics frame in a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel, such that a first portion of the extended graphics frame is associated with the first context space and a second portion of the extended graphics frame is associated with the second context space, and such that the updated surfaces of the first application are rendered to the first portion of the extended graphics frame and the updated surfaces of the second application are rendered to the second portion of the extended graphics frame.

2. The method of claim 1, further comprising:

displaying the first portion of the extended graphics frame associated with the first context space from the shared memory location to a display of a first computing environment associated with the first operating system; and

displaying the second portion of the extended graphics frame associated with the second context space from the shared memory location to a display of a second computing environment associated with the second operating system.

3. The method of claim 1, wherein the first operating system comprises a mobile operating system and the second operating system comprises a desktop operating system.

4. The method of claim 1, wherein the second application is incompatible with a graphics server of the second operating system.

5. The method of claim 1, wherein the second application comprises an Android application and a graphics server of the second operating system comprises an X-windows graphics server.

6. The method of claim 1, further comprising receiving, by a console application of the second operating system, a draw notification indicating that the second portion of the extended graphics frame is available at the shared memory location.

7. The method of claim 6, wherein the receiving of the draw notification includes receiving a file descriptor to the shared memory location.

8. The method of claim 7, wherein the draw notification is received through an inter-process communication channel.

9. The method of claim 7, further comprising receiving, by the console application, a reference to the shared memory location by mapping the file descriptor to the process space of the console application.

10. The method of claim 9, further comprising accessing, by the console application, the second portion of the graphics frame from the shared memory location through the reference.

11. The method of claim 1, further comprising registering, in a graphics server of the first operating system, a console application of the second operating system to receive draw notifications for the second application.

12. The method of claim 1, wherein the rendering of the updated surfaces using the first operating system uses hardware assisted rendering of the first operating system.

13. A mobile computing device comprising:

a first application and a second application in active concurrent execution within a first operating system;

a first graphics surface of the first operating system updated by the first application;

a second graphics surface of the first operating system updated by the second application;

an extended rendering context of the first operating system, the extended rendering context including a first context space associated with the first application and a second context space of the extended rendering context associated with the second application, the first and second context spaces being non-overlapping; and

a graphics server of the first operating system that renders updated surface information of the first surface and the second surface to generate an extended graphics frame in a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel, such that a first portion of the extended graphics frame is associated with the first context space and a second portion of the extended graphics frame is associated with the second context space, and such that updated surfaces of the first application are rendered to the first portion of the extended graphics frame and the updated surfaces of the second application are rendered to the second portion of the extended graphics frame.

14. The mobile computing device of claim 13, further comprising a console application running in the second operating system that accesses the shared memory location to display the second portion of the extended graphics frame on a display of a user environment associated with the second operating system.

15. The mobile computing device of claim 13, wherein a graphics server of the second operating system is an X-windows type graphics server and wherein the second application uses a graphics library of the first operating system that is incompatible with the X-windows type graphics server of the second operating system.

16. The mobile computing device of claim 13, further comprising a first computing environment associated with the first operating system, the first application displayed on a display of the first computing environment.

17. A computing device including a computer-readable medium storing instructions for a physical processor, the instructions, when executed, causing the processor to perform steps comprising:

executing a first application and a second application concurrently on a first operating system;

establishing an extended rendering context of the first operating system, the first operating system having a first context space of the extended rendering context associated with the first application and a second context space of the extended rendering context associated with the second application, the first and second context spaces being non-overlapping;

calculating updates to surfaces of the first application and the second application using the first operating system; and

rendering the updated surfaces using the first operating system to generate an extended graphics frame in a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel, such that a first portion of the extended graphics frame is associated with the first context space and a second portion of the extended graphics frame is associated with the second context space, and such that the updated surfaces of the first application are rendered to the first portion of the extended graphics frame and the updated surfaces of the second application are rendered to the second portion of the extended graphics frame.