CROSS-ENVIRONMENT USER INTERFACE MIRRORING

• US 20120081380A1
• Filed: 09/28/2011
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: IMERJ LLC (Flex, Ltd.)
• Inventors: Reeves, Brian, Reeves, Paul E., Teltz, Richard, Reeves, David, Sirpal, Sanjiv, Tyghe, Chris, Chincisan, Octavian
• Granted Patent: US 8,957,905 B2
• Time in Patent Office: Days
• US Class Current: 345/541
• CPC Class Codes: G06F 2209/545 Gui
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45537 Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A mobile computing device with a mobile operating system and desktop operating system running concurrently and independently on a shared kernel without virtualization. The mobile operating system provides a user experience for the mobile computing device that suits the mobile environment. The desktop operating system provides a full desktop user experience when the mobile computing device is docked to a second user environment. Cross-environment rendering and user interaction support provide a seamless computing experience in a multi-operating system computing environment. The seamless computing experience includes mirroring the active user interaction space of the mobile operating system to a display of a user environment associated with the desktop operating system. The mobile computing device may be a smartphone running the Android mobile operating system and a full desktop Linux distribution on a modified Android kernel.

Claims

1. A method comprising:

calculating, using a first operating system, updates to a set of surfaces of a first application compiled for and in active execution within the first operating system;

rendering, using the first operating system, the set of surfaces according to the updates to generate a graphics frame;

storing the graphics frame in a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel;

displaying the graphics frame to a first application display of the first application on a first display of a first computing environment using the first operating system;

passing a file descriptor indicating the shared memory location to a console application compiled for and in active execution within the second operating system; and

displaying the graphics frame from the shared memory location to a second application display of the first application on a second display of a second computing environment via the console application of the second operating system according to the file descriptor, such that the second application display is substantially identical to the first application display.

2. The method of claim 1, wherein a graphics server of the second operating system is incompatible with a drawing object instantiated by the first application.

3. The method of claim 1, wherein a graphics server of the first operating system is incompatible with a graphics server of the second operating system.

4. The method of claim 1, wherein the first operating system comprises a mobile operating system and the second operating system comprises a desktop operating system.

5. The method of claim 1, wherein the first operating system comprises an Android mobile operating system and a graphics server of the second operating system comprises an X-windows graphics server.

6. The method of claim 1, further comprising mapping, by the console application, the file descriptor to the process space of the console application to generate a reference to the shared memory location.

7. The method of claim 6, further comprising accessing, by the console application, the graphics frame from the shared memory location via the reference to the shared memory location.

8. The method of claim 1, wherein the set of surfaces of the first application are rendered through a first context space of an extended rendering context of the first operating system.

9. A mobile computing device comprising:

a first application compiled for and in active execution within a first operating system;

a graphics frame for the first application rendered in a shared memory location by the first operating system, the shared memory location accessible by both the first operating system and a second operating system running concurrently with the first operating system on a shared kernel;

a first application display of the first application displayed on a first display of a first computing environment using the first operating system; and

a console application running within the second operating system that displays the graphics frame to a second application display on a display of a second computing environment associated with the second operating system by accessing the shared memory location of the graphics frame through a file descriptor to the graphics frame passed by the first operating system.

10. The mobile computing device of claim 9, wherein a graphics server of the second operating system is an X-windows type graphics server and wherein the first application uses a graphics library of the first operating system that is incompatible with the X-windows type graphics server of the second operating system.

11. The mobile computing device of claim 9, wherein the shared memory location comprises a frame buffer of the first operating system.

12. A computing device including a computer-readable medium storing instructions for a physical processor, the instructions, when executed, causing the processor to perform steps comprising:

calculating, using a first operating system, updates to a set of surfaces of a first application compiled for and in active execution within the first operating system;

rendering, using the first operating system, the set of surfaces according to the updates to generate a graphics frame;

storing the graphics frame in a shared memory location accessible by both the first operating system and a second operating system running concurrently on a shared kernel;

displaying the graphics frame to a first application display of the first application on a first display of a first computing environment using the first operating system;

passing a file descriptor indicating the shared memory location to a console application compiled for and in active execution within the second operating system; and

displaying the graphics frame from the shared memory location to a second application display of the first application on a second display of a second computing environment via the console application of the second operating system according to the file descriptor, such that the second application display is substantially identical to the first application display.

13. The computing device of claim 12, wherein a graphics server of the second operating system is incompatible with a drawing object instantiated by the first application.

14. The computing device of claim 12, wherein a graphics server of the first operating system is incompatible with a graphics server of the second operating system.

15. The computing device of claim 12, wherein the first operating system comprises a mobile operating system and the second operating system comprises a desktop operating system.

16. The computing device of claim 12, wherein the first operating system comprises an Android mobile operating system and a graphics server of the second operating system comprises an X-windows graphics server.

17. The computing device of claim 12, wherein the instructions, when executed, cause the processor to further perform steps comprising mapping, by the console application, the file descriptor to the process space of the console application to generate a reference to the shared memory location.

18. The computing device of claim 17, wherein the instructions, when executed, cause the processor to further perform steps comprising accessing, by the console application, the graphics frame from the shared memory location via the reference to the shared memory location.

19. The computing device of claim 12, wherein the set of surfaces of the first application are rendered through a first context space of an extended rendering context of the first operating system.