Distributed cross-platform user interface and application projection

• US 10,942,735 B2
• Filed: 09/30/2013
• Issued: 03/09/2021
• Est. Priority Date: 12/04/2012
• Status: Active Grant

• Current Assignee: Abalta Technologies, Inc.
• Original Assignee: Abalta Technologies, Inc.
• Inventors: Stankoulov, Pavel
• Primary Examiner(s): Nguyen, Dustin
• Application Number:
  US14/042,365
• Publication Number:
  US 20140156734A1
• Time in Patent Office: 2,717 Days
• Field of Search: 709203, 709213, 709232, 455 306, 345634, 345173, 715751, 715769, 715772, 725 80, 370338
• CPC Class Codes: G06F 9/00 Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41422 located in transportation means, e.g. personal vehicle arrangements specially adapted for transportation systems in broadcast systems H04H20/62
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor

Abstract

A mobile device application adapted to provide multimedia content to a target canvas is described. The application includes sets of instructions for: establishing a communication link with an external system associated with the target canvas; rendering multimedia content for playback by the target canvas; and sending the rendered multimedia content to the target canvas over the communication link. A mobile device application adapted to execute web-based applications in a browser associated with an external system includes sets of instructions for: establishing a communication link with the external system; accessing a web-based application; and rendering content associated with the application and sending the rendered content to the external system for display in the browser. A system adapted to provide multimedia content includes: a target adapted to display multimedia content; a host adapted to generate multimedia content; and a remote server adapted to at least partially control the display of multimedia content.

Claims

1. An automated method that provides multimedia content to a target canvas, the automated method comprising:

establishing a communication link from a host mobile device to an external system associated with the target canvas;

rendering, at the host mobile device, multimedia content for playback by the target canvas, wherein the multimedia content is rendered off screen into screen buffers comprising sets of vector commands to be transferred to and displayed by the target canvas based at least partly on a resolution of the target canvas, wherein the rendering comprises drawing individual graphical user interface elements and content of each screen of the multimedia content to an off-screen canvas having a same resolution as the resolution of the target canvas, wherein the off-screen canvas is converted into the sets of vector commands by capturing vector information associated with scaled and calculated hypertext markup language (HTML) pages and constructing a vector description at the host mobile device by transforming the captured vector information based at least partly on the resolution of the target canvas;

encoding the screen buffers into host command messages, each host command message comprising a particular encoded screen buffer; and

sending the host command messages from the host mobile device to the external system over the communication link.

2. The automated method of claim 1 further comprising:

rendering the multimedia content for playback by an alternative target canvas of the external system; and

sending the rendered host commands to the alternative target canvas over the communication link.

3. The automated method of claim 1, wherein rendering multimedia content for playback by the target canvas comprises:

determining a bandwidth across the communication link;

determining a main application rendering rate; and

encoding the multimedia content based at least partly on at least one of the bandwidth and the main application rendering rate.

4. The automated method of claim 1, wherein the external system is a vehicle display unit or a medical device.

5. The automated method of claim 1 further comprising:

receiving a user input action from the external system across the communication link;

converting the user input action into a client command;

sending the client command across the communication link;

receiving the client command at the host mobile device;

processing the client command into a user action;

providing the user action to an application running on the host mobile device;

generating an output of the application running on the host mobile device; and

modifying the rendered multimedia content based at least partly on the output of the application running on the host mobile device.

15. The automated method of claim 1 further comprising:

receiving, at the external system, the host commands sent from the host mobile device; and

processing the rendered content included in the host commands and displaying the rendered content to the target canvas.

18. The automated method of claim 1, wherein the sets of vector commands comprise a fill rectangle command associated with a full screen bitmap.

6. A mobile device that executes web-based applications in a web browser associated with an external system, the mobile device comprising:

a processor for executing sets of instructions and

a non-transitory medium that stores the sets of instructions, wherein the sets of instruction comprise;

establishing a communication link between the mobile device and the external system;

accessing, at the mobile device, a particular web-based application;

rendering, off-screen at the mobile device, a user interface associated with the particular web-based application, wherein the rendering comprises generating screen buffers comprising sets of vector commands using a web browser running on the mobile device at an appropriate resolution for the external system by capturing vector information associated with scaled and calculated hypertext markup language (HTML) pages and constructing a vector description at the host mobile device by transforming the captured vector information based at least partly on a resolution of a display associated with the external system;

encoding the screen buffers into host command messages; and

sending the host command messages to the external system for display.

7. The mobile device of claim 6, wherein the sets of instructions further comprise:

receiving a user input action from the external system;

converting the user input action into a client command;

sending the client command across the communication link;

receiving the client command at a host mobile device;

processing the client command into a user action; and

providing the user action to the particular web-based application at the host mobile device.

8. The mobile device of claim 6, wherein the communication link with the external system comprises a wireless connection.

9. The mobile device of claim 6, wherein the external system is a vehicle display unit and wherein the mobile device is a smartphone.

10. The mobile device of claim 6, wherein the particular web-based application is accessed via at least one cellular communication network.

16. The mobile device of claim 6, wherein the sets of instructions further comprise:

establishing a communication link between the mobile device and a remote server; and

accessing, at the mobile device, the particular web-based application via the remote server.

19. The mobile device of claim 6, wherein the sets of vector commands comprise a fill rectangle command associated with a full screen bitmap.

11. A system that provides multimedia content to a user, the system comprising:

a target comprising a target display that displays multimedia content; and

a host that generates multimedia content for display by the target by rendering the multimedia content based at least partly on a resolution of the target display, wherein;

the multimedia content is rendered, at the host, into a plurality of screen buffers, each screen buffer including a set of vector commands by capturing vector information associated with scaled and calculated hypertext markup language (HTML) pages and constructing a vector description at the host by transforming the captured vector information based at least partly on a resolution of the target display,

the screen buffers are encoded into host command messages,

the host command messages are sent from the host to the target over a communication link, and

the target interacts with the host across the communication link to at least partially control the display of multimedia content.

12. The system of claim 11, wherein the host is a mobile device and the target is a vehicle display unit.

13. The system of claim 11 further comprising a remote server that interacts with the host to at least partially control the display of multimedia content, wherein the host and the remote server communicate across at least one wireless connection.

14. The system of claim 11, wherein the host and the target communicate across at least one wireless connection.

17. The system of claim 13, wherein the target comprises at least one user interface element and wherein:

user input actions are received via the at least one user interface element,

user input actions are converted into client commands,

client commands are sent across the communication link and received at the host,

client commands are processed into user actions, and

user actions are provided to the particular web-based application at the remote server.

20. The system of claim 11, wherein the set of vector commands comprises a fill rectangle command associated with a full screen bitmap.