Efficient Frame Rendering

• US 20160119669A1
• Filed: 01/06/2016
• Published: 04/28/2016
• Est. Priority Date: 06/24/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Park, Keun-Young
• Granted Patent: US 9,894,401 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/127 Updating a frame memory using a transfer of data from a source area to a destination area
• CPC Class Codes: G09G 2370/06 Consumer Electronics Control, i.e. control of another device by a display or vice versa
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: G09G 5/005 Adapting incoming signals to the display format of the display terminal
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/41422 located in transportation means, e.g. personal vehicle arrangements specially adapted for transportation systems in broadcast systems H04H20/62
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/436 Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally

Abstract

A user may interact with an interactive remote display (e.g., touch input) and the interaction may be sent to an application on a mobile device. The mobile device (application and/or encoder) may render frames for the display when there is a change. Latency in the pipeline may be alleviated by determine a rendering rate for an application. A dummy frame may be sent to the encoder if the application is not a constant rendering type of application. The dummy frames may be sent subsequent to the rendered frames to flush the pipeline.

Claims

1-14. -14. (canceled)

15. A computer-implemented method, comprising:

receiving by a device from a component of an infotainment system that is remote from the device and that is incorporated in a motor vehicle, an indicator of an input at the component that is associated with a change to a display of an application on the component;

determining by the device, a rendering rate for the application;

in response to the determined rendering rate, generating by the device, a dummy frame;

rendering by the device, a plurality of frames comprising the dummy frame; and

providing by the device to the component, the plurality of rendered frames.

16. The method of claim 15, wherein the plurality of frames comprises the dummy frame and a viewable frame.

17. The method of claim 15, wherein the input comprises a touch of a display component of the infotainment system.

18. The method of claim 15, further comprising, prior to generating the dummy frame, determining by the device that the rendering rate is below a threshold level.

19. The method of claim 15, further comprising, prior to generating the dummy frame, determining by the device that the rendering rate is non-constant.

20. The method of claim 15, further comprising, prior to rendering the plurality of frames, providing by the device to an encoder of the device, the plurality of frames comprising the dummy frame.

21. The method of claim 15, wherein providing the plurality of rendered frames comprises providing the plurality of rendered frames to a decoder of the component.

22. The method of claim 15, wherein the plurality of rendered frames updates a portion of the display of the application on the component.

23. A device comprising:

a processor; and

a non-transitory, computer-readable medium in communication with the processor and storing instructions that, when executed by the processor, cause the processor to perform operations comprising;

receiving from a component of an infotainment system that is remote from the device and that is incorporated in a motor vehicle, an indicator of an input at the component that is associated with a change to a display of an application on the component, wherein the application is executing on the device,

determining a rendering rate for the application,

in response to the determined rendering rate, generating a dummy frame,

rendering a plurality of frames comprising the dummy frame, and

providing to the component, the plurality of rendered frames.

24. The device of claim 23, further comprising an encoder in communication with the processor,

wherein the operations further comprise providing to the encoder, a plurality of frames comprising the dummy frame, and

wherein rendering comprises rendering by the encoder.

25. The device of claim 23, further comprising an encoder in communication with the processor,

wherein generating comprises generating by the encoder, and

wherein rendering comprises rendering by the encoder.

26. The device of claim 23, wherein rendering comprises filling a pipeline with the plurality of frames comprising the dummy frame.

27. The device of claim 23, further comprising an encoder in communication with the processor,

wherein the operations further comprise providing to the encoder, a plurality of frames comprising the dummy frame, and

wherein rendering comprises filling a pipeline of the encoder with the plurality of frames comprising the dummy frame.

28. The device of claim 23, wherein providing the plurality of rendered frames comprises sending the plurality of rendered frames to the component in accordance with a projected architecture.

29. A non-transitory, computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

receiving from a component of an infotainment system that is remote from the device and that is incorporated in a motor vehicle, an indicator of an input at the component that is associated with a change to a display of an application on the component;

determining a rendering rate for the application;

determining a quantity of dummy frames based on the determined rendering rate;

generating a set of dummy frames based on the determined quantity of dummy frames;

rendering a plurality of frames comprising the set of dummy frames; and

providing to the component, the plurality of rendered frames.

30. The non-transitory, computer-readable medium of claim 29, wherein determining the quantity of dummy frames is further based on a quantity of stages of a pipeline.

31. The non-transitory, computer-readable medium of claim 29, wherein determining the quantity of dummy frames is further based on a quantity of stages of a pipeline and a quantity of viewable frames.

32. The non-transitory, computer-readable medium of claim 29, wherein rendering comprises filling a pipeline.

33. The non-transitory, computer-readable medium of claim 29, wherein rendering comprises filling a pipeline with the set of dummy frames and a viewable frame.

34. The non-transitory, computer-readable medium of claim 29, wherein rendering comprises ordering each of the set of dummy frames and a viewable frame in a pipeline.