Low latency modification of display frames

• US 9,721,365 B2
• Filed: 03/31/2015
• Issued: 08/01/2017
• Est. Priority Date: 12/09/2014
• Status: Active Grant

• Current Assignee: Wacom Co Limited (Wacom)
• Original Assignee: Synaptics Incorporated
• Inventors: Worfolk, Patrick
• Primary Examiner(s): Moon, Seokyun
• Assistant Examiner(s): Shen, Peijie
• Application Number:
  US14/675,693
• Publication Number:
  US 20160162061A1
• Time in Patent Office: 854 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G09G 2320/0252 Improving the response speed
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04

Abstract

This disclosure generally provides an input device that includes a by-pass path for improving latency between a touch controller and a display driver. In one embodiment, the by-pass path directly connects the touch controller and display driver, thereby by-passing a host processor (e.g., a central processing unit (CPU) or graphic processing unit (GPU)) in the input device. By doing so, the input device can detect user input and update a display to reflect the user input faster than if the display driver had to wait until updated display frames are sent from the host processor. For example, the host processor may transmit touch data received from the touch controller to an application that uses the touch data to change the display. In some input devices, this process can take hundreds of milliseconds, which may be enough time for the user to notice a delay.

Claims

1. A processing system for displaying modified display frame data on a display, the processing system comprising:

a touch controller configured to detect an input object in a sensing region and, in response, transmit touch data to a host processor and to an overlay engine, wherein the overlay engine is configured to;

receive a first display frame and modify the first display frame with temporary image data based on the received touch data;

assign respective timers to different pixels of the temporary image data, wherein the respective timers are activated at different times; and

remove the different pixels from the temporary image data when the respective timers satisfy a predefined time window; and

a display driver configured to prepare the modified first display frame for output to the display.

2. The processing system of claim 1, wherein the overlay engine receives trace control data from the host processor, wherein the temporary image data comprises an image of a first trace, wherein the trace control data describes characteristic of the first trace.

3. The processing system of claim 2, wherein the trace control data describes a diffusion rate at which a width of the first trace expands over time.

4. The processing system of claim 3, wherein the first trace simulates diffusion of wet ink while a second trace in a second display frame received from the host processor simulates dry ink to replace the different pixels removed from the temporary image data.

5. The processing system of claim 1, further comprising:

a data interface configured to transmit the touch data from the touch controller to the host processor; and

a communication channel configured to transmit the touch data from the touch controller to the overlay engine, wherein the communication channel by-passes the host processor.

6. The processing system of claim 5, wherein the touch data transmitted from the touch controller to the host processor using the data interface is the same as the touch data transmitted from the touch controller to the overlay engine using the communication channel.

7. The processing system of claim 1, wherein the touch controller and overlay engine are one of:

(i) integrated onto a common integrated circuit and (ii) integrated onto separate integrated circuits that are communicatively coupled.

19. The processing system of claim 1, wherein the overlay engine is configured to:

receive, from the host processor, location data indicating locations of a plurality of virtual keys of a virtual keyboard currently being outputted on the display; and

determine that a user has pressed one of the plurality of virtual keys by comparing the location data of the plurality of virtual keys to a location in the touch data, wherein the temporary image data is part of a displayed animation indicating the user has pressed one of the plurality of virtual keys.

8. A method, comprising:

detecting an input object in a sensing region;

transmitting touch data to a host processor and to an overlay engine in response to detecting the input object;

receiving, at the overlay engine, a first display frame;

modifying, using the overlay engine, the first display frame with temporary image data based on the touch data;

assigning respective timers to different pixels of the temporary image data, wherein the respective timers are activated during different times;

removing the different pixels from the temporary image data when the respective timers satisfy a predefined time window; and

preparing the modified first display frame for output to a display.

9. The method of claim 8, further comprising:

receiving at the overlay engine trace control data transmitted by the host processor, wherein the temporary image data comprises an image of a first trace and wherein the trace control data describes characteristics of the first trace.

10. The method of claim 9, wherein the trace control data describes a diffusion rate at which a width of the first trace expands over time.

11. The method of claim 10, wherein the first trace simulates diffusion of wet ink while a second trace in a second display frame received from the host processor simulates dry ink to replace the different pixels removed from the temporary image data.

12. The method of claim 8, wherein transmitting the touch data to the overlay engine comprises:

transmitting the touch data on a communication channel between a touch controller and the overlay engine that by-passes the host processor.

13. The method of claim 12, wherein the touch data transmitted from the touch controller to the host processor is the same as the touch data transmitted from the touch controller to the overlay engine.

20. The method of claim 8, further comprising:

receiving, from the host processor, location data indicating locations of a plurality of virtual keys of a virtual keyboard currently being outputted on the display; and

determining that a user has pressed one of the plurality of virtual keys by comparing the location data of the plurality of virtual keys to a location in the touch data, wherein the temporary image data is part of a displayed animation indicating the user has pressed one of the plurality of virtual keys.

14. A processing system, comprising:

a touch controller configured to detect an input object in a sensing region, and in response, transmit touch data to a host processor and to an overlay engine;

the overlay engine configured to;

receive a display frame from the host processor;

update the display frame with a stored action based on the received touch data, wherein, when displayed, the stored action modifies pixel data in the display frame;

assign respective timers to different pixels of the modified pixel data, wherein the respective timers are activated during different times; and

remove the different pixels from the modified pixel data when the respective timers satisfy a predefined time window; and

a display driver configured to prepare the updated display frame for output to a display.

15. The processing system of claim 14, further comprising:

a communication channel coupling the touch controller to the overlay engine, wherein the communication channel by-passes the host processor.

16. The processing system of claim 14, wherein the overlay engine comprises a buffer configured to store the stored action.

17. The processing system of claim 16, wherein the overlay engine is configured to use the buffer to update received display frames based on a control signal received from the host processor, wherein the control signal activates the overlay engine thereby instructing the overlay engine to begin using the stored action to replace the pixel data.

18. The processing system of claim 14, wherein the stored action is one of:

drawing a trace, a graphic indicating a displayed button was activated, and scrolling.