Touch, pen and force sensor operation with variable refresh displays

• US 9,880,649 B2
• Filed: 09/29/2014
• Issued: 01/30/2018
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Agarwal, Manu, Tann, Christopher, Tripathi, Brijesh, Grunthaner, Martin Paul
• Primary Examiner(s): Edwards, Carolyn R
• Application Number:
  US14/500,653
• Publication Number:
  US 20160092010A1
• Time in Patent Office: 1,219 Days
• Field of Search: 345173-184
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04184 Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally
• CPC Class Codes: G09G 2310/04 Partial updating of the display screen
• CPC Class Codes: G09G 2330/022 in absence of operation, e.g. no data being entered during a predetermined time
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players

Abstract

Synchronization of display functions and various touch, stylus and/or force sensing functions for devices including a variable refresh rate (VRR) display is disclosed. In some examples, touch, stylus and/or force sensing functions can be synchronized with display frames and a display refresh rate can be adjusted by extended blanking of the display for one or more display frames. In other examples, touch, stylus and/or force sensing functions can be synchronized with display sub-frames and a display refresh rate can be adjusted by extended blanking of the display for one or more display sub-frames. Pre-warning synchronization signals can be generated to prepare one or more scan controllers to implement the appropriate scan events during and after extended blanking periods. Latency between the scan results and the corresponding image on the display can be corrected in software and/or firmware by time-stamping scan results or by dropping scan results from uncompleted scans.

Claims

1. An apparatus comprising:

a display;

one or more sensors proximate to the display;

one or more processing circuits capable of;

performing one or more sensing operations for the one or more sensors during a plurality of scanning frames;

performing one or more display operations for the display during a plurality of display frames, the plurality of scanning frames synchronized with the plurality of display frames; and

adjusting a frame refresh rate of the display by blanking the display for one or more display frames during an extended blanking period;

wherein the one or more processing circuits comprises a display controller, a touch controller, and a host processor, and wherein the display controller or the host processor are capable of generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the extended blanking period; and

wherein the second signal does not toggle early.

2. An apparatus comprising:

a display;

one or more sensors proximate to the display; and

one or more processing circuits capable of;

performing one or more sensing operations for the one or more sensors during a plurality of scanning frames;

performing one or more display operations for the display during a plurality of display sub-frames, the plurality of scanning frames synchronized with the plurality of display sub-frames; and

adjusting a frame refresh rate of the display by blanking the display for a duration of one or more display sub-frames between display frames;

wherein the one or more processing circuits comprises a host controller and a touch controller, the host controller capable of generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the blanking of the display for the one or more display sub-frames; and

wherein the second signal does not toggle early.

3. The apparatus of claim 2, wherein the one or more processing circuits are further capable of:

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does not match a duration of an integer number of display frames, resetting the display operations for the display sub-frame following the blanking of the display for the one or more display sub-frames; and

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does match the duration of the integer number of display frames, not resetting the display operations for the display sub-frame following the blanking of the display for the one or more display sub-frames.

4. The apparatus of claim 2, wherein the one or more processing circuits are further capable of:

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does not match a duration of an integer number of display frames, resetting the one or more sensing operations for the display sub-frame following the blanking of the display for the one or more display sub-frames; and

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does match the duration of the integer number of display frames, not resetting the one or more sensing operations for the display sub-frame following the blanking of the display for the one or more display sub-frames.

5. The apparatus of claim 4, wherein the one or more processing circuits are further capable of:

in accordance with the determination that the duration of the blanking of the display for the one or more display sub-frames does not match the duration of the integer number of display frames, discarding results of one or more incomplete sensing operations during the blanking of the display for the one or more display sub-frames.

6. The apparatus of claim 2, wherein the one or more processing circuits are further capable of continuing the display operations and sensing operations following the blanking of the display for the one or more display sub-frames.

7. The apparatus of claim 2, wherein the one or more sensing operations include timing information, the timing information used to align results of the one or more sensing operations with corresponding display information.

8. The apparatus of claim 2, wherein a sensing operation is divided into a plurality of sub-scans occurring during a plurality of sub-frames, such that the sensing operation can be performed during a display frame.

9. A method executed by one or more processing circuits, the method comprising:

performing one or more sensing operations for one or more sensors during a plurality of scanning frames;

performing one or more display operations for a display during a plurality of display frames, the plurality of scanning frames synchronized with the plurality of display frames; and

adjusting a frame refresh rate of the display by blanking the display for one or more display frames during an extended blanking period;

generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the extended blanking period; and

wherein the second signal does not toggle early.

10. The method of claim 9, wherein the signal between the host processor and the display controller is a wake-up interface protocol between the host processor and the display controller.

11. A method executed by one or more processing circuits, the method comprising:

performing one or more sensing operations for one or more sensors during a plurality of scanning frames;

performing one or more display operations for a display during a plurality of display sub-frames, the plurality of scanning frames synchronized with the plurality of display sub-frames; and

adjusting a frame refresh rate of the display by blanking the display for a duration of one or more display sub-frames between display frames;

generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the blanking of the display for the one or more display sub-frames; and

wherein the second signal does not toggle early.

12. The method of claim 11, further comprising:

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does not match a duration of an integer number of display frames, resetting the display operations for the display sub-frame following the blanking of the display for the one or more display sub-frames; and

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does match the duration of the integer number of display frames, not resetting the display operations for the display sub-frame following the blanking of the display for the one or more display sub-frames.

13. The method of claim 11, further comprising:

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does not match a duration of an integer number of display frames, resetting the one or more sensing operations for the display sub-frame following the blanking of the display for the one or more display sub-frames; and

in accordance with a determination that the duration of the blanking of the display for the one or more display sub-frames does match the duration of the integer number of display frames, not resetting the one or more sensing operations for the display sub-frame following the blanking of the display for the one or more display sub-frames.

14. The method of claim 13 further comprising:

in accordance with the determination that the duration of the blanking of the display for the one or more display sub-frames does not match the duration of the integer number of display frames, discarding results of one or more incomplete sensing operations during the blanking of the display for the one or more display sub-frames.

15. The method of claim 11, further comprising continuing the display operations and sensing operations following the blanking of the display for the one or more display sub-frames.

16. The method of claim 11, wherein results of the one or more sensing operations include timing information, the timing information used to align results of the sensing operations with corresponding display information.

17. The method of claim 11, further comprising vertical blanking the display at an end of each display sub-frame.

18. The method of claim 11, wherein a sensing operation is divided into a plurality of sub-scans occurring during a plurality of sub-frames, such that the sensing operation can be performed during a display frame.

19. The method of claim 11, further comprising generating a pre-warning synchronization signal to prepare a force controller for sensing operations during and following the blanking of the display for the one or more display sub-frames.

20. A non-transitory computer readable storage medium, the computer readable medium containing instructions that, when executed by a processor, can perform a method, the method comprising:

performing one or more sensing operations for one or more sensors during a plurality of scanning frames;

performing one or more display operations for a display during a plurality of display frames, the plurality of scanning frames synchronized with the plurality of display frames; and

adjusting a frame refresh rate of the display by blanking the display for one or more display frames during an extended blanking period;

generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the extended blanking period; and

wherein the second signal does not toggle early.

21. A non-transitory computer readable storage medium, the computer readable medium containing instructions that, when executed by a processor, can perform a method, the method comprising:

performing one or more sensing operations for one or more sensors during a plurality of scanning frames;

performing one or more display operations for a display during a plurality of display sub-frames, the plurality of scanning frames synchronized with the plurality of display sub-frames; and

adjusting a frame refresh rate of the display by blanking the display for a duration of one or more display sub-frames between display frames;

generating at least a first signal and a second signal;

wherein the first signal is a pre-warning synchronization signal, which toggles early to prepare the touch controller for sensing operations during and following the blanking of the display for the one or more display sub-frames; and

wherein the second signal does not toggle early.