Reduction in latency between user input and visual feedback

• US 9,703,411 B2
• Filed: 04/29/2010
• Issued: 07/11/2017
• Est. Priority Date: 04/30/2009
• Status: Active Grant

• Current Assignee: Wacom Co Limited (Wacom)
• Original Assignee: Synaptics Incorporated
• Inventors: Day, Shawn P., Gillespie, David W., Mackin, Thomas A., Ludden, Christopher A.
• Primary Examiner(s): Ge, Jin
• Application Number:
  US12/770,415
• Publication Number:
  US 20100277505A1
• Time in Patent Office: 2,630 Days
• Field of Search: 345634
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H10K 59/40 OLEDs integrated with touch screens

Abstract

A system comprising a display screen configured for displaying images, a touch sensor having a sensing region, a host processing system and a touch screen control system. The touch screen control system comprising touch sensor control circuitry configured to operate the touch sensor and display control circuitry configured to operate the display screen. The display circuitry comprising a first memory configured to hold a primary image and a second memory configured to hold a secondary image and display refresh circuitry. The display refresh circuitry configured to update the display screen, and in response to the user input and without requiring intervention by the host processing system, generate a blended image comprising the primary image and the secondary image, and update the display screen with the blended image.

Claims

1. An electronic device comprising:

a display screen configured for displaying images;

a touch sensor having a sensing region, wherein said sensing region overlaps with an active area of said display screen;

a host processing system configured for;

primary processing of said images for display on said display screen; and

in response to an initiation of a drag operation, provide, as a secondary image, an image to be dragged in said drag operation, so that said blended image comprises a movement of said image to be dragged with respect to said primary image; and

in response to a termination of said drag operation, update a primary image to include a copy of said image to be dragged at a location defined by said drag operation; and

a touch screen control system comprising;

touch sensor control circuitry configured to operate said touch sensor to detect user input in said sensing region, wherein responsive to said detection said touch sensor control circuitry is configured to provide an indication to said host processing system that causes said host processing system to enter a low power state; and

display control circuitry configured to operate said display screen, said display circuitry comprising;

a first memory configured to hold said primary image provided by said host processing system;

a second memory configured to hold said secondary image; and

display refresh circuitry configured to;

update said display screen with ones of said images processed by said host processing system, and

in response to said user input and without requiring intervention by said host processing system following said detection of said user input;

generate a blended image comprising said primary image and said secondary image, and update said display screen with said blended image, such that said generating a blended image and updating said display screen with said blended image reduces processing by said host processing system by allowing said host processing system to remain in said low power state while said display refresh circuitry generates said blended image and updates said display screen with said blended image.

2. The device of claim 1, wherein said display refresh circuitry is configured to generate said blended image comprising said primary image and said secondary image by:

blending said secondary image at a location relative to said primary image, wherein said location relative to said primary image is based on at least one of a position and a motion of said user input.

3. The device of claim 1, wherein said host processing system is further configured to provide one of a plurality of images as said secondary image based on an application running on said host processing system.

4. The device of claim 1, wherein said host processing system is further configured to provide one of a plurality of images as said secondary image based on an application running on said host processing system having priority.

5. The device of claim 1, wherein said host processing system is further configured to update said primary image or said secondary image at a maximum image data update rate, wherein said display control circuitry is configured to generate and update at a maximum blended image update rate, and wherein said maximum image data update rate is lower than said maximum blended image update rate.

6. A touch screen control system configured to communicate with a host processing system, wherein said host processing system is configured for primary processing of images for display on a display screen, said touch screen control system comprising:

touch sensor control circuitry configured to control touch sensor elements for detecting user input in a sensing region of said touch sensor elements, wherein said sensing region overlaps with an active area of said display screen, wherein responsive to said detection said touch sensor control circuitry is configured to provide an indication to said host processing system that causes said host processing system to enter a low power state; and

display control circuitry configured to control said display screen, said display control circuitry comprising;

a first memory configured to hold a primary image provided by said host processing system;

a second memory configured to hold a secondary image; and

a display refresh circuitry configured to;

update said display screen with ones of said images processed by said host processing system, and

in response to said user input and without intervention by said host processing system following said detection of said user input;

generate a blended image comprising at least portions of said primary image and said secondary image, and

update said display screen with said blended image, such that said generating a blended image and updating said display screen with said blended image reduces processing by said host processing system by allowing said host processing system to remain in said low power state while said display refresh circuitry generates said blended image and updates said display screen with said blended image,

wherein said host processing system is configured to;

provide, in response to an initiation of a drag operation, an image to be dragged in said drag operation as said secondary image, so that said blended image comprises a movement of said image to be dragged with respect to said primary image, and

update, in response to a termination of said drag operation, said primary image to include a copy of said image to be dragged at a location defined by said drag operation.

7. The touch screen control system of claim 6, wherein said touch screen control system is further configured to modify said secondary image in response to said user input without requiring intervention by said host processing system.

8. The touch screen control system of claim 6, wherein said second memory is smaller in memory capacity than said first memory.

9. The touch screen control system of claim 6, wherein said second memory is further configured to hold a tertiary image simultaneously with said secondary image.

10. The touch screen control system of claim 6, wherein said display refresh circuitry is configured to generate said blended image by:

blending said secondary image at a first location with respect to said primary image, and

blending a tertiary image or a second copy of said secondary image at a second location with respect to said primary image.

11. The touch screen control system of claim 6, wherein said display refresh circuitry is configured to generate said blended image by:

blending said secondary image at a location with respect to said primary image, wherein said location with respect to said primary image is based on at least one of a position and a motion of said user input.

12. A display control circuitry comprising:

a first memory configured to hold a primary image for display on a display screen;

a second memory configured to hold a secondary image for display on said display screen; and

display refresh circuitry configured to;

update said display screen with ones of said images processed by and received from a host processing system; and

in response to user input detected in a sensing region of a touch sensor, without requiring intervention by said host processing system, and following said host processing system entering a low power mode based on detection of said user input and without requiring intervention by said host processing system;

generate a blended image comprising said primary image and said secondary image and update said display screen with said blended image, wherein said sensing region of said touch screen overlaps with said active area of said display screen, such that said generating a blended image and updating said display screen with said blended image reduces processing by said host processing system by allowing said host processing system to remain in said low power state while said display refresh circuitry generates said blended image and updates said display screen with said blended image,

wherein said host processing system is configured to;

provide, in response to an initiation of a drag operation, an image to be dragged in said drag operation as said secondary image, so that said blended image comprises a movement of said image to be dragged with respect to said primary image, and

update, in response to a termination of said drag operation, said primary image to include a copy of said image to be dragged at a location defined by said drag operation.

13. The display control circuitry of claim 12, wherein said second memory is smaller in memory capacity than said first memory.

14. The display control circuitry of claim 12, wherein said display refresh circuitry is configured to generate said blended image by:

blending said secondary image at a location with respect to said primary image, wherein said location with respect to said primary image is based on at least one of a position and a motion of said user input.

15. The display control circuitry of claim 12, wherein said user input comprises input by a plurality of input objects, and wherein said display refresh circuitry is configured to generate said blended image by:

blending said secondary image at a first location with respect to said primary image, said first location associated with a position of a first input object of said plurality of input objects, and

blending a tertiary image or a second copy of said secondary image at a second location with respect to said primary image, said second location associated with a position of a second input object of said plurality of input objects.

16. A processor-implemented method for operating a touch screen control system to display images on a display screen, said method comprising:

processing initiation of a drag operation based on said user input;

receiving, as said secondary image, an image to be dragged in said drag operation, such that said blended image comprises a movement of said image to be dragged with respect to said primary image;

processing termination of said drag operation based said user input; and

receiving another primary image held in said first memory, wherein said another primary image includes said image to be dragged at a location reflective of said drag operation;

updating said display screen with ones of said images processed by and received from a host processing system, wherein said host processing system is configured for primary processing of said images for display on said display screen and following said host processing system entering a low power mode based on detection of said user input;

holding said primary image received from said host processing system in a first memory;

holding said secondary image in a second memory; and

in response to sensing user input in a sensing region overlapping an active area of said display screen;

autonomously generating said blended image comprising said primary image and said secondary image; and

autonomously updating said display screen with said blended image, such that said generating a blended image and updating said display screen with said blended image reduces processing by said host processing system by allowing said host processing system to remain in said low power state while blended image is generated and said display screen is updated with said blended image.

17. The processor-implemented method of claim 16, further comprising:

receiving said secondary image from said host processing system.

18. The processor-implemented method of claim 16, further comprising holding a tertiary image in said secondary memory simultaneously with said holding said secondary image, wherein said generating said blended image comprises:

blending said primary image, said secondary image, and said tertiary image.

19. The processor-implemented method of claim 16, further comprising

holding a tertiary image in said secondary memory simultaneously with said holding said secondary image, wherein said generating said blended image comprises:

selecting said secondary image in response to said sensing said user input.

20. The processor-implemented method of claim 16, further comprising:

determining said user input comprises a sequence of user input operations;

generating additional blended images and updating said display screen with said additional blended images without intervention by said host processing system, in response to said sequence of user input operations;

determining an end of said sequence of user input operations; and

providing an indication to said host processing system of said sequence of user input operations, in response to said end.

21. The processor-implemented method of claim 16, further comprising:

initiating an independent blending function in response to a first signal from said host processing system, wherein said touch screen control system generates blended images without requiring intervention by said host processing system for said independent blending function; and

terminating said independent blending function in response to a second signal from said host processing system.

22. The processor-implemented method of claim 16, wherein said primary image comprises an image of a virtual keyboard, and wherein said second memory contains a plurality of key images associated with said virtual keyboard, said method further comprising:

determining user actuation of a key of said virtual keyboard based on said user input, and

selecting a key image of said plurality of key images as said secondary image based on said determining, such that said blended image shows said virtual keyboard with actuation of said key.

23. The processor-implemented method of claim 16, further comprising:

modifying said secondary image in response to said user input, without intervention by said host processing system.

24. The processor-implemented method of claim 23, wherein said modifying said secondary image comprises:

adjusting said secondary image such that said blended image comprises inking associated with a movement of said user input.

25. The processor-implemented method of claim 16, wherein said user input comprises input by a plurality of input objects in said sensing region, and wherein said generating said blended image comprises:

blending said secondary image at a first location relative to said primary image, wherein said first location is based on a first position of a first input object of said plurality of input objects, and

blending a tertiary image or a second copy of said secondary image at a second location relative to said primary image, wherein said second location is based on a second position of a second input object of said plurality of input objects.

26. The processor-implemented method of claim 16, wherein said second memory is configured to hold a plurality of images, each image of said plurality of images associated with at least one gesture of a plurality of user input gestures, said method further comprising:

recognizing said user input as comprising a first gesture of said plurality of user input gestures; and

selecting as said secondary image an image of said plurality of images associated with said first gesture, such that said blended image provides visual feedback of said recognizing.

27. The processor-implemented method of claim 16, wherein said autonomously generating and autonomously updating is performed at a faster rate than a maximum image data update rate, wherein said maximum image data update rate is a fastest rate at which said processing system can update said primary image or said secondary image.