PROCESSING DIGITAL INK INPUT SUBJECT TO MONITORING AND INTERVENTION BY AN APPLICATION PROGRAM

• US 20160263911A1
• Filed: 08/19/2015
• Published: 09/15/2016
• Est. Priority Date: 03/12/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sun, Yibo, Weins, Connor, Menon, Krishnan, Tu, Xiao, Durojaiye, Olumuyiwa
• Granted Patent: US 9,950,542 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B41J 2/49 by writing
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04162 for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

A facility for processing ink input is described. In one example facility, the facility receives ink input from an input device. The facility generates and renders ink stroke data structures in response to received ink input in accordance with an ink input processing pipeline. The facility provides to an executing application access to information traversing the ink input processing pipeline at a selected point in the ink input processing pipeline.

Claims

1. A computing system for processing ink input, comprising:

at least one processor configured to execute an application;

an input device interface configured to receive ink input;

an ink input processing subsystem configured to generate and render ink stroke data structures based at least on ink input received by the input device interface in accordance with an ink input processing pipeline; and

an ink input processing pipeline access subsystem configured to provide to the application executed by the at least one processor access to information traversing the ink input processing pipeline at a selected point in the ink input processing pipeline.

2. The computing system of claim 1 wherein the application executed by the at least one processor comprises an event handler, and the ink input processing pipeline access subsystem is further configured to provide access via the event handler.

3. The computing system of claim 1 wherein the selected point in the ink input processing pipeline is a point in the ink input processing pipeline before generation of ink stroke data structures.

4. The computing system of claim 3, further comprising:

an ink input processing control subsystem configured to;

receive from the application executed on the at least one processor an instruction, and based at least on the instruction, omit generating and rendering ink stroke data structures with respect to a selected portion of ink input received by the input device interface.

5. The computing system of claim 3 wherein the application executed by the at least one processor comprises an event handler,

the ink input processing pipeline access subsystem is further configured to provide access via the event handler,

and the event handler is executed by the at least one processor in a low-latency thread.

6. The computing system of claim 5 wherein the low-latency thread in which the event handler is executed also executes code for the ink input processing subsystem.

7. The computing system of claim 5, further comprising:

a monitoring subsystem configured to monitor the event handler'"'"'s use of processing resources in the low-latency thread.

8. The computing system of claim 1 wherein the selected point in the ink input processing pipeline is a point in the ink input processing pipeline beyond generation of ink stroke data structures.

9. The computing system of claim 1 wherein the selected point in the ink input processing pipeline is a point in the ink processing pipeline traversed by ink input of one or more types not used by the ink input processing system to generate and render ink stroke data structures.

10. A method in a computing system for processing ink input in connection with an application program, the method comprising:

permitting the application program to register an event handler for input events;

permitting the application program to register an event handler for stroke events;

receiving input events from an input device;

based at least on the application registering an event handler for input events, providing access to the received input events to the registered event handler for input events;

processing at least a portion of the input events to generate a plurality of ink strokes; and

based at least on the application registering the event handler for stroke events, providing access to the generated strokes to the registered event handler for stroke events.

11. The method of claim 10, further comprising:

permitting the application program to register an event handler for non-stroke events; and

based at least on the application registering the event handler for non-stroke events, providing access to received input events not used in generating ink strokes to the registered event handler for non-stroke events.

12. The method of claim 10, further comprising

receiving from the application program a direction to cancel a selected ink stroke;

and

based at least on receiving the direction, canceling the selected ink stroke.

13. The method of claim 12, further comprising:

causing the selected ink stroke to be de-rendered; and

deleting the selected ink stroke.

14. The method of claim 12 wherein the selected ink stroke is incomplete at a time when the direction is received.

15. A method of claim 10, further comprising:

based at least on the application registering the event handler for input events, regulating the consumption of processing resources by the registered event handler for input events.

16. The method of claim 10 wherein providing access to the generated strokes to the registered event handler for stroke events comprises storing information about the generated strokes in a queue to which the registered event handler for stroke events has access.

17. A computer-readable medium having contents configured to cause a computing system to, in order to handle inking input:

in an operating system;

invoke an application;

receive a stream of inking input events;

for each inking input event received by the operating system, before generating ink based on the inking input event, provide to code specified as part of the invoked application access to the inking input event; and

for each of at least a first portion of the inking input events received by the operating system, after providing to code specified as part of the invoked application access to the inking input event, generate ink based on the inking input event.

18. The computer-readable medium of claim 17 wherein the computer-readable medium has contents configured to further cause a computing system to:

in the operating system,

receive from the code specified as part of the invoked application a direction with respect to a distinguished inking input event to which access was provided, wherein, based at least on receiving the direction, generation of ink based on the distinguished inking input event in the operating system is performed in accordance with the received direction.

19. The computer-readable medium of claim 17 wherein the computer-readable medium has contents configured to further cause a computing system to:

in the operating system,

receive from the code specified as part of the invoked application a direction with respect to a distinguished inking input event to which access was provided, wherein, based at least on receiving the direction, the distinguished inking input event is not among the inking input events for which ink generation is performed in the operating system.

20. The computer-readable medium of claim 17 wherein the computer-readable medium has contents configured to further cause a computing system to:

in the operating system;

for each of at least a second portion of the inking input events received by the operating system, render ink based on the inking input event; and

receive from the code specified as part of the invoked application a direction with respect to a distinguished inking input event to which access was provided, wherein, based at least on receiving the direction, the distinguished inking input event is not among the inking input events for which ink rendering is performed in the operating system.