EVENT REGISTRATION AND DISPATCH SYSTEM AND METHOD FOR MULTI-POINT CONTROLS

• US 20090295746A1
• Filed: 04/29/2009
• Published: 12/03/2009
• Est. Priority Date: 04/29/2008
• Status: Abandoned Application

• Current Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L.
• Application Number:
  US12/432,563
• Publication Number:
  US 20090295746A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Dynamic registration of event handlers in a computer application or operating system recognizes multiple synchronous input streams by identifying each new stroke in a frame representing a single moment in time and mapping in a registration process each identified new stroke to a listening process that is associated with the user interface element to which the new input stream is to be applied. In the same frame, released strokes are unmapped and then each active listening process is called to carry out a respective control operation. When called, the strokes have the correct data for the given frame. The process is repeated for subsequent frames. By carrying out various processes in a sequence of frames, the concept of concurrency is preserved, which is particularly beneficial to multi-touch and multi-user systems.

Claims

1. A method of handling input streams representing strokes on a multi-point input device capable of having multiple synchronous input streams, comprising the steps of:

identifying each new stroke in a given frame and mapping each identified new stroke to one or more listening processes, the given frame representing a single moment in time;

identifying each released stroke in the given frame and unmapping each identified released stroke from the respective listening process to which the stroke previously was mapped;

implementing in the given frame each listening process to which at least one stroke is mapped, each implemented listening process being carried out based on a current state of each stroke mapped to the respective listening process; and

repeating both identifying steps and the implementing step for each of a successive plurality of frames.

2. The method of claim 1, wherein each listening process is associated with a user interface element that includes an associated tangible that dynamically determines whether the new stroke is to be mapped to the respective listening process.

3. The method of claim 2, wherein mapping comprises mapping each identified new stroke based on a positional relationship between a point of contact on the multi-point input device of the identified new stroke and the tangible of each user interface element.

4. The method of claim 1, wherein implementing comprises implementing each listening process to which at least one stroke is mapped based on a current position of contact of each stroke mapped to the respective listening process.

5. The method of claim 1, wherein mapping comprises mapping an identified new stroke to a listening process to which another stroke already is mapped;

and implementing comprises implementing the listening process to which a plurality of strokes are mapped based on current states of said plurality of strokes mapped to the respective listening process.

6. The method of claim 1, wherein mapping comprises mapping an identified new stroke to a plurality of listening processes.

7. The method of claim 1, wherein mapping comprises ascertaining at least one of a plurality of user interface elements to which an identified new stroke is applicable;

and mapping the identified new stroke to a listener process associated with the ascertained user interface element.

8. The method of claim 7, wherein the user interface elements have a hierarchical structure;

and the step of ascertaining employs the hierarchical structure by assessing user interface elements higher in the hierarchical structure before assessing user interface elements lower in the hierarchical structure.

9. The method of claim 1, wherein a first stroke represents a contact of the multi-point input device, and identifying each new stroke comprises identifying a new second stroke in a given frame before the first stroke is released, the first and second strokes being made by two different people.

10. A method of handling input streams in a multi-point input device capable of having multiple synchronous input streams, comprising the steps of:

maintaining a list of one or more user interface elements that respond to contact of the multi-point input device, each of the user interface elements having an associated tangible and at least one associated listener, each listener representing a respective control operation;

processing, for each given frame of a plurality of frames each representing a single moment in time, one or more input streams representing contacts of the multi-point input device by;

identifying each new stroke in the given frame, a new stroke representing a new contact of the multi-point input device;

registering each identified new stroke to the listener of at least one of the user interface elements in the maintained list based on a relationship between a position of contact of the identified new stroke and the tangible of one or more user interface elements;

notifying, for each listener to which a new stroke has been registered in the given frame, the respective listener of the existence of the respective new stroke, each notified listener representing an active listener;

identifying each released stroke in the given frame, a released stroke representing an end of a contact of the multi-point input device;

notifying, for each identified released stroke in the given frame, the respective listener of the discontinuation of the released stroke, each active listener being notified of only released strokes becoming an inactive listener; and

dispatching, in the given frame, each of the active listeners to implement the respective control operation based on an updated state or states of one or more non-released strokes of which the respective listener is notified, the updated state of a non-released stroke corresponding to a current position of contact of the multi-point input device.

11. The method of claim 10, wherein registering comprises registering an identified new stroke to a listener to which a non-released stroke is already registered;

and dispatching comprises implementing the respective control operation corresponding to said listener to which a plurality of non-released strokes are registered based on the updated states of said plurality of non-released strokes.

12. The method of claim 10, wherein registering comprises registering an identified new stroke to a plurality of listeners.

13. The method of claim 10, wherein registering comprises ascertaining at least one of the user interface elements to which the identified new stroke is applicable;

and registering the identified new stroke to the listener of the ascertained user interface element.

14. The method of claim 13, wherein the user interface elements in the list have a hierarchical structure;

and the step of ascertaining at least one of the user interface elements to which the identified new stroke is applicable employs the hierarchical structure by assessing user interface elements higher in the hierarchical structure before assessing user interface elements lower in the hierarchical structure.

15. The method of claim 10, wherein a first stroke represents a contact of the multi-point input device by a first person, and a second stroke represents a new contact of the multi-point input device by a second person, and the second stroke is identified as a new stroke in a frame in which the first stroke is a non-released stroke.

16. A multi-point input system, comprising:

a multi-point input device having an input surface and adapted to sense simultaneous contacts of the input surface and providing as an output data streams representing strokes on the input surface over a period of time; and

a controller for receiving the data streams from the multi-point input device, carrying out a set of processes during a given frame representing a single moment in time, and repeating the set of processes for each of a successive plurality of frames over said period of time, the set of processes including;

identifying each new stroke in a given frame and mapping each identified new stroke to one or more listening processes;

identifying each released stroke in the given frame and unmapping each identified released stroke from the respective listening process to which the stroke previously was mapped; and

implementing in the given frame each listening process to which at least one stroke is mapped, each implemented listening process being carried out based on a current state of each stroke mapped to the respective listening process.

17. The system of claim 16, wherein each listening process is associated with a user interface element that includes an associated tangible that dynamically determines whether the new stroke is to be mapped to the respective listening process.

18. The system of claim 17, wherein the controller carries out the mapping process by mapping each identified new stroke based on a positional relationship between a point of contact on the input surface of the identified new stroke and the tangible of each user interface element.

19. The system of claim 16, wherein the controller carries out the implementing process by implementing each listening process to which at least one stroke is mapped based on a current position of contact of each stroke mapped to the respective listening process.

20. The system of claim 16, wherein the controller carries out the mapping process by mapping an identified new stroke to a listening process to which another stroke already is mapped, and carries out the implementing process by implementing the listening process to which a plurality of strokes are mapped based on current states of said plurality of strokes mapped to the respective listening process.

21. The system of claim 16, wherein the controller carries out the mapping process by mapping an identified new stroke to a plurality of listening processes.

22. The system of claim 16, wherein the controller carries out the mapping process by ascertaining at least one of a plurality of user interface elements to which an identified new stroke is applicable, and mapping the identified new stroke to a listener process associated with the ascertained user interface element.

23. The system of claim 22, wherein the user interface elements have a hierarchical structure;

and the controller employs the hierarchical structure by assessing user interface elements higher in the hierarchical structure before assessing user interface elements lower in the hierarchical structure.

24. The system of claim 16, wherein the multi-point input device is adapted to sense simultaneous contacts by different users on the input surface.

25. The system of claim 16, wherein the multi-point input device includes a display surface distinct from the input surface;

and the controller controls the multi-point input device to display on the display surface at least one user interface element, and at least one listening process implemented by the controller modifies an appearance of the user interface element displayed on the display surface.

26. The system of claim 16, wherein the input surface is adapted to operate as a display surface for the multi-point input device;

and the controller controls the multi-point input device to display on the display surface at least one user interface element, and at least one listening process implemented by the controller modifies an appearance of the user interface element displayed on the display surface.

27. A multi-point input system, comprising:

a multi-point input device having an input surface and adapted to sense simultaneous contacts of the input surface and providing as an output data streams representing strokes on the input surface over a period of time; and

a controller for maintaining a list of one or more user interface elements that respond to contact of the input surface and that have an associated tangible and at least one associated listener representing a respective control operation, receiving the data streams from the multi-point input device, carrying out a set of processes during a given frame representing a single moment in time, and repeating the set of processes for each of a successive plurality of frames over said period of time, the set of processes including;

identifying each new stroke in the given frame, a new stroke representing a new contact of the input surface;

registering each identified new stroke to the listener of at least one of the user interface elements in the maintained list based on a relationship between a position of contact of the identified new stroke and the tangible of each user interface element;

notifying, for each listener to which a new stroke has been registered in the given frame, the respective listener of the existence of the respective new stroke, each notified listener representing an active listener;

identifying each released stroke in the given frame, a released stroke representing an end of a contact of the input surface;

notifying, for each identified released stroke in the given frame, the respective listener of the discontinuation of the released stroke, each active listener being notified of only released strokes becoming an inactive listener; and

dispatching, in the given frame, each of the active listeners to implement the respective control operation based on an updated state or states of one or more non-released strokes of which the respective listener is notified, the updated state of a non-released stroke corresponding to a current position of contact of the input surface.

28. The system of claim 27, wherein the controller carries out the registering process by registering an identified new stroke to a listener to which a non-released stroke is already registered;

and the controller carries out the dispatching process by implementing the respective control operation corresponding to said listener to which a plurality of non-released strokes are registered based on the updated states of said plurality of non-released strokes.

29. The system of claim 27, wherein the controller carries out the registering process by registering an identified new stroke to a plurality of listeners.

30. The system of claim 27, wherein the controller carries out the registering process by ascertaining at least one of the user interface elements to which the identified new stroke is applicable, and registering the identified new stroke to the listener of the ascertained user interface element.

31. The system of claim 30, wherein the user interface elements in the list have a hierarchical structure;

and the controller ascertains at least one of the user interface elements to which the identified new stroke is applicable using the hierarchical structure by assessing user interface elements higher in the hierarchical structure before assessing user interface elements lower in the hierarchical structure.

32. The system of claim 27, wherein a first stroke represents a contact of the input surface by a first person, and a second stroke represents a new contact of the input surface by a second person, and the controller identifies the second stroke as a new stroke in a frame in which the first stroke is a non-released stroke.