Simultaneous input across multiple applications

• US 7,620,901 B2
• Filed: 03/21/2006
• Issued: 11/17/2009
• Est. Priority Date: 03/21/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sunday, Derek E., Everett, Debora, Whytock, Chris, Kramer, Doug, Carpenter, Curt
• Primary Examiner(s): Lo; Weilun
• Assistant Examiner(s): Chuong; Truc T
• Application Number:
  US11/384,510
• Publication Number:
  US 20070226636A1
• Time in Patent Office: 1,337 Days
• Field of Search: 715/804, 715/864, 715/750, 715/751, 715/754, 715/757, 715/778, 715/782
• US Class Current: 715/754
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

One or more users may interact simultaneously with different applications on the same device through an input system such as a touch-sensitive display screen. Simultaneous user input may be detected by a multiple input system and subsequently transmitted to an application using a single transmission frame. An application corresponding to the user input is determined based on an input location of the user input and a location and size of a corresponding application. Simultaneous user inputs are rendered simultaneously by each corresponding application. Each application may further include a secondary event processing thread that runs in parallel to traditional operating system event threads. The secondary event processing thread bypasses sequential processing limits of an operating system event thread, providing substantially simultaneous processing of input.

Claims

1. In a computing environment, the computing environment comprising a plurality of user input devices, a method for processing inputs from different user input devices across multiple application instances of one or more applications displayed on single display of a user interface, comprising the steps of:

receiving a first user input from a first user input device, the first user input device being associated with a first application instance by subscription of the first application to the first user input device;

simultaneous to receiving the first user input from the first user input device, receiving a different second user input from a separate second user input device, the second user input device being associated with a separate second application instance by subscription of the second application to the second user input device;

determining that the first user input corresponds to the first application instance by determining that first user input was received from the first user input device and by evaluating a first subscription request received from the first application instance subscribing to input received from the first user input device;

routing the first user input to a first input event processing thread associated with the first application instance, wherein the first input event processing thread runs in parallel to a first main event processing thread of the first application instance;

determining a first user input event from the first user input;

rendering the determined first user input event on a user interface of the first application instance;

determining that the second user input corresponds to the second application instance by determining that the second user input was received from the second user input device and by evaluating a second subscription request received from the second application instance subscribing to input received from the second user input device;

routing the second user input to a second input event processing thread associated with the second application instance, wherein the second input event processing thread runs in parallel to a second main event processing thread of the second application instance;

determining a second user input event from the second user input; and

rendering the determined second user input event on a user interface of the second application instance.

2. The method of claim 1, further comprising the steps of:

synchronizing the determined first user input event with one or more events associated with the main event processing thread of the first application instance; and

synchronizing the determined second user input event with one or more events associated with the main event processing thread of the first application instance.

3. The method of claim 2, wherein the steps of synchronizing include determining a sequence of events according to a time of receipt of the user input events and the one or more events associated with the main event processing threads.

4. The method of claim 1, further comprising displaying an indicator on the first user input device indicating that the first user input device is associated with the first application instance and displaying an indicator on the second user input device indicating that the second user input device is associated with the second application instance.

5. The method of claim 1, wherein the first user input device is a different type of device than the second hardware input device.

6. The method of claim 1, wherein the first user input device is a same type of device as the second hardware input device.

7. A multiple-user input system, comprising:

a group of one or more user input sources, wherein the group of one or more user input sources generate a plurality of simultaneous different user inputs;

a computer readable medium storing computer executable instructions that when executed by a processor are configured to implement plurality of application instances of one or more applications, wherein each of the plurality of application instances includes

an input event processing thread running in parallel to a main application instance event thread, and

a user interface; and

a computer readable medium storing computer executable instructions that when executed by a processor are configured to implement an event router, wherein the event router routes each of the plurality of simultaneous user inputs to a destination application instance selected from the plurality of application instances irrespective of an input focus, wherein at least two different inputs are routed to at least two different application instances, wherein each of the plurality of simultaneous user inputs is processed by the input event processing thread of the destination application instance and wherein the plurality of simultaneous user inputs are processed simultaneously.

8. The system of claim 7, wherein the group of input one or more user input sources includes a single multiple input device capable of detecting and transmitting simultaneous user inputs.

9. The system of claim 8, wherein input received from the single input device is routed by an event router based on one or more subscription requests from the plurality of application instances.

10. The system of claim 8, wherein the event router determines the destination application instance based on an input location on the single multiple input device for each of the plurality of user inputs.

11. The system of claim 7, wherein 7, wherein the group of input one or more user input sources includes a plurality of different input devices.

12. In a computing environment, the computing environment comprising a multiple input capable hardware system including a device capable of detecting and transmitting multiple simultaneous user inputs from different users at the same time, a method for processing inputs from different users across multiple application instances of one or more applications displayed on a single display of a user interface, comprising the steps of:

receiving a first user input from a multiple input capable hardware device;

simultaneous to receiving the first user input from the multiple input capable hardware device, receiving a different second user input from the multiple input capable hardware device;

determining that the first user input corresponds to the first application instance;

routing the first user input to a first input event processing thread associated with the first application instance, wherein the first input event processing thread runs in parallel to a first main event processing thread of the first application instance;

determining a first user input event from the first user input;

rendering the determined first user input event on a user interface of the first application instance;

determining that the second user input corresponds to a separate second application instance;

routing the second user input to a separate second input event processing thread associated with the second application instance, wherein the second input event processing thread runs in parallel to a second main event processing thread of the second application instance;

determining a second user input event from the second user input; and

rendering the determined second user input event on a user interface of the second application instance.

13. The method of claim 12, wherein determining that the first user input corresponds to the first application instance comprises determining that the first user input was input at a first input location and wherein determining that the second user input corresponds to a second application instance comprises determining that the second user input was input as a different second input location.

14. The method of claim 12, wherein determining that the first user input corresponds to the first application instance comprises determining the location and size of the first application instance and wherein determining that the second user input corresponds to a second application instance comprises determining the size and location of the second application instance.

15. The method of claim 12, wherein determining that the first user input corresponds to the first application instance and determining that the second user input corresponds to a second application instance comprises determining the type of user input.

16. The method of claim 15, wherein the multiple input capable hardware device comprises a touch screen, and wherein determining the type of user input comprises determining that the first user input is from a finger and the second user input is from a stylus.

17. The method of claim 12 further comprising synchronizing the first and second application instances, by sending the first user input to the second application instance and the second user input to the first application instance.

18. The method of claim 12 further comprising synchronizing the first and second application instances, by sending the first user input and the second user input to a third synchronization application instance.

19. The method of claim 12 further comprising transmitting the first user input and the second user input to a computing device in a single transmission frame.