Simultaneous input across multiple applications

• US 8,347,215 B2
• Filed: 11/11/2009
• Issued: 01/01/2013
• Est. Priority Date: 03/21/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Carpenter, Curt, Everett, Debora, Kramer, Doug, Whytock, Chris, Sunday, Derek E
• Primary Examiner(s): Lo, Weilun
• Assistant Examiner(s): CHUONG, TRUC T
• Application Number:
  US12/616,667
• Publication Number:
  US 20100053110A1
• Time in Patent Office: 1,147 Days
• Field of Search: 715/702, 715750-752, 715/754, 715/778, 715/864
• US Class Current: 715/754
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

One or more users may interact simultaneously with different applications on the same device through an input system such as a touch-sensitive display screen. Simultaneous user input may be detected by a multiple input system and subsequently transmitted to an application using a single transmission frame. An application corresponding to the user input is determined based on an input location of the user input and a location and size of a corresponding application. Simultaneous user inputs are rendered simultaneously by each corresponding application. Each application may further include a secondary event processing thread that runs in parallel to traditional operating system event threads. The secondary event processing thread bypasses sequential processing limits of an operating system event thread, providing substantially simultaneous processing of input.

Claims

1. In a computing environment, the computing environment comprising a multiple input capable hardware system including a device capable of detecting and transmitting multiple simultaneous user inputs at the same time, a method for processing different inputs across multiple components of one or more applications displayed on a single display of a user interface, the method comprising:

receiving a first user input from a multiple input capable hardware device having a touch-sensitive display;

simultaneous to receiving the first user input from the multiple input capable hardware device, receiving a second user input from the multiple input capable hardware device, both of the first and second user input being entered at and detected by the touch-sensitive display at substantially the same time;

determining that the first user input corresponds to a first component instance;

routing the first user input to a first input event processing thread associated with a first component instance;

determining a first user input event from the first user input;

rendering the determined first user input event on a user interface of the first component instance;

determining that the second user input corresponds to a separate second component instance;

routing the second user input to a separate second input event processing thread associated with the second component instance;

determining a second user input event from the second user input; and

rendering the determined second user input event on a user interface of the second component instance, wherein the first input event processing thread runs in parallel to a first main operating system event processing thread of the first component instance and wherein the second input event processing thread runs in parallel to a second main operating system event processing thread.

2. The method of claim 1, wherein determining that the first user input corresponds to the first component instance comprises determining that the first user input was input at a first input location and wherein determining that the second user input corresponds to a second component instance comprises determining that the second user input was input as a different second input location.

3. The method of claim 1, wherein determining that the first user input corresponds to the first component instance comprises determining the location and size of the first component instance and wherein determining that the second user input corresponds to a second component instance comprises determining the size and location of the second component instance.

4. The method of claim 1, wherein determining that the first user input corresponds to the first component instance and determining that the second user input corresponds to a second component instance comprises determining the type of user input.

5. The method of claim 4, wherein the multiple input capable hardware device comprises a touch screen, and wherein determining the type of user input comprises determining that the first user input is from a finger and the second user input is from a stylus.

6. The method of claim 1, further comprising synchronizing the first and second component instances, by sending the first user input to the second component instance and the second user input to the first component instance.

7. The method of claim 1, further comprising synchronizing the first and second component instances, by sending the first user input and the second user input to a third synchronization component instance.

8. The method of claim 1, further comprising transmitting the first user input and the second user input to a computing device in a single transmission frame.

9. In a computing environment, the computing environment comprising a multiple input capable hardware system including a device capable of detecting and transmitting multiple simultaneous user inputs at the same time, a computer readable storage medium comprising computer executable instructions that when executed by one or more processors perform the following acts to process different inputs across multiple components of one or more applications displayed on a single display of a user interface:

receiving a first user input from a multiple input capable hardware device having a touch-sensitive display;

simultaneous to receiving the first user input from the multiple input capable hardware device, receiving a different second user input from the multiple input capable hardware device, both of the first and second user input being entered at and detected by the touch-sensitive display at substantially the same time;

determining that the first user input corresponds to a first component instance;

routing the first user input to a first input event processing thread associated with a first component instance;

determining a first user input event from the first user input;

rendering the determined first user input event on a user interface of the first component instance;

determining that the second user input corresponds to a separate second component instance;

routing the second user input to a separate second input event processing thread associated with the second component instance, wherein the second input event processing thread runs in parallel to a second main operating system event processing thread of the second component instance;

determining a second user input event from the second user input; and

rendering the determined second user input event on a user interface of the second component instance, wherein the first input event processing thread runs in parallel to a first main operating system event processing thread of the first component instance.

10. The computer readable medium of claim 9, wherein determining that the first user input corresponds to the first component instance comprises determining that the first user input was input at a first input location and wherein determining that the second user input corresponds to a second component instance comprises determining that the second user input was input as a different second input location.

11. The computer readable medium of claim 9, wherein determining that the first user input corresponds to the first component instance comprises determining the location and size of the first component instance and wherein determining that the second user input corresponds to a second component instance comprises determining the size and location of the second component instance.

12. The computer readable medium of claim 9, wherein determining that the first user input corresponds to the first component instance and determining that the second user input corresponds to a second component instance comprises determining the type of user input.

13. The computer readable medium of claim 12, wherein the multiple input capable hardware device comprises a touch screen, and wherein determining the type of user input comprises determining that the first user input is from a finger and the second user input is from a stylus.

14. The computer readable medium of claim 9, further comprising computer executable instructions that when executed by one or more processors perform synchronizing the first and second component instances, by sending the first user input to the second component instance and the second user input to the first component instance.

15. The computer readable medium of claim 9, further comprising computer executable instructions that when executed by one or more processors perform synchronizing the first and second component instances, by sending the first user input and the second user input to a third synchronization component instance.

16. The computer readable medium of claim 9, further comprising computer executable instructions that when executed by one or more processors perform the transmitting the first user input and the second user input to a computing device in a single transmission frame.

17. In a computing environment, the computing environment comprising a multiple input capable hardware system including a device capable of detecting and transmitting multiple simultaneous user inputs at the same time, a computer system comprising one or more processors and computer memory, the computer memory comprising computer executable instructions that when executed by the one or more processors perform the following acts to process different inputs across multiple components of one or more applications displayed on a single display of a user interface:

receiving a first user input from a multiple input capable hardware device having a touch-sensitive display;

simultaneous to receiving the first user input from the multiple input capable hardware device, receiving a different second user input from the multiple input capable hardware device, both of the first and second user input being entered at and detected by the touch-sensitive display at substantially the same time;

determining that the first user input corresponds to a first component instance;

routing the first user input to a first input event processing thread associated with a first component instance;

determining a first user input event from the first user input;

rendering the determined first user input event on a user interface of the first component instance;

determining that the second user input corresponds to a separate second component instance;

routing the second user input to a separate second input event processing thread associated with the second component instance;

determining a second user input event from the second user input; and

rendering the determined second user input event on a user interface of the second component instance, wherein the first input event processing thread runs in parallel to a first main operating system event processing thread of the first component instance and wherein the second input event processing thread runs in parallel to a second main operating system event processing thread.

18. The computer system of claim 17, wherein determining that the first user input corresponds to the first component instance comprises determining that the first user input was input at a first input location and wherein determining that the second user input corresponds to a second component instance comprises determining that the second user input was input as a different second input location.

19. The computer system of claim 17, wherein determining that the first user input corresponds to the first component instance comprises determining the location and size of the first component instance and wherein determining that the second user input corresponds to a second component instance comprises determining the size and location of the second component instance.

20. The computer system of claim 17, wherein determining that the first user input corresponds to the first component instance and determining that the second user input corresponds to a second component instance comprises determining the type of user input.