Recognizing selection regions from multiple simultaneous inputs

• US 7,936,341 B2
• Filed: 05/30/2007
• Issued: 05/03/2011
• Est. Priority Date: 05/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Weiss, John Guido Atkins
• Primary Examiner(s): Dharia; Prabodh M
• Application Number:
  US11/755,660
• Publication Number:
  US 20080297482A1
• Time in Patent Office: 1,434 Days
• Field of Search: 345166-179, 323/304, 463/20
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The present invention extends to methods, systems, and computer program products for recognizing selection regions from multiple simultaneous inputs. Input selection regions are calculated from simultaneous contacts on a multi-touch input display surface. Computer system users can use natural and/or more intuitive hand gestures to select items on an input/display surface. Selection regions can be dynamically adjusted to in response to detected changes in contact at different locations on a surface. For example, selection region changes can be updated in essentially real-time as fingers are added and moved on the multi-touch input display surface. The calculation of selection regions can vary based on an application and/or operating system context.

Claims

1. At a computer system including a multi-touch input display surface, a method for recognizing a region for selecting items displayed on the multi-touch input display surface, the method comprising:

an act of receiving a set of points, indicating at least that;

contact with the multi-touch input display surface was detected at a first location on the multi-touch input display surface; and

contact with the multi-touch input surface was detected at a second location on the multi-touch input display surface simultaneously with the detected contact at the first location;

an act of calculating a selection region on the multi-touch input display surface based at least on the first location and the second location;

an act of identifying selected items displayed on the multi-touch input display surface that are selected by the calculated selection region;

an act of providing region visual feedback data to the multi-touch input display surface to display the calculated selection region on the multi-touch input display surface; and

an act of providing item visual feedback data to the multi-touch input display surface to visually indicate the selected items on the multi-touch input display surface.

2. The method as recited in claim 1, wherein the act of receiving a set of points comprises an act of receiving a set of coordinate locations on the multi-touch input display surface that were abstracted from areas of contact on the multi-touch input display surface.

3. The method as recited in claim 2, wherein the act of receiving a set of coordinate locations on the multi-touch input display surface that were abstracted from areas of contact comprises an act of receiving a set of coordinate locations on the multi-touch input display surface that were abstracted from areas of contact between a human hand and the multi-touch input display surface.

4. The method as recited in claim 1, wherein the act of receiving a set of points comprises an act of receiving an ordered set of points.

5. The method as recited in claim 1, wherein calculating a selection region on the multi-touch input display surface based at least on the first location and the second location comprises calculating a selection region based on a context at the computer system.

6. The method as recited in claim 1, wherein calculating a selection region on the multi-touch input display surface based at least on the first location and the second location comprises calculating a selection region in accordance with a region calculation algorithm.

7. The method as recited in claim 6, wherein calculating a selection region in accordance with a region calculation algorithm comprises an act of calculating a selection region in accordance with a region calculation algorithm selected from among:

a convex hull algorithm, a shrink wrap hull algorithm, a circle creation algorithm, a rectangle creation algorithm, an ellipsis creation algorithm, and a line creation algorithm.

8. The method as recited in claim 1, wherein the act of identifying selected items displayed on the multi-touch input display surface comprises identifying items that at least intersect the selection region.

9. The method as recited in claim 1, further comprising:

an act of displaying the boundaries of the calculated selection region on the multi-touch input display surface; and

an act of altering the display characteristics of any selected items to indicate that the selected items are selected by the selection region.

10. The method as recited in claim 5, wherein calculating the selection region on the multi-touch input display surface based on the context at the computer system causes a first selection region to be calculated from the received set of points when the computer system is in a first context and for a second selection region, which is different than the first selection region, to be calculated from the same received set of points when the computer system is in a second context that is different than the first context.

11. The method as recited in claim 1, wherein the act of providing item visual feedback includes changing a display of the selected items from a first display state to a second display state and wherein displaying the calculated selection region includes displaying at least one line or area on the display screen.

12. At a computer system including a multi-touch input display surface, a method for recognizing a region for selecting items displayed on the multi-touch input display surface, the method comprising:

an act of receiving input contact data indicating simultaneous contact on an area of the multi-touch input display surface;

an act of calculating a selection region on the multi-touch input surface based on the area of simultaneous contact;

an act of identifying selected items displayed on the multi-touch input surface that are selected by the calculated selection region;

an act of providing region visual feedback data to the multi-touch input surface to display the calculated selection region on the multi-touch input surface; and

an act of providing item visual feedback data to the multi-touch input surface to visually indicate the selected items.

13. The method as recited in claim 12, wherein the act of receiving input contact data indicating simultaneous contact on an area of the multi-touch input display surface comprises an act of receiving input contact data indicating simultaneous contact between a portion of a hand and the multi-touch input display surface.

14. The method as recited in claim 12, wherein the act of calculating a selection region on the multi-touch input surface based on the area of simultaneous contact comprises an act of setting the selection region equal to the area of simultaneous contact.

15. The method as recited in claim 12, further comprising:

an act of displaying the boundaries of the calculated selection region on the multi-touch input display surface; and

an act of altering the display characteristics of any selected items to indicate that the selected items are selected by the selection region.

16. At a computer system including a multi-touch input display surface, a method for dynamically recognizing a region for selecting items displayed on the multi-touch input display surface, the method comprising:

an act of receiving first contact input data indicating contact on one or more areas of the multi-touch input display surface;

an act of calculating a first selection region based on the multi-touch input display surface based on first contact input data;

an act of receiving second subsequent contact input data indicating contact on one or more different areas of the multi-touch input display surface subsequent to receiving the first contact input data;

an act of calculating a second selection region on the multi-touch input display surface based on second contact input data;

an act of interpolating an area on the multi-touch input display surface between the first selection region and the second selection region, the interpolated area connecting the first selection region and the second selection region;

an act of determining that the region for selecting items displayed on the multi-touch input display surface includes the first selection region, the interpolated area, and the second selection region;

an act of identifying selected items displayed on the multi-touch input display surface that intersect the region for selecting items displayed on the multi-touch input display surface;

an act of providing visual feedback data to the multi-touch input display surface to display the region for selecting items displayed on the multi-touch input display surface; and

an act of providing item visual feedback data to the multi-touch input display surface to indicate the selected items on the multi-touch input display surface.

17. The method as recited in claim 16, wherein the act of receiving first contact input data indicating contact on one or more areas of the multi-touch input display surface comprises an act of receiving a set of points.

18. The method as recited in claim 16, wherein the act of interpolating an area on the multi-touch input display surface between the first selection region and the second selection region comprises an act of interpolating between a first convex hull selection region and second convex hull selection region the multi-touch input display surface.

19. The method as recited in claim 16, wherein the act of interpolating an area on the multi-touch input display surface between the first selection region and the second selection region comprises an act of interpolating between a first area of contact and a second area of contact on the multi-touch input display surface.

20. The method as recited in claim 16, further comprising:

an act of receiving third subsequent contact input data indicating contact on one or more different areas of the multi-touch input display surface subsequent to receiving the second contact input data;

an act of calculating a third selection region on the multi-touch input display surface based on third contact input data;

an act of interpolating a second area on the multi-touch input display surface between the second selection region and the third selection region, the interpolated area connecting the second selection region and the third selection region; and

an act of determining that the region for selecting items displayed on the multi-touch input display surface includes the first selection region, the interpolated area, the second selection region, the second interpolated area and the third selection region.

21. The method as recited in claim 16, further comprising:

an act of displaying the boundaries of the region for selecting items displayed on the multi-touch input display surface; and

an act of altering the display characteristics of any selected items to indicate that the selected items are selected by the selection region.

22. The method as recited in claim 21, further comprising:

an act of dynamically updating the displayed boundaries of the region for selecting items displayed on the multi-touch input display surface and display characteristics of any selected items as the region for selecting items displayed on the multi-touch input display surface expands in area.