GRAPHICAL USER INTERFACE FOR LARGE-SCALE, MULTI-USER, MULTI-TOUCH SYSTEMS

• US 20090193366A1
• Filed: 07/30/2008
• Published: 07/30/2009
• Est. Priority Date: 07/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: DAVIDSON, PHILIP L.
• Granted Patent: US 8,181,122 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method implemented on the graphical user interface device to invoke an independent, user-localized menu in an application environment, by making a predetermined gesture with a pointing device on an arbitrary part of a display screen or surface, especially when applied in a multi-touch, multi-user environment, and in environments where multiple concurrent pointing devices are present. As an example, the user may trace out a closed loop of a specific size that invokes a default system menu at any location on the surface, even when a second user may be operating a different portion of the system elsewhere on the same surface. As an additional aspect of the invention, the method allows the user to smoothly transition between the menu-invocation and menu control.

Claims

1. A method of invoking a control element on a display device, comprising the steps of:

generating by a user a stroke on a display device using a moving pointer element;

ascertaining, during generation of the stroke, if the stroke corresponds to a predetermined shape;

invoking a control element after ascertaining that the stroke corresponds to the predetermined shape, the invoked control element being displayed on the display device.

2. The method of claim 1, wherein the display device is a touch sensitive display device and the generating step comprises generating by the user the stroke on the display device by actuating the touch sensitive display device.

3. The method of claim 1, wherein the predetermined shape is a closed loop.

4. The method of claim 1, wherein the predetermined shape is a signature shape associated with the user.

5. The method of claim 1, wherein the stroke generated by the user is a subsection of a continuous sequence of input by the user using the moving pointer element.

6. The method of claim 1, wherein the generating step comprises generating by at least one user a plurality of strokes on the display device, each of the strokes using a respective moving pointing element.

7. The method of claim 1, wherein, after ascertaining that the stroke corresponds to the predetermined shape, the method comprises the step of conducting a secondary analysis of a characteristic of the stroke;

and wherein the invoking step comprises invoking a first control element or a second control element based on results of the conducted secondary analysis.

8. The method of claim 1, wherein the ascertaining step comprises ascertaining if the stroke corresponds to either a first predetermined shape or a second predetermined shape;

and the invoking step comprises invoking a first control element after ascertaining that the stroke corresponds to the first predetermined shape and invoking a second control element after ascertaining that the stroke corresponds to the second predetermined shape.

9. The method of claim 1, wherein the ascertaining step also ascertains a direction of travel of the stroke;

and the invoking step comprises invoking a first control element after ascertaining that the stroke corresponds to a predetermined shape with a measurable clockwise or counterclockwise winding (e.g. a closed polygon or loop) and that the ascertained direction of travel corresponds to a clockwise direction, and invoking a second control element after ascertaining that the stroke corresponds to the predetermined shape and the ascertained direction of travel corresponds to a counter-clockwise direction.

10. The method of claim 1, further comprising recognizing, during generation of the stroke, if the generated stroke corresponds to a substantial portion of the predetermined shape;

and providing visual feedback on the display corresponding to at least a portion of the generated stroke if the generated stroke is recognized to correspond to said substantial portion of the predetermined shape, the visual feedback identifying to the user that the stroke being generated corresponds to a partial amount of the predetermined shape.

11. The method of claim 1, wherein the ascertaining step comprises ascertaining if the stroke corresponds to a predetermined shape and is within a predetermined size threshold;

and the invoking step comprises invoking the control element after ascertaining that the stroke corresponds to the predetermined shape and the stroke is within the predetermined size threshold.

12. The method of claim 1, wherein the generating step comprises generating by a plurality of users respective strokes on said display device;

the ascertaining step comprises ascertaining, for each of the strokes being generated, if the respective stroke corresponds to the predetermined shape; and

the invoking step comprises invoking the control element for each ascertained stroke that corresponds to the predetermined shape.

13. The method of claim 1, wherein the invoking step comprises invoking, after ascertaining that the stroke corresponds to the predetermined shape, a control element that corresponds to the environment on the display device disposed at a position of the moving point element used to generate the stroke.

14. The method of claim 1, wherein the invoking step comprises displaying the control element at a center of the stroke that corresponds to the predetermined shape.

15. The method of claim 1, wherein the invoking step comprises displaying the control element at the position at which the generating stroke was disposed when the stroke was ascertained to correspond to the predetermined shape.

16. The method of claim 1, wherein the invoking step comprises displaying the control element at a position corresponding to a position of the moving pointer element upon discontinued movement of the moving pointer element after the stroke was ascertained to correspond to the predetermined shape.

17. The method of claim 1, wherein the invoking step comprises displaying the control element at a position corresponding to a last position of the moving pointer element upon discontinued contact of the moving pointer element on the display device after the stroke was ascertained to correspond to the predetermined shape.

18. The method of claim 1, wherein the invoking step comprises displaying the control element at a position that is a function of a position of the generated stroke and also is a function of boundaries of the display device so that the displayed control element is fully displayed within the display device.

19. The method of claim 1, wherein the invoking step comprises displaying the control element at a position and orientation that is a function of the position and orientation of the generated stroke and also is a function of positions and orientations of other control elements on the display device, of application objects present on the display device, or of the activity of other users of the device.

20. A method of activating a control element on a display device, comprising the steps of:

invoking a control element on a display device in response to a generated stroke made using a moving pointer element;

ascertaining a speed of movement of the moving pointer element during generation of the stroke;

establishing a manner of activation of the control element based on the ascertained speed of movement of the moving pointer element during generation of the stroke.

21. The method of claim 20, wherein the ascertaining step comprises ascertaining whether the speed of movement of the moving pointer element corresponds to a relatively slow speed, a relatively moderate speed, or a relatively high speed.

22. The method of claim 21, wherein the establish step comprises establishing the manner of activation of the control element as being immediately activatable by subsequent movement of the moving pointer element if the speed of movement of the moving pointer element is ascertained to correspond to the relatively slow speed.

23. The method of claim 21, wherein the establish step comprises establishing the manner of activation of the control element as being activatable by subsequent movement of the moving pointer element after discontinued movement of the moving pointer element if the speed of movement of the moving pointer element is ascertained to correspond to the relatively moderate speed.

24. The method of claim 21, wherein the establish step comprises establishing the manner of activation of the control element as including placing the control element on the display device at a position disposed away from the moving pointer element and disposed behind a direction of travel of the moving pointer element if the speed of the movement of the moving pointer element is ascertained to correspond to the relatively high speed, and optionally, the control element being immediately activatable if the pointer element returns to its active area.

25. A method of activating a control element, comprising the steps of:

invoking a control element on a display device in response to a generated first stroke made using a first moving pointer element, the control element having a plurality of selectable command options; and

activating one of the selectable command options as a function of a generated second stroke made using a second moving pointer element.

26. The method of claim 25, wherein the first moving pointer element generates the first stroke by providing input to an input device, and the second moving pointer element generates the second stroke by providing input to said input device.

27. The method of claim 25, wherein the display device is a multi-input display device and the first and second moving pointer elements generate the respective first and second strokes by contacting the multi-input display device, and wherein the second stroke may begin before the first stroke is completed.

28. The method of claim 25, wherein the invoking step comprises displaying the control element on the display device at a position that corresponds to a position of the generated first stroke.

29. The method of claim 28, wherein the invoking step comprises moving the displayed control element to positions corresponding to positions of the generated first stroke during movement of the first moving pointer element.