MULTI-INPUT GESTURE CONTROL FOR A DISPLAY SCREEN

• US 20120113015A1
• Filed: 11/05/2010
• Published: 05/10/2012
• Est. Priority Date: 11/05/2010
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Werner, Horst
• Granted Patent: US 8,769,444 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Improved multi-input gesture control for a display screen is enabled by using a first screen input to determine control operations that result from a second screen input that is followed by a gesture. A gesture that corresponds to a path traced on a display screen may define a new screen input or manipulate existing screen objects depending on some specific feature of the path or the entirety of the path.

Claims

1. A method of multi-input gesture control for a display screen, the method comprising:

receiving a first screen input that identifies a first location on the display screen;

providing an image marker at the first location in response to the first screen input, wherein the image marker persists on the display screen for a response-time period after the first screen input;

receiving a second screen input that identifies a second location on the display screen during the response-time period after the first screen input;

changing to an alternative input mode based on a position of the second location relative to the image marker; and

receiving a gesture as a screen input in the alternative input mode, wherein the gesture includes a motion along a path on the display screen starting from the second location.

2. The method of claim 1, wherein

the image marker surrounds the first location as a single two-dimensional shape, and

a persistence of the image marker is displayed by changing a color or intensity of the image marker over the response-time period.

3. The method of claim 1, wherein the path of the completed gesture defines an input to an application program.

4. The method of claim 1, wherein the second location lies inside the image marker and the gesture defines a new shape on the display screen.

5. The method of claim 1, wherein the motion of the gesture defines a continuous manipulation of one or more shapes on the display screen.

6. The method of claim 1, wherein the second location lies outside the image marker and the gesture defines a resizing of one or more shapes on the display screen.

7. The method of claim 1, wherein the first and second screen inputs and the gesture each result from touching the display screen.

8. The method of claim 1, wherein the first and second screen inputs and the gesture each result from an application of a pointing device whose movements are echoed on the display screen by a pointer.

9. A non-transitory computer-readable medium embodying instructions which, when executed by a computer, cause the computer to perform operations comprising:

receiving a first screen input that identifies a first location on the display screen;

providing an image marker at the first location in response to the first screen input, wherein the image marker persists on the display screen for a response-time period after the first screen input;

receiving a second screen input that identifies a second location on the display screen during the response-time period after the first screen input;

changing to an alternative input mode based on a position of the second location relative to the image marker; and

receiving a gesture as a screen input in the alternative input mode, wherein the gesture includes a motion along a path on the display screen starting from the second location.

10. The computer-readable medium of claim 9, wherein

the image marker surrounds the first location as a single two-dimensional shape, and

a persistence of the image marker is displayed by changing a color or intensity of the image marker over the response-time period.

11. The computer-readable medium of claim 9, wherein the path of the completed gesture defines an input to an application program.

12. The computer-readable medium of claim 9, wherein the second location lies inside the image marker and the gesture defines a new shape on the display screen.

13. An apparatus including a display screen, the apparatus comprising:

a first location-receiving module to receive a first screen input that identifies a first location on the display screen;

a marker module to provide an image marker at the first location in response to the first screen input, wherein the image marker persists on the display screen for a response-time period after the first screen input;

a second location-receiving module to receive a second screen input that identifies a second location on the display screen during the response-time period after the first screen input;

a mode-changing module to change to an alternative input mode for the display screen based on a position of the second location relative to the image marker;

a gesture-receiving module to receive a gesture as a screen input in the alternative input mode, wherein the gesture includes a motion along a path on the display screen starting from the second location; and

a storage module to persistently store display screen data that identifies the first location, the image marker, the second location, and the gesture.

14. The apparatus of claim 13, wherein

the image marker surrounds the first location as a single two-dimensional shape, and

a persistence of the image marker is displayed by changing a color or intensity of the image marker over the response-time period.

15. The apparatus of claim 13, wherein the path of the completed gesture defines an input to an application program.

16. The apparatus of claim 13, wherein the second location lies inside the image marker and the gesture defines a new shape on the display screen.

17. The apparatus of claim 13, wherein the motion of the gesture defines a continuous manipulation of one or more shapes on the display screen.

18. The apparatus of claim 13, wherein the second location lies outside the image marker and the gesture defines a resizing of one or more shapes on the display screen.

19. The apparatus of claim 13, wherein the first and second screen inputs and the gesture each result from touching the display screen.

20. The apparatus of claim 13, wherein the first and second screen inputs and the gesture each result from an application of a computer pointing device whose movements are echoed on the display screen by a pointer.