NATURAL USER INTERFACE FOR SELECTING A TARGET ELEMENT

• US 20170131873A1
• Filed: 11/09/2015
• Published: 05/11/2017
• Est. Priority Date: 11/09/2015
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Koenig, Kimberly, Somji, Shiraz M., Agafonov, Evgeny
• Application Number:
  US14/936,149
• Publication Number:
  US 20170131873A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting

Abstract

Selecting an intended target element via gesture-dependent hit testing is provided. Aspects provide for receiving a gesture input on or proximate to a selection handle and neighboring content; performing a hit test for determining whether a gesture input contact area and a selection handle hit target area overlap and/or exceed an upper limit overlap value; performing gesture recognition for determining whether the gesture input is a static or a manipulation gesture; selecting an intended target element based on at least one of the results of the hit test and the gesture recognition; and manipulating the intended target element in accordance with the manipulation gesture.

Claims

1. A computer-implemented method for improving functionality of a natural user interface of a computing device, comprising:

receiving a gesture input at a selection point area on or proximate to a selection handle and neighboring content displayed on a display screen, wherein the selection handle is displayed with at least one additional selection handle and provides an indication of currently-selected content;

performing a hit test to determine whether the selection point area associated with the gesture input overlaps a hit target associated with the selection handle;

in response to a positive determination, determining whether the gesture input includes a static gesture or a manipulation gesture; and

when a determination is made that the gesture input includes a manipulation gesture;

selecting an intended target element at the selection handle; and

manipulating the intended target element in accordance with the manipulation gesture.

2. The computer-implemented method of claim 1, wherein determining whether the gesture input includes a manipulation gesture comprises determining whether the gesture input comprises a drag gesture.

3. The computer-implemented method of claim 1, wherein the positive determination comprises:

determining that an overlap of the selection point area and the hit target meets or exceeds a first predetermined threshold value; and

determining that the overlap of the selection point area and the hit target is less than a second predetermined threshold value.

4. The computer-implemented method of claim 3, wherein:

determining that an overlap of the selection point area and the hit target meets or exceeds a first predetermined threshold value comprises determining that the selection point area and the hit target overlap by at least 1%; and

determining that the overlap of the selection point area and the hit target is less than a second predetermined threshold value comprises determining that the selection point area and the hit target overlap by less than 100%.

5. The computer-implemented method of claim 4, wherein manipulating the intended target element in accordance with the manipulation gesture comprises moving the selection handle for modifying an amount of currently-selected content.

6. The computer-implemented method of claim 4, wherein manipulating the intended target element in accordance with the manipulation gesture comprises moving the selection handle for resizing the currently-selected content.

7. The computer-implemented method of claim 1, wherein determining whether the gesture input includes a static gesture comprises determining whether the gesture input comprises a tap gesture.

8. The computer-implemented method of claim 7, further comprising:

when a determination is made that the gesture input includes a static gesture, selecting an intended target element at the neighboring content; and

inserting an insertion point at the neighboring content.

9. The computer-implemented method of claim 1, wherein the positive determination comprises:

determining that an overlap of the selection point area and the hit target meets or exceeds a first predetermined threshold value; and

determining that the overlap of the selection point area and the hit target is meets or exceeds a second predetermined threshold value.

10. The computer-implemented method of claim 9, further comprising:

when a determination is made that the gesture input includes a static gesture, selecting an intended target element at the selection handle; and

invoking a contextual menu at the selection handle.

11. A system for improving functionality of a natural user interface of a computing device, comprising:

one or more processors for executing programmed instructions;

memory, coupled to the one or more processors, for storing program instruction steps for execution by the computer processor;

a display screen operative to receive natural user interface input; and

a target element selection engine comprising;

an event message receiver operative to receive a gesture input at a selection point area on or proximate to a selection handle and neighboring content displayed on a display screen, wherein the selection handle is displayed with at least one additional selection handle and provides an indication of currently-selected content;

a hit tester operative to perform a hit test to determine whether the selection point area associated with the gesture input overlaps a hit target associated with the selection handle; and

a gesture recognizer operative to determine whether the gesture input includes a static gesture or a manipulation gesture in response to a positive determination by the hit tester; and

an application executing on the computing device, which when a determination is made that the gesture input includes a manipulation gesture, is operative to;

select an intended target element at the selection handle; and

manipulate the intended target element in accordance with the manipulation gesture.

12. The system of claim 11, wherein the hit tester is operative to positively determine that the selection point area associated with the gesture input overlaps the hit target associated with the selection handle when:

an overlap of the selection point area and the hit target meets or exceeds a first predetermined threshold value; and

the overlap of the selection point area and the hit target is less than a second predetermined threshold value.

13. The system of claim 12, wherein in manipulating the intended target element in accordance with the manipulation gesture, the application is operative to move the selection handle for modifying an amount of currently-selected content.

14. The system of claim 12, wherein in manipulating the intended target element in accordance with the manipulation gesture, the application is operative to move the selection handle for resizing the currently-selected content.

15. The system of claim 11, wherein:

in determining whether the gesture input includes a manipulation gesture, the gesture recognizer is operative to determine whether the gesture input comprises a drag gesture.

in determining whether the gesture input includes a static gesture, the gesture recognizer is operative to determine whether the gesture input comprises a tap gesture.

16. The system of claim 15, wherein the application if further operative to:

select an intended target element at the neighboring content when a determination is made that the gesture input includes a static gesture; and

insert an insertion point at the neighboring content.

17. The system of claim 11, wherein the hit tester is the hit tester is operative to positively determine that the selection point area associated with the gesture input overlaps the hit target associated with the selection handle when:

an overlap of the selection point area and the hit target meets or exceeds a first predetermined threshold value; and

the overlap of the selection point area and the hit target is meets or exceeds a second predetermined threshold value.

18. The system of claim 17, wherein when a determination is made that the gesture input includes a static gesture, the application is further operative to:

select an intended target element at the selection handle; and

invoke a contextual menu at the selection handle.

19. A device for improving functionality of a natural user interface of a computing device, the device comprising:

one or more processors for executing programmed instructions;

memory, coupled to the one or more processors, for storing program instruction steps for execution by the computer processor;

a display screen operative to receive natural user interface input; and

a target element selection engine operative to;

receive a gesture input at a selection point area on or proximate to a selection handle and neighboring content displayed on a display screen, wherein the selection handle is displayed with at least one additional selection handle and provides an indication of currently-selected content;

perform a first hit test to determine whether the selection point area associated with the gesture input overlaps a hit target associated with the selection handle by at least a first predetermined threshold value and less than a second predetermined threshold value;

in response to a positive determination of the first hit test,

determine whether the gesture input includes a tap gesture or a drag gesture;

when a determination is made that the gesture input includes a drag gesture;

select an intended target element at the selection handle; and

manipulate the intended target element in accordance with the manipulation gesture; and

when a determination is made that the gesture input includes a tap gesture;

select an intended target element at the neighboring content; and

insert an insertion point at the neighboring content.

20. The device of claim 19, wherein in response to a negative determination of the first hit test, the target element selection engine is further operative to:

perform a second hit test to determine whether the selection point area associated with the gesture input overlaps the hit target associated with the selection handle by at least the first predetermined threshold value and at least the second predetermined threshold value;

in response to a positive determination of the second hit test;

select the intended target element at the selection handle; and

determine whether the gesture input includes a tap gesture or a drag gesture;

when a determination is made that the gesture input includes a drag gesture, manipulate the intended target element in accordance with the manipulation gesture; and

when a determination is made that the gesture input includes a tap gesture, insert an insertion point at the neighboring content.