Switching input rails without a release command in a natural user interface

• US 9,958,946 B2
• Filed: 06/06/2014
• Issued: 05/01/2018
• Est. Priority Date: 06/06/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bastien, David William, Kozlowski, Oscar P., Schwesinger, Mark D.
• Primary Examiner(s): Chow, Doon
• Assistant Examiner(s): NICHOLAS, WENDY K
• Application Number:
  US14/298,562
• Publication Number:
  US 20150355717A1
• Time in Patent Office: 1,425 Days
• Field of Search: 715863, 715740, 715784, 715810, 715245, 715700, 715764, 715782, 715800, 715830, 715848, 715856, 715864, 715716, 345156, 345419, 345157, 345175, 345103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

User input in the form of image data is received from a user via a natural user interface. A vector difference between an adjustment start position and a current position of the user input is calculated. The vector difference includes a vector position and a vector length. The vector position is compared to stored rail data, and the vector length is compared to a stored threshold length. The rail data describes a plurality of virtual rails associated with an application. Based on the comparisons, the user input is matched to one of the plurality of virtual rails and a notification describing the matching is provided to the application. The application, thereupon, transitions from a first command to a second command corresponding to the matching virtual rail without receiving any explicit termination gesture for the first command from the user.

Claims

1. A system for transitioning from one gesture to another in a natural user interface without an explicit termination therebetween, said system comprising:

a memory area associated with a computing device, the memory area storing image data corresponding to user input received from a user via a natural user input interface, the memory area further storing rail data describing a first virtual rail and a second virtual rail; and

a processor programmed to;

calculate a vector difference between an adjustment start position and a current position of the user input, the calculated vector difference having a vector position and a vector length, the adjustment start position representing an initial position at which the user input begins, the first virtual rail corresponding to the adjustment start position and a first command in an application, the second virtual rail corresponding to the current position and a second command in the application;

determine an angle of the vector relative to the first virtual rail;

compare the vector length to a threshold length and the angle of the vector to an angle threshold;

if the vector length and the angle of the vector indicate a transition from the first virtual rail to the second virtual rail, compare the vector length to a rail adjustment threshold;

based on the comparisons of the vector length to the threshold length, the angle of the vector to the angle threshold, and the vector length to the rail adjustment threshold, match the user input to the second virtual rail; and

identify to the application the second virtual rail such that the application performs the second command.

2. The system of claim 1, wherein the processor is further programmed to determine the threshold length empirically.

3. The system of claim 1, wherein the processor is further programmed to determine the threshold length based on crowd-sourced data collected from a plurality of users.

4. The system of claim 1, wherein the processor is programmed to compare the vector length to the threshold length through determining whether the vector length is greater than the threshold length.

5. The system of claim 1, wherein the threshold length comprises a rail determination threshold length or a rail adjustment threshold length.

6. The system of claim 1, wherein said comparing the vector position comprises comparing the vector position to a cone threshold angle for the first virtual rail for transitioning from the first command to the second command.

7. A method comprising:

accessing, by a computing device, image data corresponding to user input received from a user via a natural user interface;

calculating a vector difference between an adjustment start position and a current position of the user input, the calculated vector difference having a vector position and a vector length, the adjustment start position representing an initial position at which the user input begins;

determining an angle of the vector relative to the first virtual rail;

comparing the vector position to rail data and the angle of the vector to an angle threshold, the rail data describing a first virtual rail and a second virtual rail, the first virtual rail corresponding to the adjustment start position and a first command in an application, the second virtual rail corresponding to the current position and a second command in the application;

if the vector position and the angle of the vector indicate a transition from the first virtual rail to the second virtual rail, compare the vector length to a rail adjustment threshold;

matching the user input to the second virtual rail based, at least in part, on the comparisons of the vector position, the angle of the vector, and the vector length; and

providing notification to the application of the second virtual rail, wherein the application automatically transitions from the first command to the second command.

8. The method of claim 7, wherein matching the user input comprises switching from the first virtual rail to the second virtual rail.

9. The method of claim 7, wherein the first virtual rail corresponds to a rail along an X-axis and the second virtual rail corresponds to a rail along a Y-axis.

10. The method of claim 7, wherein said automatically transitioning from the first command to the second command comprises at least one of the following:

transitioning from panning to zooming, from zooming to panning, from panning horizontally to panning vertically, or from panning vertically to panning horizontally.

11. The method of claim 7, wherein the rail threshold length comprises a rail determination threshold length or a rail adjustment threshold length.

12. The method of claim 7, wherein comparing the vector position comprises determining whether an angle of the current vector position with respect to the first virtual rail is within a cone angle threshold.

13. The method of claim 7, further comprising presenting an indication that the current position is closer to the second virtual rail than the first virtual rail.

14. The method of claim 7, further comprising:

calculating an apex angle of the vector based on the determined angle of the vector; and

comparing the apex angle of the vector to a cone angle threshold associated with the vector,

wherein said matching of the user input to the second virtual rail is based, at least in part, on the comparison of the apex angle to the cone angle threshold, and

wherein the rail data comprises the cone angle threshold.

15. The method of claim 14, wherein the apex angle is calculated by doubling the angle of the vector.

16. One or more computer storage media embodying computer-executable components, said components comprising:

a user interface component executable to cause at least one processor to access image data corresponding to user input received from a user via a natural user interface;

a motion tracking component executable to cause at least one processor to calculate a vector difference between an adjustment start position and a current position of the user input, the calculated vector difference having a vector position a vector length, and an angle of the vector relative to a first virtual rail, the adjustment start position representing an initial position at which the user input begins;

a rail adjustment component executable to cause at least one processor to compare the vector position to rail data and compare the angle of the vector to an angle threshold, the rail data describing a first virtual rail and a second virtual rail, the first virtual rail corresponding to the adjustment start position and a first command in an application, the second virtual rail corresponding to the current position and a second command in the application, and if the vector position and the angle of the vector indicate a transition from the first virtual rail to the second virtual rail, compare the vector length to a rail adjustment threshold, wherein the rail adjustment component matches the user input to the second virtual rail based, at least in part, on the comparison of the vector position to the data, the angle of the vector to the angle threshold, and the vector length to the rail adjustment threshold; and

a control component executable to cause at least one processor to identify, to the application, the second virtual rail, wherein the application automatically transitions from the first command to the second command.

17. The computer storage media of claim 16, wherein the application adjusts performance of the second command by adjusting a parameter value associated with the second command.

18. The computer storage media of claim 16, wherein the rail data includes a cone angle threshold for transitioning from the first command to the second command.

19. The computer storage media of claim 16, wherein matching the user input comprises switching from the first virtual rail to the second virtual rail.

20. The computer storage media of claim 16,

wherein the rail adjustment component causes said at least one processor to calculate an accuracy value based on the comparison of the angle of the vector to the angle threshold;

and

wherein the control component causes said at least one processor to adjust performance of the second comment based on the accuracy value.