Gesture controlled persistent handles for interface guides

• US 8,920,241 B2
• Filed: 12/15/2010
• Issued: 12/30/2014
• Est. Priority Date: 12/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Reville, Brendan, Bridges, Jack, Mattingly, Andy, Andersen, Jordan, Klein, Christian, Dayal, Arjun
• Primary Examiner(s): MYHR, JUSTIN L
• Application Number:
  US12/969,377
• Publication Number:
  US 20120157208A1
• Time in Patent Office: 1,476 Days
• Field of Search: 463 36- 39, 345156-159
• US Class Current: 463/37
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

A computing system translates a world space position of a hand of a human target to a screen space position of a user interface and locks the hand to a handle of the user interface if world space parameters of the hand overcome a grab threshold of the handle. When the hand is locked to the handle, the computing system translates a world space position of the hand to a screen space handle position that is constrained along one or more interface guides. The hand is unlocked from the handle at a release position of the handle if world space parameters of the hand overcome a release threshold of the handle. The handle is retained at the release position after the hand is unlocked from the handle.

Claims

1. A computing system, comprising:

a logic subsystem;

a data holding subsystem holding instructions executable by the logic subsystem to;

translate a world space position of a hand of a human target to a screen space position of a user interface;

lock the hand to a visible handle of the user interface if world space parameters of the hand overcome a grab threshold of the visible handle;

when the hand is locked to the visible handle, translate a world space position of the hand to a screen space visible handle position that is constrained along one or more interface guides;

unlock the hand from the visible handle at a release position of the visible handle if world space parameters of the hand overcome a release threshold of the visible handle; and

retain the visible handle at the release position after the hand is unlocked from the visible handle.

2. The computing system of claim 1, where the world space parameters of the hand overcome the grab threshold of the visible handle if the hand is closed by the human target.

3. The computing system of claim 1, where the world space parameters of the hand overcome the grab threshold of the visible handle if a screen space position of the hand is within a threshold distance of the visible handle for a duration threshold.

4. The computing system of claim 1, where the world space parameters of the hand overcome the grab threshold of the visible handle if a screen space position of the hand is within a threshold distance of the visible handle and a speed of the hand is less than a speed threshold for a duration threshold.

5. The computing system of claim 1, where the world space parameters of the hand overcome the release threshold if the hand is opened by the human target.

6. The computing system of claim 1, where the world space parameters of the hand overcome the release threshold if a screen space position of the hand is outside of a threshold distance of the visible handle for a duration threshold.

7. The computing system of claim 1, where translating the world space position of the hand to the screen space position of the user interface includes translating the world space position of the hand to a cursor position of the user interface such that movement of the hand in world space causes corresponding movement of a cursor in screen space.

8. The computing system of claim 1, where the one or more interface guides are connected in a branched configuration.

9. A computing system, comprising:

a peripheral input configured to receive depth images from a depth camera;

a display output configured to output a user interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a human target;

translate a world space position of a hand of the human target to a screen space cursor position of the user interface such that movement of the hand in world space causes corresponding movement of a cursor in screen space;

lock the cursor to a visible handle of the user interface if world space parameters of the hand overcome a grab threshold of the visible handle;

when the cursor is locked to the visible handle, translate a world space position of the hand to a screen space visible handle position that is constrained along a primary interface guide such that movement of the hand in world space causes constrained movement of the visible handle in screen space, the constrained movement deviating from movement of the hand in world space;

unlock the cursor from the visible handle at a release position of the visible handle if world space parameters of the hand overcome a release threshold of the visible handle; and

retain the visible handle at the release position after the cursor is unlocked from the visible handle.

10. The computing system of claim 9, where one or more secondary interface guides extend from the primary interface guide in a branched configuration.

11. The computing system of claim 10, where the primary interface guide includes a stop corresponding to each of the one or more secondary interface guides, and where one of the one or more secondary interface guides is displayed when a corresponding stop is activated.

12. The computing system of claim 11, where the stop is activated when the screen space visible handle position is within a threshold distance of a screen space position of the stop.

13. The computing system of claim 10, where the primary interface guide includes a stop corresponding to each of the one or more secondary interface guides, and where if the world space parameters of the hand overcome the release threshold of the visible handle between two stops, the visible handle snaps to a closer of the two stops.

14. The computing system of claim 10, where the primary interface guide includes a stop corresponding to each of the one or more secondary interface guides, and where the one or more secondary interface guides are displayed whenever the primary interface guide is displayed.

15. The computing system of claim 10, where the primary interface guide is associated with a menu and each of the one or more secondary interface guides correspond to an item in the menu.

16. The computing system of claim 9, further comprising instructions executable by the logic subsystem to initiate scrolling of a list associated with the primary interface guide when the screen space visible handle position reaches an end of the primary interface guide.

17. A computing system, comprising:

a logic subsystem;

a data holding subsystem holding instructions executable by the logic subsystem to;

translate a world space position of a hand of a human target to a screen space position of a user interface;

lock the hand to a visible handle of the user interface if world space parameters of the hand overcome a grab threshold of the visible handle;

when the hand is locked to the visible handle, translate a world space position of the hand to a screen space visible handle position that is constrained along an interface guide, the interface guide having one or more stops; and

when the hand approaches one of the one or more stops, extend the interface guide in a divergent direction at the one stop.

18. The computing system of claim 17, where the interface guide is associated with a menu, and where the one or more stops are each associated with an item in the menu.

19. The computing system of claim 18, where if the hand moves the visible handle in the divergent direction at the one stop, the item associated with the one stop is selected and a release threshold of the visible handle is overcome and the hand unlocks from the visible handle.

20. The computing system of claim 17, where the interface guide has a linear shape, and where extending the interface guide in a divergent direction includes extending the interface guide in a perpendicular direction.