Dynamic user interactions for display control and manipulation of display objects

• US 10,241,639 B2
• Filed: 01/15/2014
• Issued: 03/26/2019
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David
• Primary Examiner(s): Nguyen, Tuan S
• Application Number:
  US14/156,426
• Publication Number:
  US 20140201684A1
• Time in Patent Office: 1,896 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to distinguishing meaningful gestures from proximate non-meaningful gestures in a three-dimensional (3D) sensory space. In particular, it relates to calculating spatial trajectories of different gestures and determining a dominant gesture based on magnitudes of the spatial trajectories. The technology disclosed also relates to uniformly responding to gestural inputs from a user irrespective of a position of the user. In particular, it relates to automatically adapting a responsiveness scale between gestures in a physical space and resulting responses in a gestural interface by automatically proportioning on-screen responsiveness to scaled movement distances of gestures in the physical space, user spacing with the 3D sensory space, or virtual object density in the gestural interface. The technology disclosed further relates to detecting if a user has intended to interact with a virtual object based on measuring a degree of completion of gestures and creating interface elements in the 3D space.

Claims

1. A method of creating interface elements in a three-dimensional (3D) sensory space, the method including:

detecting a first motion in a three-dimensional (3D) sensory space;

constructing, responsive to the first motion, a user interface element for display on a gestural interface, the user interface element being specifically constructed according to the detected first motion and having at least one associated function;

wherein the associated function of user interface element is selected based on a location of the user interface element on the gestural interface, the location of the user interface element on the gestural interface being movable and dependent upon the detected first motion;

detecting, using an electronic sensor, a subsequent second motion in the 3D sensory space, the second motion different from the first motion and proximate to the location of the constructed user interface element on the gestural interface;

changing state of the user interface element responsive to the subsequent second motion; and

responsive to the user interface element changing state, performing the at least one associated function selected.

2. The method of claim 1, wherein the constructed user interface element is a vertical slide bar and the first motion is a two-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of two fingers of a hand in free space.

3. The method of claim 2, wherein the two-finger vertical sweeping motion includes other fingers of the hand that are curled.

4. The method of claim 2, wherein responsive to the second motion the vertical slide bar is scrolled.

5. The method of claim 1, wherein the constructed user interface element is a vertical slide bar and the subsequent second motion is a one-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of one finger of a hand in free space.

6. The method of claim 5, wherein the one-finger vertical sweeping motion includes other fingers of the hand that are curled.

7. The method of claim 5, wherein the associated function is also selected based on a context of the gestural interface.

8. The method of claim 5, wherein the constructed user interface element is a vertical slide bar and the associated function is selected based on a location at which the vertical slide bar is created on the gestural interface.

9. A system including one or more processors coupled to memory storing computer instructions to create interface elements in a three-dimensional (3D) sensory space, which computer instructions, when executed on the one or more processors, implement actions comprising:

detecting a first motion in a three-dimensional (3D) sensory space;

constructing, responsive to the first motion, a user interface element for display on a gestural interface, the user interface element being specifically constructed according to the detected first motion and having at least one associated function;

wherein the associated function of the user interface element is selected based on a location of the user interface element on the gestural interface, the location of the user interface element on the gestural interface being movable and dependent upon the detected first motion;

detecting, using an electronic sensor, a subsequent second motion in the 3D sensory space, the second motion different from the first motion and proximate to the location of the constructed user interface element on the gestural interface;

changing state of the user interface element responsive to the subsequent second motion; and

responsive to the user interface element changing state, performing the at least one associated function selected.

10. The system of claim 9, wherein the user interface element is a vertical slide bar and the first motion is a two-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of two fingers of a hand in free space.

11. The system of claim 10, wherein the two-finger vertical sweeping motion includes other fingers of the hand that are curled.

12. The system of claim 9, wherein the subsequent second motion is a one-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of one finger of a hand in free space.

13. The system of claim 12, wherein the one-finger vertical sweeping motion includes other fingers of the hand that are curled.

14. The system of claim 9, wherein the associated function is also selected based on a context of the gestural interface.

15. The system of claim 9, wherein the constructed user interface element is a vertical slide bar and the associated function is selected based on a location at which the vertical slide bar is created on the gestural interface.

16. A non-transitory computer readable storage medium impressed with computer program instructions to create interface elements in a three-dimensional (3D) sensory space, which computer program instructions, when executed on one or more processors, implement actions comprising:

detecting a first motion in a three-dimensional (3D) sensory space;

constructing, responsive to the first motion, a user interface element for display on a gestural interface, the user interface element being specifically constructed according to the detected first motion and having at least one associated function;

wherein the associated function of the user interface element is selected based on a location of the user interface element on the gestural interface, the location of the user interface element on the gestural interface being movable and dependent upon the detected first motion;

detecting a subsequent second motion in the 3D sensory space, the second motion different from the first motion and proximate to the location of the constructed user interface element on the gestural interface;

changing state of the user interface element responsive to the subsequent second motion; and

responsive to the user interface element changing state, performing the at least one associated function selected.

17. The non-transitory computer readable storage medium of claim 16, wherein the user interface element is a vertical slide bar and the first motion is a two-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of two fingers of a hand in free space.

18. The non-transitory computer readable storage medium of claim 17, wherein the two-finger vertical sweeping motion includes other fingers of the hand that are curled.

19. The non-transitory computer readable storage medium of claim 16, wherein the subsequent second motion is a one-finger vertical sweeping motion that includes one selected from an upward movement and a downward movement of one finger of a hand in free space.

20. The non-transitory computer readable storage medium of claim 19, wherein the one-finger vertical sweeping motion includes other fingers of the hand that are curled.

21. The non-transitory computer readable storage medium of claim 16, wherein the associated function is also selected based on a context of the gestural interface.

22. The non-transitory computer readable storage medium of claim 16, wherein the constructed user interface element is a vertical slide bar and the associated function is selected based on a location at which the vertical slide bar is created on the gestural interface.