VELOCITY FIELD INTERACTION FOR FREE SPACE GESTURE INTERFACE AND CONTROL

• US 20150103004A1
• Filed: 10/16/2014
• Published: 04/16/2015
• Est. Priority Date: 10/16/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: COHEN, Isaac, HOLZ, David S, SILLS, Maxwell
• Granted Patent: US 10,152,136 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to automatically interpreting a gesture of a control object in a three dimensional sensor space by sensing a movement of the control object in the three dimensional sensor space, sensing orientation of the control object, defining a control plane tangential to a surface of the control object and interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane.

Claims

1. A method including:

automatically interpreting a gesture of a control object in a three dimensional sensor space by;

sensing a movement of the control object in the three dimensional sensor space;

sensing orientation of the control object;

defining a control plane tangential to a surface of the control object; and

interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane.

2. The method of claim 1, wherein:

the control object is a hand; and

the control plane is tangential to a palm or back of the hand.

3. The method of claim 1, wherein interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane includes:

calculating an angular trajectory of the movement of the control object; and

determining whether the gesture engages a virtual control based on whether the trajectory is more normal or more parallel to the control plane.

4. A method including:

automatically interpreting a gesture of a control object in a three dimensional sensor space relative to a flow depicted in a display by;

sensing a movement of the control object in the three dimensional sensor space;

sensing orientation of the control object;

defining a control plane tangential to a surface of the control object; and

interpreting the gesture based on whether the control plane and the movement of the control object are more normal or more parallel to the flow depicted in the display.

5. The method of claim 4, wherein:

the control object is a hand; and

the control plane is tangential to a palm or back of the hand.

6. The method of claim 4, wherein interpreting the gesture based on whether the control plane and the movement of the control object are more normal or more parallel to the flow depicted in the display includes:

calculating a velocity and weighted velocity of the movement of the control object, the weighted velocity being weighted by a projection of the velocity onto a direction parallel to the flow depicted in the display and by a projection of a normal vector of the control plane onto the direction parallel to the flow depicted in the display; and

determining an amount the gesture engages a virtual control of the flow depicted in the display based on the calculated velocity.

7. A method including:

navigating a multi-layer presentation tree using gestures of a control object in a three dimensional sensor space by distinguishing between the control object and a sub-object of the control object, including;

sensing a movement of the control object in the three dimensional sensor space;

interpreting the movement of the control object as scrolling through a particular level of the multi-layer presentation tree; and

sensing a movement of the sub-object in the three dimensional sensor space and interpreting the movement of the sub-object as selecting a different level in the multi-layer presentation tree, the different level being either a deeper or higher level from the particular level, and subsequently interpreting the movement of the control object as scrolling through the different level of the multi-layer presentation tree.

8. The method of claim 7, wherein the sub-object is a portion of the control object.

9. The method of claim 7, wherein the control object is a hand and the sub-object is a finger of the hand.

10. A method including:

navigating a multi-layer presentation tree using gestures of a control object in a three dimensional sensor space by distinguishing between the control object and one or more sub-objects of the control object, including;

sensing a movement of the control object in the three dimensional sensor space;

interpreting the movement of the control object as traversing through a particular level of the presentation tree;

sensing a movement of a first sub-object of the control object in the three dimensional sensor space and interpreting the movement of the first sub-object as selecting a different level in the presentation tree, the different level being either deeper or higher level from the particular level, and subsequently interpreting the movement of the control object as traversing through the different level of the presentation tree; and

sensing a movement of a second sub-object of the control object in the three dimensional sensor space and interpreting the movement of the second sub-object as selecting a different presentation layout from a current presentation layout of the presentation tree, and subsequently presenting the presentation tree in the different presentation layout.

11. The method of claim 10, wherein:

the control object is a hand; and

the first and second sub-objects of the control objects are a finger, a palm, or a back of the hand.

12. A method including:

automatically determining a control to a virtual control by a control object in a three dimensional sensor space by distinguishing the control object and a sub-object of the control object, including;

sensing a location of the control object in a three dimensional sensor space;

determining whether the control object engages the virtual control based on the location of the control object;

sensing a movement of the sub-object of the control object in the three dimensional sensor space; and

interpreting the movement of the sub-object as a gesture controlling the virtual control if the control object engages the virtual control.

13. The method of claim 12, wherein:

the control object is a hand; and

the sub-object is a finger of the hand.

14. The method of claim 12, wherein the interpreting the movement of the sub-object as a gesture controlling the virtual control includes:

identifying a controllable parameter of the virtual control; and

selecting or adjusting the controllable parameter based on the movement of the sub-object.

15. A method including:

automatically determining a control to a virtual control by a control object in a three dimensional sensor space by;

sensing a location of the control object in the three dimensional sensor space;

determining whether the control object engages the virtual control based on the location of the control object;

sensing orientation of the control object;

defining a control plane tangential to a surface of the control object; and

interpreting a direction of the control plane as a gesture controlling the virtual control if the control object engages the virtual control.

16. The method of claim 15, wherein:

the control object is a hand; and

the surface is tangential to a palm or a back of the hand.

17. The method of claim 15, wherein the interpreting a direction of the control plane as a gesture controlling the virtual control includes:

identifying a controllable parameter of the virtual control; and

selecting or adjusting the controllable parameter based on the direction of the control plane.

18. A method including:

automatically interpreting a gesture of a control object in a three dimensional space relative to one or more objects depicted in a display by;

sensing a speed of a movement of the control object moving through the three dimensional sensor space;

interpreting the movement as a path on the display if the speed of the movement exceeds a pre-determined threshold; and

duplicating one or more of the objects in the display that intersect the interpreted path.

19. The method of claim 18, wherein the objects are pre-selected.