Velocity field interaction for free space gesture interface and control

• US 10,635,185 B2
• Filed: 09/13/2019
• Issued: 04/28/2020
• Est. Priority Date: 10/16/2013
• Status: Active Grant

• Current Assignee: Ultrahaptics Ip Two Limited (Ultraleap Holdings Ltd.)
• Original Assignee: Ultrahaptics Ip Two Limited (Ultraleap Holdings Ltd.)
• Inventors: Cohen, Isaac, Holz, David S., Sills, Maxwell
• Primary Examiner(s): Ketema, Benyam
• Application Number:
  US16/570,914
• Publication Number:
  US 20200004344A1
• Time in Patent Office: 228 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to automatically interpreting a gesture of a control object in a three dimensional sensor space by sensing a movement of the control object in the three dimensional sensor space, sensing orientation of the control object, defining a control plane tangential to a surface of the control object and interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane.

Claims

1. A method of automatically interpreting a gesture of a control object, in a three-dimensional (3D) sensor space using a 3D sensor, as a first gesture or a second gesture, the method including:

determining that the gesture is the first gesture when a direction of a sensed trajectory of a sensed movement of the control object is within a pre-determined range of a normal vector that is normal to a defined control plane that remains tangent to a surface of the control object throughout a sensed movement of the control object in any direction in the 3D sensor space, the control plane being defined by an orientation of the control object as sensed by a video camera, whereby the direction of the movement of the trajectory of the control object is more normal to a surface of the control plane than parallel to the surface of the control plane; and

determining that the gesture is the second gesture when the direction of the trajectory of the movement of the control object is parallel to the surface of the control plane, within a pre-determined range, whereby the direction of the movement of the trajectory of the control object is more parallel to the surface of the control plane than normal to the surface of the control plane.

2. The method of claim 1, wherein:

the control object is a hand; and

the control plane is tangent to a palm or back of the hand.

3. The method of claim 1, wherein the interpreting of the gesture includes:

calculating an angular trajectory of the movement of the control object; and

determining whether the gesture engages a virtual control based on whether the trajectory is normal or parallel to the control plane.

4. The method of claim 1, further including determining the control plane is normal to the trajectory when the normal vector of the control plane is within +/−

10 degrees from a tangent vector of the trajectory.

5. The method of claim 1, wherein the first gesture is an interaction gesture for interacting with an object and the second gesture in a non-interaction gesture.

6. A method of automatically interpreting a gesture of a control object, in a three-dimensional (3D) sensor space relative to a flow depicted in a display, as a first gesture or a second gesture, the method including:

determining that the gesture is the first gesture when a direction of a sensed flow is within a pre-determined range of a normal vector that is normal to a defined control plane that remains tangent to a surface of the control object throughout a sensed movement of the control object in any direction in the 3D sensor space, the control plane being defined by an orientation of the control object as sensed by a video camera, whereby the direction of the flow is more normal to the surface of the control plane than parallel to the surface of the control plane; and

determining that the gesture is the second gesture when the direction of the flow is parallel to the surface of the control plane, within a pre-determined range, whereby the direction of the flow is more parallel to the surface of the control plane than normal to the surface of the control plane.

7. The method of claim 6, wherein:

the control object is a hand; and

the control plane is tangent to a palm or back of the hand.

8. The method of claim 6, wherein the interpreting of the gesture includes:

calculating a velocity and weighted velocity of the movement of the control object, the weighted velocity being weighted by a projection of the velocity onto a direction parallel to the flow depicted in the display and by a projection of a normal vector of the control plane onto the direction parallel to the flow depicted in the display; and

determining an amount the gesture engages a virtual control of the flow depicted in the display based on the calculated velocity.

9. The method of claim 6, further including determining an amount that a hand gesture engages vectors describing the flow using the normal vector based on whether the control plane and movement of the hand are determined to be normal or parallel to the vectors of the flow.

10. A non-transitory computer-readable recording medium having a program for automatically interpreting a gesture of a control object, in a three-dimensional (3D) sensor space relative to a flow depicted in a display, as a first gesture or a second gesture recorded thereon, the program, when executed by one or more processors, causing the one or more processors to perform a method including:

determining that the gesture is the first gesture when a direction of a sensed flow is within a pre-determined range of a normal vector that is normal to a defined control plane, that remains tangent to a surface of the control object throughout a sensed movement of the control object in any direction in the 3D sensor space, the control plane being defined by an orientation of the control object as sensed by a video camera, whereby the direction of the flow is more normal to the surface of the control plane than parallel to the surface of the control plane; and

determining that the gesture is the second gesture when the direction of the flow is parallel to the surface of the control plane, within a pre-determined range, whereby the direction of the flow is more parallel to the surface of the control plane than normal to the surface of the control plane.