User-defined virtual interaction space and manipulation of virtual cameras with vectors

• US 10,281,992 B2
• Filed: 01/03/2018
• Issued: 05/07/2019
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Cohen, Isaac, Sills, Maxwell, Durdik, Paul
• Primary Examiner(s): Reed, Stephen T.
• Application Number:
  US15/861,578
• Publication Number:
  US 20180253150A1
• Time in Patent Office: 489 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H05K 999/99 dummy group

Abstract

The technology disclosed relates to creating user-defined interaction spaces and modalities in a three dimensional (3D) sensor space in response to control gestures. It also relates to controlling virtual cameras in the 3D sensor space using control gestures and manipulating controls of the virtual cameras through the control gestures. In particular, it relates to defining one or more spatial attributes of the interaction spaces and modalities in response to one or more gesture parameters of the control gesture. It also particularly relates to defining one or more visual parameters of a virtual camera in response to one or more gesture parameters of the control gesture.

Claims

1. A method of interacting with a virtual vector field in a three-dimensional (3D) sensor space, the method including:

defining a vector field at least responsive to curling of fingers of a hand and degrees of freedom between fingers of the curled fingers;

constructing a virtual sphere along a plurality of points on contour of curled fingers in the 3D sensor space;

extending radially, inward or outward, one or more interaction vectors on the virtual sphere, wherein magnitudes of the interaction vectors are determined by a radius of the virtual sphere;

compounding interactions of the vector field with the interaction vectors based on their respective magnitudes; and

providing for display a user manipulable persistent rendering of the constructed virtual sphere, wherein a radius of the persistent rendering of the constructed virtual sphere increases and decreases in proportion to the inward or outward extending of the one or more interaction vectors of the virtual sphere.

2. The method of claim 1, wherein a motion of the displayable rendering varies in proportion to the compounding interactions of the vector field with the one or more interaction vectors of the virtual sphere.

3. A method of manipulating virtual objects in a three-dimensional (3D) sensor space, the method including:

creating a virtual vector field in response to a control gesture that makes swirling motions in a 3D sensor space, wherein the virtual vector field is a vortex;

creating a plurality of virtual objects in response to subsequent control gestures that make circular sweeps in the 3D sensor space and define object vectors on respective virtual objects;

compounding interactions of the vector field with the object vectors based on their respective magnitudes; and

adjusting a whirlpool effect of the vortex on the virtual objects in proportion to the interactions of the vector field and the object vectors to cause the object vectors of the respective virtual objects to bring one or more of the virtual objects closer to a center of the vortex or to push one or more of the virtual objects away from the center of the vortex.

4. The method of claim 3, further comprising:

determining from the plurality of virtual objects a set of virtual objects within the vortex based upon the size of the vortex; and

moving members of the set of virtual objects in unison responsive to the control gesture.

5. The method of claim 3, further comprising determining from a compounding of the size of the vortex and curling of fingers of a hand an appropriate velocity of motion for the virtual objects.

6. The method of claim 3, further comprising determining from a compounding of the size of the vortex and degrees of freedom between fingers of curled fingers of a hand an appropriate velocity of motion for the virtual objects.

7. The method of claim 3, further comprising determining a repulsive vortex in which virtual objects move further apart from one another.

8. A non-transitory computer-readable recording medium having a program recorded thereon for interacting with a virtual vector field in a three-dimensional (3D) sensor space, the program, when executed on a processor, implementing a method comprising:

defining a vector field at least responsive to curling of fingers of a hand and degrees of freedom between fingers of the curled fingers;

constructing a virtual sphere along a plurality of points on contour of curled fingers in the 3D sensor space;

extending radially, inward or outward, one or more interaction vectors on the virtual sphere, wherein magnitudes of the interaction vectors are determined by a radius of the virtual sphere;

compounding interactions of the vector field with the interaction vectors based on their respective magnitudes; and

providing for display a user manipulable persistent rendering of the constructed virtual sphere, wherein a radius of the persistent rendering of the constructed virtual sphere increases and decreases in proportion to the inward or outward extending of the one or more interaction vectors of the virtual sphere.

9. The non-transitory computer-readable recording medium of claim 8, wherein a motion of the displayable rendering varies in proportion to the compounding interactions of the vector field with the one or more interaction vectors of the virtual sphere.

10. A non-transitory computer-readable recording medium having a program recorded thereon for manipulating virtual objects in a three-dimensional (3D) sensor space, the program, when executed on a processor, implementing a method comprising:

creating a virtual vector field in response to a control gesture that makes swirling motions in a 3D sensor space, wherein the virtual vector field is a vortex;

creating a plurality of virtual objects in response to subsequent control gestures that make circular sweeps in the 3D sensor space and define object vectors on respective virtual objects;

compounding interactions of the vector field with the object vectors based on their respective magnitudes; and

adjusting a whirlpool effect of the vortex on the virtual objects in proportion to the interactions of the vector field and the object vectors to cause the object vectors of the respective virtual objects to bring one or more of the virtual objects closer to a center of the vortex or to push one or more of the virtual objects away from the center of the vortex.

11. The non-transitory computer-readable recording medium method of claim 10, further comprising:

determining from the plurality of virtual objects a set of virtual objects within the vortex based upon the size of the vortex; and

moving members of the set of virtual objects in unison responsive to the control gesture.

12. The non-transitory computer-readable recording medium of claim 10, wherein the method further includes determining from a compounding of the size of the vortex and curling of fingers of a hand an appropriate velocity of motion for the virtual objects.

13. The non-transitory computer-readable recording medium of claim 10, wherein the method further includes determining from a compounding of the size of the vortex and degrees of freedom between fingers of curled fingers of a hand an appropriate velocity of motion for the virtual objects.

14. The non-transitory computer-readable recording medium of claim 10, wherein the method further includes determining a repulsive vortex in which virtual objects move further apart from one another.

15. A smart phone having an interface that identifies fingers of a hand moving in a three-dimensional (3D) sensor space, the smart phone comprising:

a hardware controller storing instructions that, when executed, implement actions including;

defining a vector field at least responsive to curling of the fingers of the hand, as viewed by a camera having a particular vantage point, and degrees of freedom between fingers of the curled fingers;

constructing a virtual sphere along a plurality of points on contour of curled fingers in the 3D sensor space;

extending radially, inward or outward, one or more interaction vectors on the virtual sphere, wherein magnitudes of the interaction vectors are determined by a radius of the virtual sphere;

compounding interactions of the vector field with the interaction vectors based on their respective magnitudes; and

providing for display a user manipulable persistent rendering of the constructed virtual sphere, wherein a radius of the persistent rendering of the constructed virtual sphere increases and decreases in proportion to the inward or outward extending of the one or more interaction vectors of the virtual sphere.

16. The smart phone of claim 15, wherein a motion of the displayable rendering varies in proportion to the compounding interactions of the vector field with the one or more interaction vectors of the virtual sphere.

17. A smart phone having an interface that manipulates virtual objects in a three-dimensional (3D) sensor space, the smart phone comprising:

a hardware controller storing instructions that, when executed, implement actions including;

creating a virtual vector field in response to a control gesture, as viewed by a camera having a particular vantage point, that makes swirling motions in a 3D sensor space, wherein the virtual vector field is a vortex;

creating a plurality of virtual objects in response to subsequent control gestures that make circular sweeps in the 3D sensor space and define object vectors on respective virtual objects;

compounding interactions of the vector field with the object vectors based on their respective magnitudes; and

adjusting a whirlpool effect of the vortex on the virtual objects in proportion to the interactions of the vector field and the object vectors to cause the object vectors of the respective virtual objects to bring one or more of the virtual objects closer to a center of the vortex or to push one or more of the virtual objects away from the center of the vortex.

18. The smart phone of claim 17, wherein the actions further include:

determining from the plurality of virtual objects a set of virtual objects within the vortex based upon the size of the vortex; and

moving members of the set of virtual objects in unison responsive to the control gesture.

19. The smart phone of claim 17, wherein the actions further include determining from a compounding of the size of the vortex and curling of fingers of a hand an appropriate velocity of motion for the virtual objects.

20. The smart phone of claim 17, wherein the actions further include determining from a compounding of the size of the vortex and degrees of freedom between fingers of curled fingers of a hand an appropriate velocity of motion for the virtual objects.