User-defined virtual interaction space and manipulation of virtual cameras in the interaction space

• US 10,275,039 B2
• Filed: 08/18/2017
• Issued: 04/30/2019
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Cohen, Isaac, Sills, Maxwell
• Primary Examiner(s): Dunphy, David F
• Application Number:
  US15/681,149
• Publication Number:
  US 20180039334A1
• Time in Patent Office: 620 Days
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

The technology disclosed relates to creating user-defined interaction spaces and modalities in a three dimensional (3D) sensor space in response to control gestures. It also relates to controlling virtual cameras in the 3D sensor space using control gestures and manipulating controls of the virtual cameras through the control gestures. In particular, it relates to defining one or more spatial attributes of the interaction spaces and modalities in response to one or more gesture parameters of the control gesture. It also particularly relates to defining one or more visual parameters of a virtual camera in response to one or more gesture parameters of the control gesture.

Claims

1. A method for defining interface modalities in a three dimensional (3D) sensor space, the method including:

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes a curved sweep that defines a collection of points within a distance to a point in the 3D sensor space;

calculating gesture parameters of the control gesture that was detected; and

defining spatial attributes for an interaction modality in the 3D sensor space responsive to the gesture parameters of the control gesture; and

manipulating controls in a virtual interaction space based upon a linked correspondence with the interaction modality to an image presented to a viewer.

2. The method of claim 1, wherein the gesture parameters include at least length and width of the control gesture.

3. The method of claim 1, wherein the gesture parameters include at least structure, scale, orientation, or density of the control object.

4. The method of claim 1, wherein spatial attributes include at least height and width of an interaction space.

5. The method of claim 1, wherein spatial attributes include at least numerosity of elements in the interaction modality.

6. The method of claim 1, further including detecting a context-setting control gesture that identifies a context for interpreting a subsequent control gesture that defines spatial attributes of the interaction modality.

7. The method of claim 6, wherein the context-setting control gesture is a voice, visual, or device command.

8. The method of claim 6, wherein the subsequent control gesture applies to an entire interaction space.

9. The method of claim 6, wherein the subsequent control gesture applies to an element of the virtual interaction space.

10. The method of claim 6, further including creating context-aware elements of the interaction modality that automatically interpret the context-setting control gesture and subsequent control gesture to define spatial attributes of the interaction modality.

11. The method of claim 1, wherein the control gesture is a stroke of a user appendage.

12. The method of claim 1, wherein the control object is a detectable object and the control gesture defines a collection of continuous points that have at least one parameter in common within a threshold deviation.

13. The method of claim 1, wherein a threshold deviation is determined by a variation in angle along velocity vectors that are continuous in time.

14. The method of claim 1, the method further including:

defining a vertical extent of the virtual interaction space in proportion to length of vertical portion of the curved sweep made by the control object;

defining a horizontal extent of the virtual interaction space in proportion to width of horizontal portion of the curved sweep made by the control object; and

manipulating controls in a physical interaction space by superimposing the virtual interaction space on the physical interaction space responsive to the vertical extent and horizontal extent.

15. The method of claim 1, the method further including:

defining a vertical extent of a virtual interaction space in proportion to length of vertical portion of the curved sweep made by the control object;

defining a horizontal extent of the virtual interaction space in proportion to width of horizontal portion of the curved sweep made by the control object; and

manipulating controls in a synthetic interaction space by linking the virtual interaction space to an image responsive to the vertical extent and horizontal extent.

16. The method of claim 1, the method further including:

detecting a circular sweep of a control object responsive to a control gesture in a three dimensional (3D) sensor space;

calculating a radius for a circular sweep based on a found point that is equidistant to a plurality of points defined on contour of the control gesture;

constructing a radial-based virtual interaction modality in the 3D sensor space that is in proportion to the radius of the circular sweep; and

manipulating controls in a physical interaction space by superimposing the radial-based virtual interaction modality on the physical interaction space responsive to the circular sweep.

17. The method of claim 1, the method further including:

identifying a pair of starting points in respective centers of control points of one or more control objects that are detected in a three dimensional (3D) sensor space, wherein the pair of starting points are fixed distance apart;

detecting an outward expanding movement of control points of one or more control objects in the 3D sensor space;

identifying a pair of resting points in respective centers of control points of one or more control objects when control points of one or more control objects come to rest;

defining a horizontal extent of a virtual interaction space in proportion to distance between starting points and resting points;

defining a vertical extent of the virtual interaction space in proportion to width of one or more control objects; and

presenting the virtual interaction space responsive to the vertical extent and horizontal extent.

18. The method of claim 17, wherein control objects are hands and control points are finger tips.

19. A non-transitory computer readable medium storing code that implements a method of defining interface modalities in a three dimensional (3D) sensor space, which code when executed by one or more processors performs actions including:

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes a curved sweep that defines a collection of points within a distance to a point in the 3D sensor space;

calculating gesture parameters of the control gesture that was detected; and

defining spatial attributes for an interaction modality in the 3D sensor space responsive to the gesture parameters of the control gesture; and

manipulating controls in a virtual interaction space based upon a linked correspondence with the interaction modality to an image presented to a viewer.

20. A system, for controlling a machine, including:

at least one camera;

one or more processors coupled to the at least one camera to process image information;

a computer readable medium storing code that implements a method of defining interface modalities in a three dimensional (3D) sensor space, which code when executed by one or more processors performs actions including;

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes a curved sweep that defines a collection of points within a distance to a point in the 3D sensor space;

calculating gesture parameters of the control gesture that was detected; and

defining spatial attributes for an interaction modality in the 3D sensor space responsive to the gesture parameters of the control gesture; and

manipulating controls in a virtual interaction space based upon a linked correspondence with the interaction modality to an image presented to a viewer.