Dynamic user interactions for display control and scaling responsiveness of display objects

• US 9,632,658 B2
• Filed: 01/15/2014
• Issued: 04/25/2017
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David
• Primary Examiner(s): Nguyen, Tuan S
• Application Number:
  US14/156,418
• Publication Number:
  US 20140201690A1
• Time in Patent Office: 1,196 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to distinguishing meaningful gestures from proximate non-meaningful gestures in a three-dimensional (3D) sensory space. In particular, it relates to calculating spatial trajectories of different gestures and determining a dominant gesture based on magnitudes of the spatial trajectories. The technology disclosed also relates to uniformly responding to gestural inputs from a user irrespective of a position of the user. In particular, it relates to automatically adapting a responsiveness scale between gestures in a physical space and resulting responses in a gestural interface by automatically proportioning on-screen responsiveness to scaled movement distances of gestures in the physical space, user spacing with the 3D sensory space, or virtual object density in the gestural interface. The technology disclosed further relates to detecting if a user has intended to interact with a virtual object based on measuring a degree of completion of gestures and creating interface elements in the 3D space.

Claims

1. A method of uniformly responding to gestural inputs from multiple users within a three-dimensional (3D) sensory space, the method including:

automatically adapting a responsiveness scale between gestures in a physical space from multiple users and resulting responses in a shared gestural interface by;

calculating a user spacing within a three-dimensional (3D) sensory space based on a spacing of users detected in the 3D sensory space; and

automatically proportioning on-screen responsiveness of the shared gestural interface responsive to the user spacing when interpreting movement distances of the gestures in the physical space.

2. The method of claim 1, further including:

detecting a presence of multiple users present within a field of view of a camera electronically coupled to the gestural interface.

3. The method of claim 1, wherein the automatically proportioning includes:

computing a scaling factor between movement made by a user in a field of view of a camera electronically coupled to the gestural interface to a scaled movement distance in the 3D sensory space.

4. The method of claim 3, further including:

altering the scaling factor to reflect a restricted range of motion, thereby enabling small physical movements to correspond to larger on-screen movements.

5. The method of claim 3, further including:

altering the scaling factor to reflect a distance between the camera and the users.

6. The method of claim 3, further including:

altering the scaling factor to reflect a quantity of rendered items on a display.

7. The method of claim 6, further including:

reducing the scaling factor;

thereby enabling a user to navigate between rendered items on a display.

8. The method of claim 3, wherein the computing the scaling factor includes:

computing a ratio of movement distances of a gesture in the physical space m to a pixel size M in a captured image captured by the camera.

9. The method of claim 3, further including:

detecting a movement below a threshold; and

responsive to the detection, switching the gesture interface from detecting using a low-sensitivity detection mode to detecting using a high-sensitivity mode;

wherein the detecting using a high-sensitivity mode includes a 3D image of the gesture being accurately reconstructed based on one or more captured 2D images captured by the camera.

10. A non-transitory computer readable medium storing instructions for uniformly responding to gestural inputs from multiple users within a three-dimensional (3D) sensory space, which instructions when executed by one or more processors perform:

automatically adapting a responsiveness scale between gestures in a physical space from multiple users and resulting responses in a shared gestural interface by;

calculating a user spacing within a three-dimensional (3D) sensory space based on a spacing of users detected in the 3D sensory space; and

automatically proportioning on-screen responsiveness of the shared gestural interface responsive to the user spacing when interpreting movement distances of the gestures in the physical space.

11. The non-transitory computer readable medium of claim 10, further including:

detecting a presence of multiple users present within a field of view of a camera electronically coupled to the gestural interface.

12. The non-transitory computer readable medium of claim 10, wherein the automatically proportioning includes:

computing a scaling factor between movement made by a user in a field of view of a camera electronically coupled to the gestural interface to a scaled movement distance in the 3D sensory space.

13. The non-transitory computer readable medium of claim 12, further including:

altering the scaling factor to reflect a restricted range of motion, thereby enabling small physical movements to correspond to larger on-screen movements.

14. The non-transitory computer readable medium of claim 12, further including:

altering the scaling factor to reflect a distance between the camera and the users.

15. The non-transitory computer readable medium of claim 12, further including:

altering the scaling factor to reflect a quantity of rendered items on a display.

16. The non-transitory computer readable medium of claim 12, further including:

reducing the scaling factor;

thereby enabling a user to navigate between rendered items on a display.

17. The non-transitory computer readable medium of claim 12, wherein computing the scaling factor includes:

computing a ratio of movement distances of a gesture in the physical space m to a pixel size M in a captured image captured by the camera.

18. The non-transitory computer readable medium of claim 12, further including:

detecting a movement below a threshold; and

responsive to the detection, switching the gesture interface from detecting using a low-sensitivity detection mode to detecting using a high-sensitivity mode;

wherein the detecting using a high-sensitivity mode includes a 3D image of the gesture being accurately reconstructed based on one or more captured 2D images captured by the camera.

19. A system for uniformly responding to gestural inputs from multiple users within a three-dimensional (3D) sensory space, the system including:

a camera; and

one or more processors coupled to the camera and to a non-transitory computer readable medium storing instructions thereon, which when executed by the processors perform;

automatically adapting a responsiveness scale between gestures performed in a physical space within a field of view of the camera from multiple users and resulting responses in a shared gestural interface by;

calculating a user spacing within a three-dimensional (3D) sensory space based on a spacing of users detected in the 3D sensory space; and

automatically proportioning on-screen responsiveness of the shared gestural interface responsive to the user spacing when interpreting movement distances of the gestures in the physical space.