Methods for system engagement via 3D object detection

• US 9,720,504 B2
• Filed: 09/23/2013
• Issued: 08/01/2017
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ton, Phuong Lam, Nelson, Hui-ya Liao, Li, Ren, Van Dyken, Shaun William
• Primary Examiner(s): NABI, REZA U
• Application Number:
  US14/034,204
• Publication Number:
  US 20140223385A1
• Time in Patent Office: 1,408 Days
• Field of Search: 715863
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Methods and apparatuses are presented for controlling an application on a device. In some embodiments, a method may include detecting that a user is maintaining an object or gesture at a position hovering near the device for a threshold length of time. The method may also include anchoring an initial position of the object or gesture to the device based on the detection of the maintained position, and controlling the application using the anchored initial position. In some embodiments, controlling the application using the anchored initial position may include manipulating the application based on detecting within a stable zone associated with the anchored initial position a change in height of the gesture or object relative to the device, and not manipulating the application whenever the object or gesture is detected to move along a plane above and parallel to the device and within the stable zone.

Claims

1. A method for controlling an application on a device, the method comprising:

detecting that a user is maintaining an object at a position hovering, for a threshold length of time, within a threshold distance of the device;

anchoring an initial position of the object to the device based on the detection of the maintained position;

in response to the detecting, engaging the device including displaying a user interface indicating one or more commands associated with a change of the object from the anchored initial position; and

controlling the application using the anchored initial position, the controlling including determining that a detected change of the object from the anchored initial position corresponds to a command of the one or more commands, wherein controlling the application using the anchored initial position comprises detecting three -dimensional movement of the object through space, wherein the space is defined by three dimensional coordinates and wherein the three-dimensional movement comprises movement in the three dimensional coordinates, and wherein the application is manipulated between a plurality of states, each of the plurality of states corresponding to a respective one of several distances between the object and the device.

2. The method of claim 1, wherein controlling the application using the anchored initial position further comprises manipulating the application whenever the object is detected to be outside of a stable zone, the stable zone including the anchored initial position,

wherein manipulating the application comprises:

performing a first panning action having a first speed if the object is detected to be within a first region outside of the stable zone; and

performing a second panning action having a second speed if the object is detected to be within a second region outside of the stable zone, wherein the second region is farther away from the stable zone than the first region.

3. The method of claim 2, wherein manipulating the application is based on the position of the object hovering over the device relative to the initial position.

4. The method of claim 2, wherein manipulating the application further comprises:

performing a third action if the object is detected within a third region outside of the stable zone, wherein the third region is farther away from the stable zone than the second region.

5. The method of claim 2, wherein the application is not manipulated due to the object, while in the stable zone, moving substantially parallel to a surface of the device over which the object hovered to anchor the initial position of the device.

6. The method of claim 2, wherein the application is manipulated between three states, each of the three states corresponding to a respective one of three distances between the object and the device.

7. The method of claim 1, wherein the user interface indicates the anchored initial position relative to a display used to display the user interface.

8. The method of claim 7, wherein the user interface is centered on the display relative to the initial position of the detected object.

9. The method of claim 1, wherein the controlling the application includes:

in response to determining that the object moves substantially parallel to a surface of the device over which the object hovered to anchor the initial position of the device, performing a first command of the one or more commands; and

in response to determining that the object moves substantially perpendicular to the surface of the device, performing a second command of the one or more commands.

10. The method of claim 9, wherein the controlling the application includes performing the first command simultaneously with the second command.

11. The method of claim 9, wherein first command is a command to perform a zoom in or a zoom out action on an image displayed by the device.

12. The method of claim 11, wherein the second command is a command to perform a panning action on the image displayed by the device.

13. The method of claim 12, wherein the panning action includes panning the image in a direction corresponding to the a direction that the object moved substantially parallel to the surface from the anchored initial position.

14. The method of claim 1, wherein the controlling the application using the anchored initial position comprises:

detecting a change of the object from the anchored initial position without detecting contact between the object and the device.

15. The method of claim 1, wherein the detecting three-dimensional movement of the object through space comprises:

detecting that the object moves in each of three dimensions of the space.

16. An apparatus comprising:

at least one sensor configured to detect that a user is maintaining an object at a position hovering, for a threshold length of time, within a threshold distance of the apparatus; and

a processor configured to;

anchor an initial position of the object to the apparatus based on detection of the maintained position;

detect, using the at least one sensor, that a user has maintained the object at the position for the threshold length of time and, in response, engage the apparatus including display of a user interface indicating one or more commands associated with a change of the object from the anchored initial position; and

control an application configured to operate on the apparatus using the anchored initial position, the control including determining that a detected change of the object from the anchored initial position corresponds to a command of the one or more commands, wherein controlling the application using the anchored initial position comprises detecting three-dimensional movement of the object through space, wherein the space is defined by three dimensional coordinates and wherein the three-dimensional movement comprises movement in the three dimensional coordinates, and wherein the application is manipulated between a plurality of states, each of the plurality of states corresponding to a respective one of several distances between the object and the apparatus.

17. The apparatus of claim 16, wherein the processor is further configured to:

manipulate the application whenever the object is detected to be outside of a stable zone, the stable zone including the anchored initial position;

perform, in the application, a first panning action having a first speed if the object is detected to be within a first region outside of the stable zone; and

perform, in the application, a second panning action having a second speed if the object is detected to be within a second region outside of the stable zone, wherein the second region is farther away from the stable zone than the first region.

18. The apparatus of claim 17, wherein manipulating the application is based on the position of the object hovering over the apparatus relative to the initial position.

19. The apparatus of claim 17, wherein the processor is further configured to:

perform, in the application, a third action if the object is detected within a third region outside of the stable zone, wherein the third region is farther away from the stable zone than the second region.

20. The apparatus of claim 17, wherein the application is not manipulated due to the object, while in the stable zone, moving substantially parallel to a surface of the apparatus over which the object hovered to anchor the initial position of the apparatus.

21. The apparatus of claim 17, wherein the application is manipulated between three states, each of the three states corresponding to a respective one of three distances between the object and the apparatus.

22. The apparatus of claim 16, wherein the displaying the user interface includes indicating the anchored initial position relative to a display used to display the user interface.

23. The apparatus of claim 22, wherein the user interface is centered on the display relative to the initial position of the detected object.

24. An apparatus comprising:

means for detecting that a user is maintaining an object at a position hovering, for a threshold length of time, within a threshold distance of the apparatus;

means for anchoring an initial position of the object to the apparatus based on the detection of the maintained position;

means for engaging the device in response to detecting that a user maintained an object at a position hovering within predefined coordinates above the apparatus for a threshold length of time, including display of a user interface indicating one or more commands associated with a change of the object from the anchored initial position; and

means for controlling an application operating on the apparatus using the anchored initial position, the means for controlling including determining that a detected change of the object from the anchored initial position corresponds to a command of the one or more commands, wherein controlling the application using the anchored initial position comprises detecting three-dimensional movement of the object through space wherein the space is defined by three dimensional coordinates and, wherein the three-dimensional movement comprises movement in the three dimensional coordinates, and wherein the application is manipulated between a plurality of states, each of the plurality of states corresponding to a respective one of several distances between the object and the apparatus.

25. The apparatus of claim 24, wherein the means for controlling the application using the anchored initial position further comprises means for manipulating the application whenever the object is detected to be outside of a stable zone, the stable zone including the anchored initial position,

wherein the means for manipulating the application comprises:

means for performing a first panning action having a first speed if the object is detected to be within a first region outside of the stable zone; and

means for performing a second panning action having a second speed if the object is detected to be within a second region outside of the stable zone, wherein the second region is farther away from the stable zone than the first region.

26. The apparatus of claim 25, wherein the means for manipulating the application is based on the position of the object hovering over the apparatus relative to the initial position.

27. The apparatus of claim 25, wherein the means for manipulating the application further comprises:

means for performing a third action if the object is detected within a third region outside of the stable zone, wherein the third region is farther away from the stable zone than the second region.

28. The apparatus of claim 25, wherein the application is manipulated between three states, each of the three states corresponding to a respective one of three distances between the object and the apparatus.

29. The apparatus of claim 24, wherein the means for controlling the application comprises means for using a user interface anchored to the application based on the anchored initial position relative to a display used to display the user interface.

30. The apparatus of claim 29, wherein the user interface is centered on the display relative to the initial position of the detected object.

31. A non-transitory processor-readable medium comprising processor-readable instructions that, when executed by one or more processors, cause the one or more processors to:

detect that a user is maintaining an object at a position hovering within predefined coordinates, for a threshold length of time, within a threshold distance of a device;

anchor an initial position of the object to the device based on detection of the maintained position;

in response to the detection, engage the device including display of a user interface indicating one or more commands associated with a change of the object from the anchored initial position; and

control an application configured to operate on the device using the anchored initial position, the control including determining that a detected change of the object from the anchored initial position corresponds to a command of the one or more commands, wherein controlling the application using the anchored initial position comprises detecting three-dimensional movement of the object through space near the device, wherein the space is defined by three dimensional coordinates and wherein the three-dimensional movement comprises movement in the three dimensional coordinates, and wherein the application is manipulated between a plurality of states, each of the plurality of states corresponding to a respective one of several distances between the object and the device.

32. The non-transitory processor-readable medium of claim 31, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

manipulate the application whenever the object is detected to be outside of a stable zone, the stable zone including the anchored initial position;

perform, in the application, a first panning action having a first speed if the object is detected to be within a first region outside of the stable zone; and

perform, in the application, a second panning action having a second speed if the object is detected to be within a second region outside of the stable zone, wherein the second region is farther away from the stable zone than the first region.

33. The non-transitory processor-readable medium of claim 32, wherein the application is not manipulated due to the object, while in the stable zone, moving substantially parallel to a surface of the device over which the object hovered to anchor the initial position of the device.

34. The non-transitory processor-readable medium of claim 32, wherein the application is manipulated between three states, each of the three states corresponding to a respective one of three distances between the object and the device.