Crane Gesture

• US 20150160819A1
• Filed: 12/06/2013
• Published: 06/11/2015
• Est. Priority Date: 12/06/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hwang, Dan, Abanami, Thamer, Rodriguez, Jose, Greenlay, Scott, Fellows, Christopher, Tobens, Joe
• Application Number:
  US14/098,952
• Publication Number:
  US 20150160819A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Example apparatus and methods concern detecting and responding to a crane gesture performed for a touch or hover-sensitive device. An example apparatus may include a hover-sensitive input/output interface configured to display an object that can be manipulated using a crane gesture. The apparatus may include a proximity detector configured to detect an object in a hover-space associated with the hover-sensitive input/output interface. The apparatus may include fogies configured to change a state of the object from untouched to target to pinched to lifted to released in response to detecting the appearance and movement of bracket points. The appearance of the object may change in response to detecting the state changes.

Claims

1. A method, comprising:

accessing a user interface for an apparatus having a hover-sensitive input/output display; and

selectively controlling the user interface in response to a crane gesture performed using the hover-sensitive input/output display.

2. The method of claim 1, where selectively controlling the user interface includes:

changing a state associated with the user interface to a crane-start state associated with the crane gesture upon detecting two bracket points associated with the display, where the two bracket points are located at least a crane-start minimum distance apart, where the two bracket points are located at most a crane-start maximum distance apart, and where an object displayed on the display is located at least partially between the two bracket points;

changing the state from the crane-start state to a crane-grab state upon detecting that the two bracket points have moved together to within a crane-grab tolerance distance within a crane-grab tolerance period of time, where the crane-grab tolerance distance depends, at least in part, on the size of the object;

changing the state from the crane-grab state to a crane-lift state upon detecting that the two bracket points have either transitioned from two touch points to two hover points or have moved away from the display more than a threshold distance in the z direction;

changing the state from the crane-lift state to a crane-carry state upon detecting that at least one of the two bracket points has been re-positioned more than a movement threshold amount while remaining within the crane-grab tolerance distance; and

changing the state from the crane-lift state to a crane-release state or changing the state from the crane-carry state to the crane-release state upon detecting that the two bracket points have moved apart by more than a crane-release threshold distance,

where changing the state from a first state to a second state includes changing a value in a memory or changing an appearance of the user interface, and

where changing the state to the crane-release state causes the object to be displayed at a location determined by the positions of the two bracket points after the two bracket points have moved apart by more than the crane-release threshold distance.

3. The method of claim 2, comprising:

upon detecting that the state has changed to the crane-start state, updating the display to indicate that the crane-start state has been achieved, displaying a connecting line between the two bracket points, changing the appearance of the object to indicate that the object is a potential target for the crane gesture, generating a crane gesture sound, or generating a crane-start event.

4. The method of claim 3, comprising:

upon detecting that the state has changed to the crane-grab state, updating the display to indicate that the crane-grab state has been achieved, changing the appearance of the object to indicate that the object is an actual target for the crane gesture, generating an object grabbed sound, or generating a crane-grab event.

5. The method of claim 4, comprising;

upon detecting that the state has changed to the crane-lift state, updating the display to indicate that the crane-lift state has been achieved, changing the appearance of the object to indicate that the object has been lifted, displaying a shadow of the object on the display, displaying a point at which the object would appear if released from the crane-lift state, generating an object lifted sound, or generating a crane-lift event.

6. The method of claim 5, comprising:

upon detecting that the state has changed to the crane-carry state, updating the display to indicate that the crane-carry state has been achieved, changing the location of the object on the display, changing the position of the shadow on the display, changing the point at which the object would appear if released on the display, generating an object carry sound, or generating a crane-carry event.

7. The method of claim 6, comprising:

upon detecting that the state has changed to the crane-release state, updating the display to indicate that the crane-release state has been achieved, removing the shadow on the display, positioning the object on the display, generating a crane release sound, or generating a crane-release event.

8. The method of claim 1, where detecting two bracket points includes receiving two touch point events, receiving two hover point entry events, or receiving two hover point to touch point transition events, and

where detecting that the two bracket points have moved together includes receiving a touch point move event, receiving a touch pinch event, receiving a hover point move event, or receiving a hover pinch event.

9. The method of claim 1, where detecting that a bracket point has been re-positioned more than a movement threshold amount while remaining within the crane-grab tolerance distance includes receiving a hover point movement event.

10. The method of claim 1, where detecting that the two bracket points have moved apart by more than a crane-release threshold distance includes receiving a hover point movement event or a hover point spread event.

11. The method of claim 1, comprising:

changing the state from the crane-carry state to the crane-release state upon detecting that the two bracket points have transitioned from two hover points to two touch points.

12. The method of claim 1, comprising:

changing the state from the crane-release state back to the crane-lift state upon detecting that the two bracket points have re-grabbed the object within a re-grab threshold period of time.

13. The method of claim 1, comprising:

controlling an appearance of the object after the state changes to the crane-release state, where the appearance is based, at least in part, on movement of the object in an x-y plane when the crane-release state is detected, on x-y rotation of the object when the crane-release state is detected, or on movement of the object in a z direction when the crane-release state is detected.

14. The method of claim 1, comprising:

changing the state to a crane-discard state upon detecting that the two bracket points have exited the hover space for more than a discard threshold period of time.

15. A computer-readable storage medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:

accessing a user interface on an apparatus having a hover-sensitive input/output display; and

selectively controlling the user interface in response to a crane gesture performed using the hover-sensitive input/output display, where selectively controlling the user interface includes;

changing a state associated with the user interface to a crane-start state associated with the crane gesture upon detecting two bracket points associated with the display, where the two bracket points are located at least a crane-start minimum distance apart, where the two bracket points are located at most a crane-start maximum distance apart, and where an object displayed on the display is located at least partially between the two bracket points, where detecting two bracket points includes receiving two touch point events, receiving two hover point entry events, or receiving two hover point to touch point transition events;

upon detecting that the state has changed to the crane-start state, updating the display to indicate that the crane-start state has been achieved, displaying a connecting line between the two bracket points, changing the appearance of the object to indicate that the object is a potential target for the crane gesture, generating a crane gesture sound, or generating a crane-start event;

changing the state from the crane-start state to a crane-grab state upon detecting that the two bracket points have moved together to within a crane-grab tolerance distance within a crane-grab tolerance period of time, where the crane-grab tolerance distance depends, at least in part, on the size of the object, where detecting that the two bracket points have moved together includes receiving a touch point move event, receiving a touch pinch event, receiving a hover point move event, or receiving a hover pinch event;

upon detecting that the state has changed to the crane-grab state, updating the display to indicate that the crane-grab state has been achieved, changing the appearance of the object to indicate that the object is an actual target for the crane gesture, generating an object grabbed sound, or generating a crab-grab event;

changing the state from the crane-grab state to a crane-lift state upon detecting that the two bracket points have either transitioned from two touch points to two hover points or have moved away from the display more than a threshold distance in the z direction;

upon detecting that the state has changed to the crane-lift state, updating the display to indicate that the crane-lift state has been achieved, changing the appearance of the object to indicate that the object has been lifted, displaying a shadow of the object on the display, displaying a point at which the object would appear if released from the crane-lift state, generating an object lifted sound, or generating a crane-lift event;

changing the state from the crane-lift state to a crane-carry state upon detecting that at least one of the two bracket points has been re-positioned more than a movement threshold amount while remaining within the crane-grab tolerance distance, where detecting that a bracket point has been re-positioned more than a movement threshold amount while remaining within the crane-grab tolerance distance includes receiving a hover point movement event;

upon detecting that the state has changed to the crane-carry state, updating the display to indicate that the crane-carry state has been achieved, changing the location of the object on the display, changing the position of the shadow on the display, changing the point at which the object would appear if released on the display, generating an object carry sound, or generating a crane-carry event;

changing the state from the crane-lift state to a crane-release state or changing the state from the crane-carry state to the crane-release state upon detecting that the two bracket points have moved apart by more than a crane-release threshold distance, where detecting that the two bracket points have moved apart by more than a crane-release threshold distance includes receiving a hover point movement event or a hover point spread event;

changing the state from the crane-carry state to the crane-release state upon detecting that the two bracket points have transitioned from two hover points to two touch points;

upon detecting that the state has changed to the crane-release state, updating the display to indicate that the crane-release state has been achieved, removing the shadow on the display, positioning the object on the display, generating a crane release sound, or generating a crane-release event;

controlling a location of the object after the state changes to the crane-release state, where the location is determined, at least in part, by the positions of the two bracket points after the two bracket points have moved apart by more than the crane-release threshold distance;

controlling an appearance of the object after the state changes to the crane-release state, where the appearance is based, at least in part, on movement of the object in an x-y plane when the crane-release state is detected, on x-y rotation of the object when the crane-release state is detected, or on movement of the object in a z direction when the crane-release state is detected;

changing the state to a crane-discard state upon detecting that the two bracket points have exited the hover space for more than a discard threshold period of time;

changing the state from the crane-release state back to the crane-lift state upon detecting that the two bracket points have re-grabbed the object within a re-grab threshold period of time; and

where changing the state from a first state to a second state includes changing a value in a memory or changing an appearance of the user interface.

16. An apparatus, comprising:

a processor;

a hover-sensitive input/output interface configured to display an item that can be manipulated by a crane gesture;

a memory configured to store a state associated with the item;

a proximity detector configured to detect an object in a hover-space associated with the hover-sensitive input/output interface;

a set of logics configured to manipulate the state of the item in response to the crane gesture; and

an interface configured to connect the processor, the hover-sensitive input/output interface, the proximity detector, the memory, and the set of logics;

the set of logics including;

a first logic configured to change a state associated with the item from untouched to target in response to detecting the item being bracketed by two bracket points, where the bracket points are hover points or touch points;

a second logic configured to change the state from target to pinched upon detecting that the two bracket points have moved to within a pinch threshold distance of the item;

a third logic configured to change the state from pinched to lifted upon detecting that the bracket points have moved more than a lift threshold distance away from the hover-sensitive input/output interface in the z direction; and

a fourth logic configured to change the state from lifted to released upon detecting that the bracket points have moved more than a release threshold distance apart,

where the first logic, second logic, third logic, or fourth logic selectively change the appearance of the item as displayed on the input/output interface upon changing the state.

17. The apparatus of claim 16, where the third logic is configured to reposition the item on the display in response to detecting that the bracket points have moved more than a movement threshold amount in an x or y direction with respect to the input/output interface.

18. The apparatus of claim 17, comprising:

a fifth logic configured to change the state to discarded upon detecting that the bracket points have left the hover-space.

19. The apparatus of claim 18, where the fifth logic is configured to remove the item from the input/output interface upon determining that the state has changed to discarded.

20. The apparatus of cam 15, where the fourth logic is configured to change the state from released to lifted upon detecting that the two bracket points have moved back to within the pinch threshold distance of the item within a re-pinch threshold period of time.