Detecting and Interpreting Real-World and Security Gestures on Touch and Hover Sensitive Devices

• US 20100192109A1
• Filed: 04/02/2010
• Published: 07/29/2010
• Est. Priority Date: 01/06/2007
• Status: Active Grant

• Current Assignee: Myra Mary Haggerty, Wayne Carl Westerman
• Original Assignee: Myra Mary Haggerty, Wayne Carl Westerman
• Inventors: Haggerty, Myra Mary, WESTERMAN, Wayne Carl
• Granted Patent: US 9,158,454 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

“Real-world” gestures such as hand or finger movements/orientations that are generally recognized to mean certain things (e.g., an “OK” hand signal generally indicates an affirmative response) can be interpreted by a touch or hover sensitive device to more efficiently and accurately effect intended operations. These gestures can include, but are not limited to, “OK gestures,” “grasp everything gestures,” “stamp of approval gestures,” “circle select gestures,” “X to delete gestures,” “knock to inquire gestures,” “hitchhiker directional gestures,” and “shape gestures.” In addition, gestures can be used to provide identification and allow or deny access to applications, files, and the like.

Claims

1. A method for receiving a directional gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel generated from a hand with thumb extended and all other fingers curled;

determining that the one or more images are arranged in a pattern corresponding to a predetermined hitchhiker gesture;

determining a directional parameter from the one or more images;

associating the hitchhiker gesture with a user interface (UI) element; and

performing a directional action in accordance with the UI element.

2. The method of claim 1, further comprising determining that the one or more images are arranged in a pattern corresponding to a predetermined hitchhiker gesture by:

identifying four curled finger features;

identifying an extended thumb feature; and

identifying an palm heel feature.

3. A method for receiving a shape gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel generated from a hand formed in a particular shape;

determining that the one or more images are arranged in a pattern corresponding to a predetermined shape gesture;

associating the shape gesture with a user interface (UI) element; and

performing an action associated with the shape gesture upon the UI element.

4. The method of claim 3, wherein the shape gesture is a triangle shape gesture.

5. The method of claim 3, wherein the shape gesture is a V shape gesture.

6. The method of claim 3, wherein the shape gesture is an inverted V shape gesture.

7. The method of claim 3, wherein the shape gesture is a wing shape gesture.

8. A computer-readable medium comprising program code for receiving an affirmative gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel generated from a clenched fist;

determining that the one or more images are arranged in a pattern corresponding to a predetermined stamp of approval gesture;

determining a centering parameter from the one or more images;

associating the stamp of approval gesture with a user interface (UI) element coincident with the centering parameter; and

performing an affirmative action in accordance with the UI element.

9. The computer-readable medium of claim 8, the method further comprising determining that the one or more images are arranged in a pattern corresponding to a predetermined stamp of approval gesture by:

tracking the one or more images over time; and

determining that the one or more images rapidly appeared and then disappeared.

10. A computer-readable medium comprising program code for receiving a deletion gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting a first image at the sensor panel generated from a single finger and representative of a first touch;

tracking movement of the first image over time;

detecting a second image at the sensor panel generate from the same finger and representative of a second touch;

tracking movement of the second image over time;

determining that the movement of the first and second images are arranged in space and time in a pattern and sequence corresponding to a predetermined X to delete gesture;

determining an intersection of the movements of the first and second images;

associating the X to delete gesture with a user interface (UI) element coincident with the intersection; and

performing a deleting action in accordance with the UI element coincident with the intersection.

11. The computer-readable medium of claim 10, the method further comprising determining that the movement of the first and second images are arranged in space and time in a pattern and sequence corresponding to a predetermined X to delete gesture by:

determining that the first and second images were detected within a predetermined time interval.

12. A computer-readable medium comprising program code for receiving a directional gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel generated from a hand with thumb extended and all other fingers curled;

determining that the one or more images are arranged in a pattern corresponding to a predetermined hitchhiker gesture;

determining a directional parameter from the one or more images;

associating the hitchhiker gesture with a user interface (UI) element; and

performing a directional action in accordance with the UI element.

13. The computer-readable medium of claim 12, the method further comprising determining that the one or more images are arranged in a pattern corresponding to a predetermined hitchhiker gesture by:

identifying four curled finger features;

identifying an extended thumb feature; and

identifying an palm heel feature.

14. A method for receiving a security gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel generated from two fingers and two thumbs of two hands touching near a center of the sensor panel;

determining that the one or more images correspond to a predetermined unlock gesture; and

performing an unlocking action.

15. The method of claim 14, further comprising determining that the one or more images correspond to a predetermined unlock gesture by

identifying four initial images corresponding to the two fingers and two thumbs from the two hands;

and

determining that the four initial images move towards the corners of the sensor panel over time.

16. A method for receiving a security gesture formed on or about a sensor panel, comprising:

detecting four images at the sensor panel generated from two fingers and two thumbs of two hands, each image touching near a different corner of the sensor panel;

determining that the four images correspond to a predetermined lock gesture; and

performing an locking action.

17. The method of claim 16, further comprising determining that the one or more images correspond to a predetermined lock gesture by

determining that the four initial images move towards a center of the sensor panel over time.

18. A method for receiving a hand edge gesture formed on or about a sensor panel, comprising:

detecting one or more images at a first location on the sensor panel generated from a side of a palm and pinky finger rotating or translating to a second location on the sensor panel;

determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture; and

performing a predetermined action in accordance with the detected hand edge gesture.

19. The method of claim 18, further comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images rotate over time.

20. The method of claim 18, further comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images translate over time.

21. The method of claim 18, further comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images move up and down in a vertical direction over time.

22. The method of claim 18, further comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that one or more of the initial images corresponding to the pinky finger rotate over time while the one or more initial images corresponding to the palm remain substantially stationary.

23. A method for receiving a framing gesture formed on or about a sensor panel, comprising:

detecting a plurality of images at the sensor panel generated from the palms and pinky fingers of two hands touching the sensor panel in an approximate upside-down U shape;

determining that the one or more images correspond to a predetermined framing gesture;

computing a frame area based on a location of the plurality of detected images; and

performing an action within the frame area.

24. A computer-readable medium comprising program code for receiving a security gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel generated from two fingers and two thumbs of two hands touching near a center of the sensor panel;

determining that the one or more images correspond to a predetermined unlock gesture; and

performing an unlocking action.

25. The computer-readable medium of claim 24, the program code further for causing performance of a method comprising determining that the one or more images correspond to a predetermined unlock gesture by

identifying four initial images corresponding to the two fingers and two thumbs from the two hands;

and

determining that the four initial images move towards the corners of the sensor panel over time.

26. A computer-readable medium comprising program code for receiving a security gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting four images at the sensor panel generated from two fingers and two thumbs of two hands, each image touching near a different corner of the sensor panel;

determining that the four images correspond to a predetermined lock gesture; and

performing an locking action.

27. The computer-readable medium of claim 26, the program code further for causing performance of a method comprising determining that the one or more images correspond to a predetermined lock gesture by

determining that the four initial images move towards a center of the sensor panel over time.

28. A computer-readable medium comprising program code for receiving a hand edge gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at a first location on the sensor panel generated from a side of a palm and pinky finger rotating or translating to a second location on the sensor panel;

determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture; and

performing a predetermined action in accordance with the detected hand edge gesture.

29. The computer-readable medium of claim 28, the program code further for causing performance of a method comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images rotate over time.

30. The computer-readable medium of claim 28, the program code further for causing performance of a method comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images translate over time.

31. The computer-readable medium of claim 28, the program code further for causing performance of a method comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that the one or more initial images move up and down in a vertical direction over time.

32. The computer-readable medium of claim 28, the program code further for causing performance of a method comprising determining that the one or more images are arranged in space and time in a pattern and sequence corresponding to a predetermined hand edge gesture by

identifying one or more initial images corresponding to the side of the palm and pinky finger from a single hand;

and

determining that one or more of the initial images corresponding to the pinky finger rotate over time while the one or more initial images corresponding to the palm remain substantially stationary.

33. A computer-readable medium comprising program code for receiving a framing gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting a plurality of images at the sensor panel generated from the palms and pinky fingers of two hands touching the sensor panel in an approximate upside-down U shape;

determining that the one or more images correspond to a predetermined framing gesture;

computing a frame area based on a location of the plurality of detected images; and

performing an action within the frame area.