Detecting and interpreting real-world and security gestures on touch and hover sensitive devices

• US 9,367,235 B2
• Filed: 04/02/2010
• Issued: 06/14/2016
• Est. Priority Date: 01/06/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Haggerty, Myra Mary
• Primary Examiner(s): Nguyen, Kevin
• Application Number:
  US12/753,209
• Publication Number:
  US 20100211920A1
• Time in Patent Office: 2,265 Days
• Field of Search: 715/860, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

“Real-world” gestures such as hand or finger movements/orientations that are generally recognized to mean certain things (e.g., an “OK” hand signal generally indicates an affirmative response) can be interpreted by a touch or hover sensitive device to more efficiently and accurately effect intended operations. These gestures can include, but are not limited to, “OK gestures,” “grasp everything gestures,” “stamp of approval gestures,” “circle select gestures,” “X to delete gestures,” “knock to inquire gestures,” “hitchhiker directional gestures,” and “shape gestures.” In addition, gestures can be used to provide identification and allow or deny access to applications, files, and the like.

Claims

1. A method for receiving a confirming gesture formed on or about a sensor panel, comprising:

detecting one or more images at a first time at the sensor panel;

determining that the one or more images at the first time are arranged in a pattern corresponding to a predetermined OK gesture;

determining a centering parameter from the one or more images;

associating the OK gesture with a user interface (UI) element coincident with the centering parameter, the UI element accepting a confirming input; and

providing the confirming input to the UI element.

2. The method of claim 1, further comprising determining that the one or more images at the first time are arranged in a pattern corresponding to a predetermined OK gesture by:

identifying one or more palm edge and pinky features; and

identifying a thumb and finger feature.

15. The method of claim 1, wherein detecting the one or more images at the first time at the sensor panel comprises concurrently detecting the one or more images at the first time at the sensor panel.

16. The method of claim 1, wherein determining that the one or more images at the first time are arranged in the pattern corresponding to the predetermined OK gesture is independent of a movement indicated by the one or more images.

17. The method of claim 1, wherein the one or more images correspond to a hand with a curled thumb and a curled index finger touching at a tip portion of each, and extended middle, ring and pinky fingers.

3. A method for receiving an identification gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel;

determining that the one or more images represent an attempted identification gesture;

associating the attempted identification gesture with a first user interface (UI) element in a UI, the UI including the first UI element and a second UI element;

determining whether movement of the one or more images is arranged in space and time in a pattern and sequence corresponding to a user authorized to access the first UI element;

in accordance with a determination that the movement of the one or more images is arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, granting access to the first UI element, but not the second UI element; and

in accordance with a determination that the movement of the one or more images is not arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, denying access to the first UI element.

4. A non-transitory computer-readable medium comprising program code for receiving a confirming gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at a first time at the sensor panel;

determining that the one or more images at the first time are arranged in a pattern corresponding to a predetermined OK gesture;

determining a centering parameter from the one or more images;

associating the OK gesture with a user interface (UI) element coincident with the centering parameter, the UI element accepting a confirming input; and

providing the confirming input to the UI element.

5. The computer-readable medium of claim 4, the method further comprising determining that the one or more images at the first time are arranged in a pattern corresponding to a predetermined OK gesture by:

identifying one or more palm edge and pinky features; and

identifying a thumb and finger feature.

18. The computer-readable medium of claim 4, wherein detecting the one or more images at the first time at the sensor panel comprises concurrently detecting the one or more images at the first time at the sensor panel.

19. The computer-readable medium of claim 4, wherein determining that the one or more images at the first time are arranged in the pattern corresponding to the predetermined OK gesture is independent of a movement indicated by the one or more images.

20. The computer-readable medium of claim 4, wherein the one or more images correspond to a hand with a curled thumb and a curled index finger touching at a tip portion of each, and extended middle, ring and pinky fingers.

6. A non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined triangle shape gesture;

associating the triangle shape gesture with a user interface (UI) element;

determining that the UI element is within a triangle formed by the triangle shape gesture; and

performing an action comprising zooming the UI element.

7. A non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined V shape gesture;

associating the V shape gesture with a user interface (UI) element;

determining that the V shape gesture comprises an upward movement; and

performing an action comprising translating the UI element in accordance with the upward movement.

8. A non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined inverted V shape gesture;

associating the inverted V shape gesture with a user interface (UI) element;

determining that the inverted V shape gesture comprises a downward movement; and

performing an action comprising translating the UI element in accordance with the downward movement.

9. A non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined wing shape gesture;

associating the wing shape gesture with a user interface (UI) element; and

performing an action comprising sending the UI element to a recipient.

10. A non-transitory computer-readable medium comprising program code for receiving an identification gesture formed on or about a sensor panel, the program code for causing performance of a method comprising:

detecting one or more images at the sensor panel;

determining that the one or more images represent an attempted identification gesture;

associating the attempted identification gesture with a first user interface (UI) element in a UI, the UI including the first UI element and a second UI element;

determining whether movement of the one or more images is arranged in space and time in a pattern and sequence corresponding to a user authorized to access the first UI element;

in accordance with a determination that the movement of the one or more images is arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, granting access to the first UI element, but not the second UI element and

in accordance with a determination that the movement of the one or more images is not arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, denying access to the first UI element.

11. A method for receiving a shape gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined triangle shape gesture;

associating the triangle shape gesture with a user interface (UI) element;

determining that the UI element is within a triangle formed by the triangle shape gesture; and

performing an action comprising zooming the UI element.

12. A method for receiving a shape gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined V shape gesture;

associating the V shape gesture with a user interface (UI) element;

determining that the V shape gesture comprises an upward movement; and

performing an action comprising translating the UI element in accordance with the upward movement.

13. A method for receiving a shape gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined inverted V shape gesture;

associating the inverted V shape gesture with a user interface (UI) element;

determining that the inverted V shape gesture comprises a downward movement; and

performing an action comprising translating the UI element in accordance with the downward movement.

14. A method for receiving a shape gesture formed on or about a sensor panel, comprising:

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined wing shape gesture;

associating the wing shape gesture with a user interface (UI) element; and

performing an action comprising sending the UI element to a recipient.

21. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving a confirming gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at a first time at the sensor panel;

determining that the one or more images at the first time are arranged in a pattern corresponding to a predetermined OK gesture;

determining a centering parameter from the one or more images;

associating the OK gesture with a user interface (UI) element coincident with the centering parameter, the UI element accepting a confirming input; and

providing the confirming input to the UI element.

22. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving an identification gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at the sensor panel;

determining that the one or more images represent an attempted identification gesture;

associating the attempted identification gesture with a first user interface (UI) element in a UI, the UI including the first UI element and a second UI element;

determining whether movement of the one or more images is arranged in space and time in a pattern and sequence corresponding to a user authorized to access the first UI element;

in accordance with a determination that the movement of the one or more images is arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, granting access to the first UI element, but not the second UI element and

in accordance with a determination that the movement of the one or more images is not arranged in space and time in the pattern and the sequence corresponding to the user authorized to access the first UI element, denying access to the first UI element.

23. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined triangle shape gesture;

associating the triangle shape gesture with a user interface (UI) element;

determining that the UI element is within a triangle formed by the triangle shape gesture; and

performing an action comprising zooming the UI element.

24. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined V shape gesture;

associating the V shape gesture with a user interface (UI) element;

determining that the V shape gesture comprises an upward movement; and

performing an action comprising translating the UI element in accordance with the upward movement.

25. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined inverted V shape gesture;

associating the inverted V shape gesture with a user interface (UI) element;

determining that the inverted V shape gesture comprises a downward movement; and

performing an action comprising translating the UI element in accordance with the downward movement.

26. An electronic device comprising:

one or more processors;

a sensor panel; and

a non-transitory computer-readable medium comprising program code for receiving a shape gesture formed on or about the sensor panel, the program code causing the one or more processors to perform a method comprising;

detecting one or more images at the sensor panel;

determining that the one or more images are arranged in a pattern corresponding to a predetermined wing shape gesture;

associating the wing shape gesture with a user interface (UI) element; and

performing an action comprising sending the UI element to a recipient.