ZOOM-BASED GESTURE USER INTERFACE

• US 20140380241A1
• Filed: 09/15/2014
• Published: 12/25/2014
• Est. Priority Date: 07/05/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Frey, Martin, Hauer, Marcus, Buzzini, Dario, Schaefer, Philipp, Berenson, Adi, Galor, Micha, Metuki, Nili, Shpunt, Alexander
• Granted Patent: US 9,836,201 B2
• Time in Patent Office: Days
• US Class Current: 715/815
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

A user interface method, including presenting by a computer executing a user interface, multiple interactive items on a display. A first sequence of images is captured indicating a position in space of a hand of a user in proximity to the display, and responsively to the position, one of the interactive items is associated with the hand. After associating the item, a second sequence of images is captured indicating a movement of the hand, and responsively to the movement, a size of the one of the items is changed on the display.

Claims

1. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;

responsively to the position, associating one of the interactive items with the hand;

after associating the item, capturing a second sequence of images indicating a movement of the hand; and

responsively to the movement, changing a size of the one of the items on the display.

2. The method according to claim 1, wherein the given interactive item is enlarged in response to the movement of the hand comprising motion away from the display.

3. The method according to claim 1, wherein the given interactive item is associated with a function of the computer, and wherein the method comprises activating the function when the given interactive item has been enlarged, in response to the movement, to at least a predetermined threshold size.

4. The method according to claim 3, wherein activating the function comprises playing a media item associated with the given interactive item.

5. The method according to claim 3, wherein activating the function comprises executing a software application.

6. The method according to claim 3, wherein activating the function comprises displaying an image over a full area of the display.

7. The method according to claim 1, wherein associating the given interactive item comprises choosing the interactive item that is presented at a center of the display.

8. The method according to claim 1, wherein associating the given interactive item comprises detecting, in the first sequence of images, a predetermined gesture of the hand, and initiating association of the given interactive item with the hand responsively to the gesture.

9. The method according to claim 1, wherein at least some of the interactive items contain sub-items, which are not displayed as long as the interactive items presented on the display are smaller than a predetermined size, and wherein enlarging the given interactive item comprises displaying the sub-items contained by the given interactive item when the given interactive item is enlarged to at least the predetermined size.

10. The method according to claim 9, wherein the given interactive item is enlarged in response the movement comprising significant motion away from the display.

11. The method according to claim 1, wherein sensing the position comprises generating a three-dimensional (3D) map of at least a part of body of the user, including the hand.

12. The method according to claim 1, wherein associating the item with the hand comprises identifying the item based on a gaze of the user.

13. The method according to claim 1, wherein associating the item with the hand comprises identifying the item based on a pointing gesture performed the user.

14. The method according to claim 1, wherein presenting the multiple interactive items comprises organizing the multiple interactive items as a grid on the display.

15. The method according to claim 1, wherein changing the size of the one of the items comprises enlarging the one of the items on the display.

16. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;

capturing a second sequence of images indicating a movement of the hand transverse to the display in a first direction; and

responsively to the movement, panning the interactive items on the display in a second direction, which is different from the first direction.

17. The method according to claim 16, wherein the second direction comprises a direction opposite to the first direction.

18. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

receiving, from a handheld remote control device coupled to the computer, a first signal indicating a position in space of a hand of a user in proximity to the handheld remote control device;

responsively to the position, associating one of the interactive items with the hand;

after associating the item, receiving from the handheld remote control device a second signal indicating a movement of the hand relative to the handheld remote control device; and

responsively to the movement, changing a size of the one of the interactive items on the display.