Method and apparatus for selecting an object within a user interface by performing a gesture

• US 20090077501A1
• Filed: 09/18/2007
• Published: 03/19/2009
• Est. Priority Date: 09/18/2007
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Goldberg, David, Partridge, Kurt E., Newman, Mark W., Yatani, Koji
• Granted Patent: US 8,122,384 B2
• Time in Patent Office: Days
• US Class Current: 715/846
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

One embodiment of the present invention provides a system that facilitates invoking a command. During operation, the system suggests with a graphic element a gesture to use to invoke a command. The system then receives the gesture from a user at a device. Note that the gesture is received via an input mechanism, and also note that the gesture is a predetermined manipulation of the input mechanism. The system then determines a graphic element within the user interface that is associated with the gesture. Finally, upon determining the object associated with the gesture, the system invokes the command associated with the graphic element.

Claims

1. A method for invoking a command, the method comprising:

suggesting with a graphic element a gesture to use to invoke a command;

receiving the gesture from a user at a device, wherein the gesture is received via an input mechanism, and wherein the gesture is a predetermined manipulation of the input mechanism;

determining a graphic element within the user interface that is associated with the gesture; and

invoking the command associated with the graphic element.

2. The method of claim 1, wherein the graphic element is an icon;

and wherein suggesting the gesture with the graphic element involves displaying an icon associated with the graphic element.

3. The method of claim 2, wherein the color, shape, and/or spatial arrangement of the icon can indicate at least one of:

a direction of the gesture; and

a type of the graphic element associated with the gesture.

4. The method of claim 1, wherein determining the graphic element that is associated with the gesture involves:

determining a location of the gesture; and

determining whether the gesture is made within a predetermined distance of an icon representing the graphic element.

5. The method of claim 1, wherein determining the graphic element that is associated with the gesture involves:

determining a location of the gesture;

determining a direction of the gesture relative to the user interface;

calculating a generalized distance metric, wherein the generalized distance metric is based on;

a location determined by a graphic element;

a location determined by the gesture;

a direction determined by the gesture and

the direction suggested by the graphic element t; and

invoking a command associated with the graphic element that optimizes the generalized distance metric.

6. The method of claim 1:

wherein receiving the gesture further involves receiving a combination of gestures; and

wherein determining the graphic element that is associated with the gesture further involves determining the graphic element within the user interface that is associated with the combination of gestures.

7. The method of claim 1, wherein if the gesture exceeds a first predetermined threshold or if the gesture does not exceed a second predetermined threshold, the method further comprises determining that the gesture is not intended for selecting the graphic element;

and

wherein the first and second thresholds can be based on time and/or distance.

8. The method of claim 1, wherein a graphic element that is within a predetermined distance of a second graphic element is associated with a different gesture from a gesture associated with the second graphic element.

9. The method of claim 1, wherein while receiving the gesture from the user, the method further comprises providing visual feedback to the user indicating a graphic element within the user interface that will be selected upon completion of the gesture.

10. The method of claim 1, further comprising learning the user'"'"'s past gesture habit, thereby reducing user gesture errors.

11. The method of claim 1, wherein the input mechanism can include at least one of:

a touch-sensitive screen;

a mouse;

a trackball;

a touch-sensitive pad;

a light pen;

a joystick; and

an eye-tracking device; and

wherein the device is a handheld device that can include at least one of;

a cell phone;

a Personal Digital Assistant (PDA);

a portable media player;

a gaming system;

a camera;

a tablet computer;

a laptop computer; and

a desktop computer.

12. The method of claim 1, further comprising:

determining that the received gesture is sufficiently different from any possible gesture suggested by any graphic element, thereby allowing the user to invoke a different command in lieu of invoking a command associated with a graphic element which suggests a gesture.

13. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for invoking a command, the method comprising:

suggesting with a graphic element a gesture to use to invoke a command;

receiving the gesture from a user at a device, wherein the gesture is received via an input mechanism, and wherein the gesture is a predetermined manipulation of the input mechanism;

determining a graphic element within the user interface that is associated with the gesture; and

invoking the command associated with the graphic element.

14. The computer-readable storage medium of claim 13, wherein the graphic element is an icon;

and wherein suggesting the gesture with the graphic element involves displaying an icon associated with the graphic element.

15. The computer-readable storage medium of claim 14, wherein the color, shape, and/or spatial arrangement of the icon can indicate at least one of:

a direction of the gesture; and

a type of the graphic element associated with the gesture.

16. The computer-readable storage medium of claim 13, wherein determining the graphic element that is associated with the gesture involves:

determining a location of the gesture; and

determining whether the gesture is made within a predetermined distance of an icon representing the graphic element.

17. The computer-readable storage medium of claim 13, wherein determining the graphic element that is associated with the gesture involves:

determining a location of the gesture;

determining a direction of the gesture relative to the user interface;

calculating a generalized distance metric, wherein the generalized distance metric is based on;

a location determined by a graphic element;

a location determined by the gesture;

a determined by the direction of the gesture; and

the direction suggested by the graphic element; and

invoking a command associated with the graphic element that optimizes the generalized distance metric.

18. The computer-readable storage medium of claim 13:

wherein receiving the gesture further involves receiving a combination of gestures; and

wherein determining the graphic element that is associated with the gesture further involves determining the graphic element within the user interface that is associated with the combination of gestures.

19. The computer-readable storage medium of claim 13, wherein if the gesture exceeds a first predetermined threshold or if the gesture does not exceed a second predetermined threshold, the method further comprises determining that the gesture is not intended for selecting the graphic element;

and

wherein the first and second thresholds can be based on time and/or distance.

20. The computer-readable storage medium of claim 13, wherein a graphic element that is within a predetermined distance of a second graphic element is associated with a different gesture from a gesture associated with the second graphic element.

21. The computer-readable storage medium of claim 13, wherein while receiving the gesture from the user, the method further comprises providing visual feedback to the user indicating a graphic element within the user interface that will be selected upon completion of the gesture.

22. The computer-readable storage medium of claim 13, further comprising learning the user'"'"'s past gesture habit, thereby reducing user gesture errors.

23. The computer-readable storage medium of claim 13, wherein the input mechanism can include at least one of:

a touch-sensitive screen;

a mouse;

a trackball;

a touch-sensitive pad;

a light pen;

a joystick; and

an eye-tracking device; and

wherein the device is a handheld device that can include at least one of;

a cell phone;

a Personal Digital Assistant (PDA);

a portable media player;

a gaming system;

a camera;

a tablet computer;

a laptop computer; and

a desktop computer.

24. The computer-readable storage medium of claim 13, wherein the method further comprises determining that the received gesture is sufficiently different from any possible gesture suggested by any graphic element, thereby allowing the user to invoke a different command in lieu invoking a command associated with a graphic element which suggests a gesture.

25. An apparatus configured for invoking a command, comprising:

a graphic element configured to suggest a gesture to use to invoke a command;

a receiving mechanism configured to receive the gesture from a user at a device, wherein the gesture is received via an input mechanism, and wherein the gesture is a predetermined manipulation of the input mechanism;

a determination mechanism configured to determine a graphic element within the user interface that is associated with the gesture; and

a command invoking mechanism configured to invoke the command associated with the graphic element.