Enhanced Gesture-Based Image Manipulation

• US 20090228841A1
• Filed: 03/04/2008
• Published: 09/10/2009
• Est. Priority Date: 03/04/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Hildreth, Evan
• Granted Patent: US 9,772,689 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Enhanced image viewing, in which a user'"'"'s gesture is recognized from first and second images, an interaction command corresponding to the recognized user'"'"'s gesture is determined, and, based on the determined interaction command, an image object displayed in a user interface is manipulated.

Claims

1. A computer-implemented method comprising:

recognizing, from first and second images, a user'"'"'s gesture;

determining an interaction command corresponding to the recognized user'"'"'s gesture; and

manipulating, based on the determined interaction command, an image object displayed in a user interface.

2. The method of claim 1, wherein:

the interaction command comprises a selection command, and

manipulating the image object further comprises selecting the image object for further manipulation.

3. The method of claim 2, wherein recognizing the user'"'"'s gesture further comprises:

detecting an arm-extended, fingers-extended, palm-forward hand pose of the user in the first image; and

detecting an arm-extended, fingers curled, palm-down hand pose of the user in the second image.

4. The method of claim 1, wherein:

the interaction command comprises an image pan command, and

manipulating the image object further comprises panning the image object relative to the user interface.

5. The method of claim 4, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

6. The method of claim 5, wherein, in a distance mode, manipulating the image object further comprises:

determining a displacement position of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement position.

7. The method of claim 5, wherein, in a velocity mode, manipulating the image object further comprises:

determining a scroll magnitude and direction correlating to the determined magnitude and direction; and

scrolling the image object based on the determined scroll magnitude and direction.

8. The method of claim 1, wherein:

the interaction command comprises an image zoom command, and

manipulating the image object further comprises zooming the image object relative to the user interface.

9. The method of claim 8, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

10. The method of claim 9, wherein, in a distance mode, manipulating the image object further comprises:

determining a magnification factor correlating to the determined magnitude and direction; and

applying the determined magnification factor to the image object.

11. The method of claim 9, wherein, in a velocity mode, manipulating the image object further comprises:

determining an adjustment magnitude and direction correlating to the determined magnitude and direction; and

iteratively adjusting a magnification factor of the image object based on the determined adjustment magnitude and direction.

12. The method of claim 1, wherein:

the interaction command comprises a rotation command, and

manipulating the object further comprises rotating the image object relative to the user interface.

13. The method of claim 12, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first orientation of a hand of the user in the first image;

detecting a second orientation of the hand of the user in the second image; and

determining an orientation change between the first position and the second position.

14. The method of claim 13, wherein, in a distance mode, manipulating the image object further comprises:

determining displacement orientation of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement orientation.

15. The method of claim 13, wherein, in a velocity mode, manipulating the image object further comprises:

determining an adjustment magnitude and direction correlating to the determined magnitude and direction; and

iteratively adjusting an orientation of the image object based on the determined adjustment magnitude and direction.

16. The method of claim 1, wherein the image object is manipulated if a magnitude of the user'"'"'s gesture exceeds a predetermined threshold.

17. The method of claim 1, wherein the image object is manipulated in a manipulation direction mirroring a direction of the user'"'"'s gesture.

18. The method of claim 1, wherein:

recognizing the user'"'"'s gesture further comprises;

recognizing a first selection gesture,

recognizing a first interaction gesture,

recognizing a de-selection gesture,

recognizing a repositioning gesture,

recognizing a second selection gesture, and

recognizing a second interaction gesture; and

manipulating the image object further comprises;

selecting the image object based on recognizing the first selection gesture,

adjusting, using a single adjustment technique associated with the first and second interaction gestures, the image object based on recognizing the first and second interaction gestures, and

filtering the repositioning gesture.

19. The method of claim 1, wherein:

the interaction command comprises a preview image command, and

manipulating the image object further comprises selecting, from a plurality of preview image objects, the image object.

20. A system comprising:

a user interface configured to display an image; and

a processor configured to;

recognize, from first and second images, a user'"'"'s gesture,

determine an interaction command corresponding to the recognized user'"'"'s gesture, and

manipulate, based on the determined interaction command, the image object.

21. The method of claim 20, further comprising a time-of-flight camera configured to generate the first and second images.

22. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

recognize, from first and second images, a user'"'"'s gesture;

determine an interaction command corresponding to the recognized user'"'"'s gesture; and

manipulate, based on the determined interaction command, an image object displayed in a user interface.