Enhanced gesture-based image manipulation

• US 9,772,689 B2
• Filed: 03/04/2008
• Issued: 09/26/2017
• Est. Priority Date: 03/04/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan
• Primary Examiner(s): ENGLAND, SARA M
• Application Number:
  US12/041,927
• Publication Number:
  US 20090228841A1
• Time in Patent Office: 3,493 Days
• Field of Search: 715810, 715863, 715864
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Enhanced image viewing, in which a user'"'"'s gesture is recognized from first and second images, an interaction command corresponding to the recognized user'"'"'s gesture is determined, and, based on the determined interaction command, an image object displayed in a user interface is manipulated.

Claims

1. A computer-implemented method comprising:

causing a display to render an image object in a user interface;

determining a position of a user'"'"'s torso;

defining, with a processing unit, a plane relative to the user'"'"'s torso, based on the determined position;

ignoring, by the processing unit, movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determining, with the processing unit, an extent to which the user is able to reach;

recognizing, from first and second images captured by the camera, a user'"'"'s gesture performed between the camera and the plane;

determining, with the processing unit, an interaction command corresponding to the recognized user'"'"'s gesture; and

manipulating, with a processing unit and based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user'"'"'s gesture to the determined extent to which the user is able to reach.

2. The method of claim 1, wherein:

the interaction command comprises a selection command, and manipulating the image object further comprises selecting the image object for further manipulation.

3. The method of claim 1, wherein:

the interaction command comprises an image pan command, and manipulating the image object further comprises panning the image object relative to the user interface.

4. The method of claim 3, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

5. The method of claim 4, wherein, in a distance mode, manipulating the image object further comprises:

determining a displacement position of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement position.

6. The method of claim 4, wherein, in a velocity mode, the magnitude of the manipulation comprises a scroll magnitude, and the scroll magnitude and direction correlates to the determined magnitude and direction;

and

manipulating the image object further comprises scrolling the image object based on the determined scroll magnitude and direction.

7. The method of claim 1, wherein:

the interaction command comprises an image zoom command, and manipulating the image object further comprises zooming the image object relative to the user interface.

8. The method of claim 7, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

9. The method of claim 8, wherein, in a distance mode, manipulating the image object further comprises:

determining a magnification factor correlating to the determined magnitude and direction; and

applying the determined magnification factor to the image object.

10. The method of claim 8, wherein, in a velocity mode, the magnitude of the manipulation comprises an adjustment magnitude, and the adjustment magnitude and direction correlates to the determined magnitude and direction;

and

manipulating the image object further comprises iteratively adjusting a magnification factor of the image object based on the determined adjustment magnitude and direction.

11. The method of claim 1, wherein:

the interaction command comprises a rotation command, and manipulating the object further comprises rotating the image object relative to the user interface.

12. The method of claim 11, wherein recognizing the user'"'"'s gesture further comprises:

detecting a first orientation of a hand of the user in the first image;

detecting a second orientation of the hand of the user in the second image; and

determining an orientation change between the first position and the second position.

13. The method of claim 12, wherein, in a distance mode, manipulating the image object further comprises:

determining displacement orientation of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement orientation.

14. The method of claim 12, wherein, in a velocity mode, manipulating the image object further comprises:

the magnitude of the manipulation comprises an adjustment magnitude, and the adjustment magnitude and direction correlates to the determined magnitude and direction; and

manipulating the image object further comprises iteratively adjusting an orientation of the image object based on the determined adjustment magnitude and direction.

15. The method of claim 1, wherein the image object is manipulated in a manipulation direction mirroring a direction of the user'"'"'s gesture.

16. The method of claim 1, wherein:

recognizing the user'"'"'s gesture further comprises;

recognizing a first selection gesture, recognizing a first interaction gesture, recognizing a de-selection gesture, recognizing a repositioning gesture, recognizing a second selection gesture, and recognizing a second interaction gesture; and

manipulating the image object further comprises;

selecting the image object based on recognizing the first selection gesture, adjusting, using a single adjustment technique associated with the first and second interaction gestures, the image object based on recognizing the first and second interaction gestures, and filtering the repositioning gesture.

17. The method of claim 1, wherein:

the interaction command comprises a preview image command, and manipulating the image object further comprises selecting, from a plurality of preview image objects, the image object.

18. The computer-implemented method if claim 1, wherein causing the display to render the image object in the user interface comprises sending information to the display via a display interface of a computer.

19. The computer-implemented method if claim 1, further comprising receiving the first and second images from the camera via a digital input interface of a computer.

22. The computer-implemented method if claim 1, wherein determining an extent of the user'"'"'s reach comprises using an anatomical model.

20. An apparatus comprising:

a processor configured to;

cause a display to render an image object in a user interface;

determine a position of a user'"'"'s torso;

define a plane relative to the user'"'"'s torso, based on the determined position;

ignore movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determine an extent to which the user is able to reach;

recognize, from first and second images captured by the camera, a user'"'"'s gesture performed between the camera and the plane;

determine an interaction command corresponding to the recognized user'"'"'s gesture; and

manipulate, based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user'"'"'s gesture to the determined extent to which the user is able to reach.

21. A non-transitory processor-readable medium comprising processor-readable instructions configured to cause a processor to:

causing a display to render an image object in a user interface;

determine a position of a user'"'"'s torso;

define a plane relative to the user'"'"'s torso, based on the determined position;

ignore movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determine an extent to which the user is able to reach;

recognize, from first and second images captured by the camera, a user'"'"'s gesture performed between the camera and the plane;

determine an interaction command corresponding to the recognized user'"'"'s gesture; and

manipulate, based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user'"'"'s gesture to the determined extent to which the user is able to reach.