ENHANCED INPUT USING RECOGNIZED GESTURES

• US 20090217211A1
• Filed: 02/27/2008
• Published: 08/27/2009
• Est. Priority Date: 02/27/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: HILDRETH, Evan, MacDougall, Francis
• Granted Patent: US 8,555,207 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Enhanced input using recognized gestures, in which a user'"'"'s gesture is recognized from first and second images, and a representation of the user is displayed in a central region of a control that further includes interaction elements disposed radially in relation to the central region. The enhanced input also includes interacting with the control based on the recognized user'"'"'s gesture, and controlling an application based on interacting with the control.

Claims

1. A computer-implemented method comprising:

recognizing, from first and second images, a user'"'"'s gesture;

displaying a representation of the user in a central region of a control that further comprises interaction elements disposed radially in relation to the central region;

interacting with the control based on the recognized user'"'"'s gesture; and

controlling an application based on interacting with the control.

2. The method of claim 1, wherein the user'"'"'s gesture further comprises a single or multiple finger gesture;

a single hand gesture;

a single hand and arm gesture;

a single hand and arm, and body gesture;

a bimanual gesture;

a change in a head pose or posture;

a change in an eye position;

a change in facial expression;

or a change in a body pose or posture.

3. The method of claim 1, wherein the representation of the user further comprises a cursor, an image, an icon, or an avatar.

4. The method of claim 1, wherein the control comprises a square shape, a circular shape, or a circular sector shape.

5. The method of claim 1, wherein:

each interaction element further comprises an icon associated with a candidate application;

wherein interacting with the control further comprises highlighting the icon associated with the recognized user'"'"'s gesture, and

wherein controlling the application further comprises controlling the candidate application associated with the highlighted icon.

6. The method of claim 1, wherein the user'"'"'s gesture comprises a two- or three-dimensional position displacement, or a transformation without position displacement.

7. The method of claim 1,

wherein recognizing the user'"'"'s gesture further comprises determining a magnitude and direction of the user'"'"'s gesture,

wherein interacting with the control further comprises moving the representation based on the determined magnitude and direction, and

wherein controlling the application further comprises controlling a selected application associated with an interaction element adjacent to the moved representation.

8. The method of claim 1,

wherein the user'"'"'s gesture comprises an enabling gesture;

and

wherein the control is displayed based on recognizing the enabling gesture.

9. The method of claim 1, wherein displaying the representation of the user in the central region further comprises:

displaying the representation of the user outside the central region;

displaying a target image in the central region;

recognizing, from at least a third image, a realignment gesture;

moving the representation in relation to the target image based on the realignment gesture; and

displaying the moved representation in the central region if the moved representation aligns with the target image.

10. The method of claim 1, wherein the control further comprises at least eight interaction elements disposed radially in relation to the central region.

11. The method of claim 1, wherein recognizing the user'"'"'s gesture further comprises:

recognizing a first displacement in a first direction; and

recognizing a second displacement in a second direction.

12. The method of claim 1, wherein interacting with the control further comprises deactivating the control.

13. The method of claim 1, wherein interacting with the control further comprises activating a mouse control mode.

14. The method of claim 13, wherein, in the mouse control mode, further gestures are mapped to mouse events.

15. The method of claim 14, wherein the further gestures further comprises finger motions.

16. The method of claim 1, further comprising:

recognizing an identity of the user; and

selecting the interaction elements based on the recognized identity.

17. The method of claim 1, wherein the user'"'"'s gesture is recognized if detected in front of a plane defined at a predetermined distance in front of a camera.

18. The method of claim 1, wherein controlling the application further comprises interacting with a media center.

19. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

recognize, from first and second images, a user'"'"'s gesture;

display a representation of the user in a central region of a control that further comprises interaction elements disposed radially in relation to the central region;

interact with the control based on the recognized user'"'"'s gesture; and

control an application based on interacting with the control.

20. A device comprising:

a user interface configured to display a representation of a user in a central region of a control that further comprises interaction elements disposed radially in relation to the central region; and

a processor configured to;

recognize, from first and second images, a user'"'"'s gesture, interact with the control based on the recognized user'"'"'s gesture, and control an application based on interacting with the control.