Enhanced input using recognized gestures

• US 8,555,207 B2
• Filed: 02/27/2008
• Issued: 10/08/2013
• Est. Priority Date: 02/27/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan, MacDougall, Francis
• Primary Examiner(s): Nguyen, Cao Kevin
• Application Number:
  US12/038,626
• Publication Number:
  US 20090217211A1
• Time in Patent Office: 2,050 Days
• Field of Search: 345156-184, 715/863, 715/701, 715/707, 715/706, 715/709
• US Class Current: 715/863
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Enhanced input using recognized gestures, in which a user'"'"'s gesture is recognized from first and second images, and a representation of the user is displayed in a central region of a control that further includes interaction elements disposed radially in relation to the central region. The enhanced input also includes interacting with the control based on the recognized user'"'"'s gesture, and controlling an application based on interacting with the control.

Claims

1. A computer-implemented method comprising:

recognizing, from first and second images, a user'"'"'s gesture;

recognizing an identity of the user;

selecting interaction elements based on the recognized identity;

displaying a representation of the user in a control that further comprises the interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with the central region;

interacting with the control based on the recognized user'"'"'s gesture; and

controlling an application based on interacting with the control.

6. The method of claim 1,

wherein recognizing the user'"'"'s gesture further comprises determining a magnitude and direction of the user'"'"'s gesture,

wherein interacting with the control further comprises moving the representation based on the determined magnitude and direction, and

wherein controlling the application further comprises controlling a selected application associated with an interaction element adjacent to the moved representation.

7. The method of claim 1,

wherein the user'"'"'s gesture comprises an enabling gesture;

and

wherein the control is displayed based on recognizing the enabling gesture.

8. The method of claim 1, wherein displaying the representation of the user in the central region further comprises:

displaying the representation of the user outside the central region;

displaying a target image in the central region;

recognizing, from at least a third image, a realignment gesture;

moving the representation in relation to the target image based on the realignment gesture; and

displaying the moved representation in the central region if the moved representation aligns with the target image.

9. The method of claim 1, wherein the control further comprises at least eight interaction elements disposed radially in relation to the central region.

10. The method of claim 1, wherein recognizing the user'"'"'s gesture further comprises:

recognizing a first displacement in a first direction; and

recognizing a second displacement in a second direction.

11. The method of claim 1, wherein interacting with the control further comprises deactivating the control.

12. The method of claim 1, wherein interacting with the control further comprises activating a mouse control mode.

13. The method of claim 12, wherein, in the mouse control mode, further gestures are mapped to mouse events.

14. The method of claim 13, wherein the further gestures further comprises finger motions.

15. The method of claim 1, wherein the user'"'"'s gesture is recognized if detected in front of a plane defined at a predetermined distance in front of a camera.

16. The method of claim 1, wherein controlling the application further comprises interacting with a media center.

2. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

recognize, from first and second images, a user'"'"'s gesture;

recognize an identity of the user;

select interaction elements based on the recognized identity;

display a representation of the user in a control that further comprises the interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with the central region;

interact with the control based on the recognized user'"'"'s gesture; and

control an application based on interacting with the control.

3. A device comprising:

a user interface configured to display a representation of a user in a control that further comprises interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with in the central region; and

a processor configured to;

recognize, from first and second images, a user'"'"'s gesture,

recognize an identity of the user,

select the interaction elements based on the recognized identity,

interact with the control based on the recognized user'"'"'s gesture, and

control an application based on interacting with the control.

4. An apparatus comprising:

means for recognizing, from first and second images, a user'"'"'s gesture;

means for recognizing an identity of the user;

means for selecting interaction elements based on the recognized identity;

means for displaying a representation of the user in a control that further comprises the interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with the central region;

means for interacting with the control based on the recognized user'"'"'s gesture; and

means for controlling an application based on interacting with the control.

5. A computer-implemented method comprising:

recognizing, from first and second images, a user'"'"'s gesture;

recognizing an identity of the user;

selecting interaction elements based on the recognized identity;

displaying a representation of the user in a control that further comprises the interaction elements disposed radially in relation to a central region of the control, wherein alignment of the representation of the user with the central region causes the interaction elements to appear;

interacting with the control based on the recognized user'"'"'s gesture; and

controlling an application based on interacting with the control.

17. A computer program product, tangibly embodied in a non-transitory machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

recognize, from first and second images, a user'"'"'s gesture;

display a representation of the user in a control that further comprises interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with the central region;

interact with the control based on the recognized user'"'"'s gesture; and

control an application based on interacting with the control.

18. A device comprising:

a user interface configured to display a representation of a user in a control that further comprises interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with in the central region; and

a processor configured to;

recognize, from first and second images, a user'"'"'s gesture,

interact with the control based on the recognized user'"'"'s gesture, and

control an application based on interacting with the control.

19. An apparatus comprising:

means for recognizing, from first and second images, a user'"'"'s gesture;

means for displaying a representation of the user in a control that further comprises interaction elements disposed radially in relation to a central region of the control, wherein displaying the interaction elements is deferred until the representation of the user is aligned with the central region;

means for interacting with the control based on the recognized user'"'"'s gesture; and

means for controlling an application based on interacting with the control.

20. A computer-implemented method comprising:

recognizing, from first and second images, a user'"'"'s gesture;

displaying a representation of the user in a control that further comprises interaction elements disposed radially in relation to a central region of the control, wherein alignment of the representation of the user with the central region causes the interaction elements to appear;

interacting with the control based on the recognized user'"'"'s gesture; and

controlling an application based on interacting with the control.