METHOD AND APPARATUS FOR TWO-HANDED COMPUTER USER INTERFACE WITH GESTURE RECOGNITION

• US 20090172606A1
• Filed: 06/30/2008
• Published: 07/02/2009
• Est. Priority Date: 12/31/2007
• Status: Abandoned Application

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.), NMI Safety Systems Limited
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: DUNN, Joseph Wesslund, Super, Boaz J., Dunn, Gregory Joseph
• Application Number:
  US12/164,235
• Publication Number:
  US 20090172606A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method and apparatus for manipulating displayed content using first and second types of human-machine interface in combination are disclosed. Machine operations are divided into two sets and the first type of user interface controls a first set and a second set of operations, while the second type of user interface controls only the second set. In a preferred method embodiment, one hand controls the first set via a mouse interface and the other hand controls the second set via a stereo camera based hand gesture recognition interface. In a preferred apparatus embodiment, the apparatus has a manipulable input device capable of interacting with displayed content and visualization of the displayed content. Additionally, the apparatus has a gesture based input device capable of interacting only with the visualization of the displayed content.

Claims

1. An electronic device, comprising:

a display capable of displaying content;

a manipulable input device capable of enabling a user to interact with at least one of the displayed content and a visualization of the displayed content; and

a gesture based input device capable of enabling the user to interact with the visualization of the displayed content.

2. The electronic device of claim 1, wherein interacting with the visualization of the displayed content comprises at least one of window manipulation, inducing transparency, panning, zooming, or maximizing, minimizing, or hiding windows.

3. The electronic device of claim 1, wherein the visualization of the displayed content reverts to its prior state upon cessation of a gesture.

4. The electronic device of claim 1, wherein the gesture based input device is a vision based input device.

5. The electronic device of claim 4, wherein the vision based input device comprises at least one of a stereo camera system and a monocular camera.

6. The electronic device of claim 1, the electronic device further comprising:

a processor configured to generate a command based on data output from a gesture based input device, wherein the command instructs the electronic device to perform an action on the visualization of the displayed content.

7. The electronic device of claim 6, wherein the data output from a gesture based input device data is created using at least one of luminance data, color data, and depth imaging data.

8. A method performed by an electronic device, comprising:

enabling a user through a manipulable input device to interact with at least one of the displayed content and a visualization of the displayed content; and

enabling the user through a gesture based input device to interact with the visualization of the displayed content.

9. The method of claim 8, wherein interacting with the visualization of the displayed content comprises at least one of window manipulation, inducing transparency, panning, zooming, or maximizing, minimizing, or hiding windows.

10. The method of claim 8, wherein the visualization of the displayed content reverts to its prior state upon cessation of a gesture.

11. The method of claim 8, wherein the gesture based input device is a vision based input device.

12. The method of claim 11, wherein the vision based input device comprises at least one of a stereo camera system and a monocular camera.

13. The method of claim 8, wherein a command is generated based on data output from a gesture based input device, the command instructing the electronic device to perform an action on the visualization of the displayed content.

14. The method of claim 13, wherein the data output from a gesture based input device is created using at least one of luminance data, color data, and depth imaging data.

15. A computer-readable medium having stored thereon a plurality of instructions which, when executed by at least one processor, causes the at least one processor to:

generate displaying content for a display device;

receive from a manipulable input device at least one interaction with at least one of the displayed content and a visualization of the displayed content; and

receive from a gesture based input device at least one interaction with the visualization of the displayed content.

16. The computer-readable medium of claim 15, wherein interacting with the visualization of the displayed content comprises at least one of window manipulation, inducing transparency, panning, zooming, or maximizing, minimizing, or hiding windows.

17. The computer-readable medium of claim 15, wherein the visualization of the displayed content reverts to its prior state upon cessation of a gesture.

18. The computer-readable medium of claim 15, wherein the gesture based input device is a vision based input device.

19. The computer-readable medium of claim 18, wherein the vision based input device comprises at least one of a stereo camera system and a monocular camera.

20. The computer-readable medium of claim 15, wherein the plurality of instructions further causes the at least one processor to:

generate a command based on data output from a gesture based input device, the command instructing the processor to perform an action on the visualization of the displayed content;

wherein the data output from a gesture based input device data are created using at least one of luminance data, color data, and depth imaging data.