Hand gesture recognition input system and method for a mobile phone

• US 8,064,704 B2
• Filed: 07/11/2007
• Issued: 11/22/2011
• Est. Priority Date: 10/11/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Tea Sul, Kim, Sang Heon
• Primary Examiner(s): KOZIOL, STEPHEN R
• Application Number:
  US11/827,272
• Publication Number:
  US 20080089587A1
• Time in Patent Office: 1,595 Days
• Field of Search: 382/100, 382/103, 382/190, 382/209
• US Class Current: 382/190
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A handheld gesture recognition control apparatus and its method are provided for a mobile phone. The input method of the present invention includes collecting a plurality of images; storing the images as control images; mapping the control images to corresponding control commands; capturing an image taken by a camera as a current image; comparing the current image to the control images; selecting one of the control images as a target control image according to a comparison result; extracting a control command mapped to the target control image; and executing the control command.

Claims

1. An input method for a mobile phone, comprising:

storing a plurality of images as control images;

mapping the control images to corresponding control commands;

capturing an image taken by a camera as a current image;

comparing the captured current image to the control images;

selecting one of the control images as a target control image according to a comparison result;

extracting a control command corresponding to the target control image; and

executing the control command;

wherein selecting one of the control images as a target control image comprises;

determining whether a difference offset between the current image and the control images are less than a first threshold offset;

selecting the control images of which difference offset with the current image as candidate control images;

determining whether a difference offset between the current image and the candidate control images are less than a second threshold offset; and

selecting, if there is a candidate control image of which difference offset with the current image, the candidate control image as the target control image.

2. The input method of claim 1, wherein the control images are hand gesture images.

3. The input method of claim 1, wherein the control commands are commands for controlling pointer operations on a screen of the mobile phone.

4. The input method of claim 3, wherein the pointer operations comprise:

pointer activation, inter-layer navigation, inter-window navigation on an identical layer, object flip, left click, and right click, in a three dimensional coordinate space defined with x, y, and z axes.

5. The input method of claim 1, wherein mapping the control images to corresponding control commands comprises:

building a lookup table having an index field, a control image filed, and control command field.

6. The input method of claim 1, wherein executing the control command comprises:

turning on a screen of the mobile phone;

activating menu windows on the screen;

activating a pointer; and

controlling the pointer to executing a specific operation in association with the menu windows.

7. The input method of claim 6, wherein the screen is implemented with a virtual three-dimensional space having a depth such that the menu windows are arranged in different depths or are arranged in the form of a perspective view.

8. The input method of claim 6, wherein the screen presents shortcut icons for executing application programs, a navigation icon for indicating a current user view, and an icon list representing menu windows on a currently activated layer.

9. An input system for a mobile phone, comprising:

a camera for taking images to be used as control images and capturing an input image;

a memory for storing the control images and control commands mapped to the control images and for temporarily storing the input image;

a screen for displaying at least one menu window and operations of the menu window in accordance with the control commands; and

a controller for controlling a menu window operation in accordance with a control command selected by comparing the input image and the control images;

wherein selecting one of the control images as a target control image comprises;

determining whether a difference offset between the current image and the control images are less than a first threshold offset;

selecting the control images of which difference offset with the current image as candidate control images;

determining whether a difference offset between the current image and the candidate control images are less than a second threshold offset; and

selecting, if there is a candidate control image of which difference offset with the current image, the candidate control image as the target control image.

10. The input system of claim 9, wherein the controller comprises:

an image generator for generating the control images from images taken by the camera;

a control image mapper for mapping the control images to the control commands;

an image comparator for selecting a target control image by comparing the input image with the control images; and

a control image adapter for executing the control commands mapped to the target control image.

11. The input system of claim 9, wherein the screen is implemented with a virtual three-dimensional space having a depth such that the menu windows are arranged in different depths or are arranged in the form of a perspective view.

12. The input system of claim 9, wherein the screen presents shortcut icons for executing application programs, a navigation icon for indicating a current user view, and an icon list representing menu windows on a currently activated layer.

13. The input system of claim 9, wherein the controller controls a pointer for operations of the at least one menu window.

14. The input system of claim 13, wherein the control command comprises a pointer activation command, inter-layer navigation command, inter-window navigation command, object flip command, left click command, and right click command, in a three-dimensional coordinate space defined with x, y, and z axes.