Input method and apparatus of portable device for mapping segments of a hand to a plurality of keys

• US 9,448,620 B2
• Filed: 08/21/2013
• Issued: 09/20/2016
• Est. Priority Date: 08/21/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jeonghun
• Primary Examiner(s): Crawley, Keith
• Application Number:
  US13/971,972
• Publication Number:
  US 20140055343A1
• Time in Patent Office: 1,126 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04M 2250/70 methods for entering alphabetical characters, e.g. multi-tap or dictionary disambiguation methods for entering alphabetical characters per se G06F3/0237

Abstract

An input apparatus for use in a portable device is provided. The input apparatus includes a camera for capturing an image; a storage unit which stores a key-hand mapping table for mapping segments of a hand to a plurality of keys, respectively, according to predetermined criteria; a display unit displaying the captured image during an input mode; and a control unit which activates the camera in the input mode, controls the display unit to display the captured image, assign the plural keys to the segments of the hand based on mapping information of the key-hand mapping table, detect an image change in one of the segments displayed on the display unit, and input the key mapped to the segment at which the image change is detected on the display unit.

Claims

1. An input apparatus for use in a portable device, comprising:

a camera for capturing an image of a hand;

a storage unit which stores a key-hand mapping table for mapping segments of a hand to a plurality of keys, respectively, according to predetermined criteria;

a display unit which displays, in an input mode, the captured image thereon; and

a control unit configured to;

activate the camera during the input mode and control the display unit to display the captured image,

display the plurality of keys mapped to the segments of the hand based on the key-hand mapping table, in response to detecting that a shape of the hand satisfies a predetermined condition,

output a message requesting a change of the shape of the hand, in response to detecting that the shape of the hand does not satisfy the predetermined condition,

in response to detecting an image change in one of the segments displayed on the display unit, input a key of the plurality of keys mapped to the segment where the image change is detected on the display unit, and

in response to detecting, via the camera a particular change in the hand folding at least one finger while other fingers are extended, removing the displayed plurality of keys to display a numeric keypad mapped to at least some of the segments of the hand based on the key-hand mapping table.

wherein the control unit controls the display unit to display a character input region displaying one or more characters inputted via the mapped plurality of keys, the character input region displayed separately from the mapped plurality of keys.

2. The apparatus of claim 1, wherein the control unit controls the display unit to display a specific key map including the plural keys mapped to the segments of the hand over the captured image.

3. The apparatus of claim 2, wherein the control unit controls the display unit to display a pictogram corresponding to the specific key map.

4. The apparatus of claim 3, wherein the control unit controls the display unit to highlight the segment pointed by a thumb of the hand.

5. The apparatus of claim 1, wherein the control unit controls the display unit to display the image captured during the input mode on a full screen and display a character input window on the full screen for displaying the key input.

6. The apparatus of claim 1, wherein the control unit tracks a movement of an input object and processes a change in the image caused by the input object as the key input.

7. The apparatus of claim 1, wherein the control unit recognizes the shape of the hand and performs one of switching from a first key map to a second key map, executing a specific function, and inputting a specific key according to the shape of the hand.

8. The apparatus of claim 1, wherein the control unit detects a hand gesture and performs one of switching from a first key map to a second key map, executing a specific function, and inputting a specific key according to the detected hand gesture.

9. The apparatus of claim 1, wherein the segments are defined by finger phalanges, finger tips, and finger joints.

10. The apparatus of claim 1, wherein the key-hand mapping table comprises at least one of key maps for left and right hands.

22. The input apparatus of claim 1, the control unit further configured to:

in response to detecting, via the camera, unfolding of the folded at least one finger, removing the numeric keypad from display to redisplay the mapped plurality of keys.

11. An input method for use in a portable device, the method comprising:

activating a camera during an input mode and displaying an image captured by the camera on a screen;

determining whether a hand is recognized from the image;

assigning a plurality of keys to segments of the hand based on predetermined criteria, in response to detecting that a shape of the hand satisfies a predetermined condition;

outputting a message requesting a change of the shape of the hand, in response to detecting that the shape of the hand does not satisfy the predetermined condition; and

detecting an image change in one of the segments and inputting the key mapped to the segment at which the image change is detected; and

in response to detecting, via the camera, a particular change in the hand folding at least one finger while other fingers are extended, removing the assigned plurality of keys to display a numeric keypad mapped to at least some of the segments of the hand based on the predetermined criteria,

wherein a character input region displaying one or more characters inputted via the assigned plurality of keys is displayed separately from the assigned plurality of keys on the screen.

12. The method of claim 11, further comprising displaying a specific key map including the plurality of keys mapped to the segments of the hand over the captured image.

13. The method of claim 12, wherein displaying comprises displaying a pictogram corresponding to the specific key map.

14. The method of claim 11, wherein inputting the key comprises:

tracking a movement of an input object; and

processing a change in the image caused by the input object as the key input.

15. The method of claim 14, further comprising highlighting the segment pointed by a thumb of the hand.

16. The method of claim 11, further comprising:

recognizing the shape of the hand or a hand gesture; and

performing one of switching from a first key map to a second key map, executing a specific function, and inputting a specific key according to the shape of the hand or the hand gesture.

17. The method of claim 11, wherein the segments are defined by finger phalanges, finger tips, and finger joints.

18. The method of claim 11, wherein the plurality of keys is assigned to segments of the hand based on a key-hand mapping table comprising at least one of key maps for left and right hands.

20. A non-transitory computer-readable storage medium storing one or more programs comprising instructions which, when executed by an electronic device, cause the electronic device to execute the method according to claim 11.

23. The input method of claim 11, further comprising:

in response to detecting, via the camera, unfolding of the folded at least one finger, removing the numeric keypad from display to redisplay the assigned plurality of keys.

19. An input method of a portable device, the method comprising:

capturing an image during an input mode;

recognizing a hand and segments of the hand of the captured image;

mapping keys to the segments, in response to detecting that a shape of the hand satisfies a predetermined condition;

outputting a message requesting a change of the shape of the hand, in response to detecting that the shape of the hand does not satisfy the predetermined condition;

detecting an input of a key mapped to the segment via an input object, the input of the key being activated in response to a change in the image at a particular segment pointed by the input object; and

in response to detecting a particular change in the folding at least one finger while other fingers are extended, removing the mapped keys from display to display a numeric keypad mapped to at least some of the segments tithe hand,

wherein a character input region displaying the input of the key inputted via the mapped keys is displayed separately from the mapped keys.

21. A non-transitory computer-readable storage medium storing one or more programs comprising instructions which, when executed by an electronic device, cause the electronic device to execute the method according to claim 19.

24. The input method of claim 19, further comprising:

in response to detecting unfolding of the folded at least one finger, removing the numeric keypad from display to redisplay the mapped keys.