Hand and finger line grid for hand based interactions

• US 10,345,895 B2
• Filed: 11/07/2017
• Issued: 07/09/2019
• Est. Priority Date: 11/07/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Greenberger, Jeremy A., Greenberger, Zachary M., Rakshit, Sarbajit K.
• Primary Examiner(s): Sherman, Stephen G
• Application Number:
  US15/805,652
• Publication Number:
  US 20190138082A1
• Time in Patent Office: 609 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

An approach is provided that maps a user'"'"'s hand into a number of areas. The approach detects when a selection of one of the mapped areas by the user, such as by using a finger from the user'"'"'s other hand. The approach then identifies an input that corresponds to the area selected by the user. This input is then transmitted to the information handling system for processing.

Claims

1. An information handling system comprising:

one or more processors;

a digital camera accessible by at least one of the processors;

a memory coupled to at least one of the processors; and

a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions comprising;

mapping a user'"'"'s hand into a plurality of areas and at least one keypad switch area;

assigning a first set of virtual keys to the plurality of areas;

in response to detecting a keypad switch selection on the keypad switch area, reassigning the plurality of areas from the first set of virtual keys to a second set of virtual keys;

detecting, by the digital camera, a selection of one of the plurality of areas assigned to a selected one of the second set of virtual keys;

identifying an input corresponding to the selected one of the second set of virtual keys; and

transmitting the identified input to the information handling system.

2. The information handling system of claim 1 further comprising:

configuring a plurality of keypads wherein each of the keypads includes a different plurality of virtual keys.

3. The information handling system of claim 1 wherein the keypad switch request includes a detection of a hand gesture made by the user'"'"'s hand that is mapped to the plurality of areas.

4. The information handling system of claim 1 wherein the detecting is performed by a digital camera that is included in the information handling system and accessible by the processor.

5. The information handling system of claim 1 further comprising:

configuring the plurality of areas of the user'"'"'s hand by detecting hand and finger creases found on the user'"'"'s hand that separates the user'"'"'s hand into the plurality of areas, wherein the detecting is performed by a digital camera that is included in the information handling system and accessible by the processor.

6. The information handling system of claim 1 wherein the information handling system is communicatively coupled with a camera, the actions further comprising:

capturing a first image of a palmar side of the user'"'"'s hand, the first image including a first finger of the user'"'"'s hand;

determining, in the first image, a first and a second side edges of the first finger, a proximal digital crease of the first finger, a middle digital crease of the first finger, a distal digital crease of the first finger, and a tip edge of the first finger, wherein;

a first area of the plurality of areas is defined by the first side edge, the second side edge, the proximal digital crease, and the middle digital crease of the first finger, a second of the plurality of areas is defined by the first side edge, the second side edge, the middle digital crease, and the distal digital crease of the first finger, and a third area of the plurality of areas is defined by the first side edge, the second side edge, the distal digital crease, and the tip edge of the first finger;

causing the camera to capture a second image of the palmar side of the user'"'"'s hand, the second image including the first finger of the user'"'"'s hand;

determining whether the second image includes an object overlaid in one of the first, second, or third areas; and

in response to determining that the second image includes an object overlaid in one of the first, second, or third regions, causing the input corresponding to the area in which the object is overlaid to be provided to an application program executing on the information handling system.

7. A computer program product stored in a non-transitory computer readable storage medium, comprising computer program code that, when executed by an information handling system, performs actions comprising:

mapping a user'"'"'s hand into a plurality of areas and at least one keypad switch area;

assigning a first set of virtual keys to the plurality of areas;

in response to detecting a keypad switch selection on the keypad switch area, reassigning the plurality of areas from the first set of virtual keys to a second set of virtual keys;

detecting a selection of one of the plurality of areas assigned to a selected one of the second set of virtual keys;

identifying an input corresponding to the selected one of the second set of virtual keys; and

transmitting the identified input to the information handling system.

8. The computer program product of claim 7 further comprising:

configuring a plurality of keypads wherein each of the keypads includes a different plurality of virtual keys.

9. The computer program product of claim 7 wherein the keypad switch request includes a detection of a hand gesture made by the user'"'"'s hand that is mapped to the plurality of areas.

10. The computer program product of claim 7 further comprising:

configuring the plurality of areas of the user'"'"'s hand by detecting hand and finger creases found on the user'"'"'s hand that separates the user'"'"'s hand into the plurality of areas, wherein the detecting is performed by a digital camera that is included in the information handling system and accessible by the processor.

11. The computer program product of claim 7 wherein the information handling system is communicatively coupled with a camera, the actions further comprising:

capturing a first image of a palmar side of the user'"'"'s hand, the first image including a first finger of the user'"'"'s hand;

determining, in the first image, a first and a second side edges of the first finger, a proximal digital crease of the first finger, a middle digital crease of the first finger, a distal digital crease of the first finger, and a tip edge of the first finger, wherein;

a first area of the plurality of areas is defined by the first side edge, the second side edge, the proximal digital crease, and the middle digital crease of the first finger, a second of the plurality of areas is defined by the first side edge, the second side edge, the middle digital crease, and the distal digital crease of the first finger, and a third area of the plurality of areas is defined by the first side edge, the second side edge, the distal digital crease, and the tip edge of the first finger;

causing the camera to capture a second image of the palmar side of the user'"'"'s hand, the second image including the first finger of the user'"'"'s hand;

determining whether the second image includes an object overlaid in one of the first, second, or third areas; and

in response to determining that the second image includes an object overlaid in one of the first, second, or third regions, causing the input corresponding to the area in which the object is overlaid to be provided to an application program executing on the information handling system.