SYSTEM AND METHOD FOR RECEIVING USER INPUT AND PROGRAM STORAGE MEDIUM THEREOF

• US 20150153950A1
• Filed: 06/12/2014
• Published: 06/04/2015
• Est. Priority Date: 12/02/2013
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: CHANG, Chia-Ming, CHIUEH, Tzi-Cker, CHANG, Ming-Wei
• Granted Patent: US 9,857,971 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A user input method includes the following steps. A virtual keyboard layout and a control region are displayed. The virtual keyboard layout includes a plurality of key subgroups each mapped to a respective one of a plurality of regions of the control region. Locations of an object from at least one captured image are extracted to identify a location of a feature point of the object. A target region in which the feature point is located is determined. Keys mapped to the target region are determined. Movements of the object are translated as input data to the user interface system.

Claims

1. A method for receiving input data comprising:

displaying a virtual keyboard layout and a control region, the virtual keyboard layout including a plurality of key subgroups each mapped to a respective one of a plurality of regions of the control region;

extracting a plurality of locations of an object from at least one captured image to identify a location of a feature point of the object;

determining, among the plurality of regions of the control region, a target region in which the feature point is located;

determining a plurality of keys mapped to the target region; and

translating movements of the object as input data to a user interface system.

2. The method as claimed in claim 1, wherein a respective key subgroup of the virtual keyboard layout is mapped to the respective one of the plurality of regions of the control region based on a position of the plurality of key subgroups in the virtual keyboard layout and a position of the respective one of the plurality of regions in the control region.

3. The method as claimed in claim 1, wherein the step of extracting the plurality of locations of the object includes:

extracting at least one image of at least one of a plurality of user fingers, a plurality of user fingertips and/or the feature point of at least one user hand from the at least one captured image.

4. The method as claimed in claim 1, wherein

the feature point includes at least one point of a back of at least one user hand, and

the at least one point of the back of the at least one user hand includes a thumb-index web or a center of the back of the at least one user hand.

5. The method as claimed in claim 1, wherein

the object includes a glove or gloves optionally having a feature point marker and a plurality of fingernail-type markers, or the object includes a bare hand with/without markers;

and

the steps of extracting the plurality of locations of the object includes;

extracting a plurality of images of the plurality of fingernail-type markers and the feature point marker from the at least one captured image.

6. The method according to claim 1, further comprising a step of:

displaying an active identifier surrounding the plurality of keys mapped to the target region.

7. The method as claimed in 1, further comprising a step of:

displaying at least one of an input result, tracking information and the at least one captured image of the object.

8. The method as claimed in 1, wherein the plurality of regions of the control region are combined into the plurality of key subgroups of the virtual keyboard layout, and a plurality of locations of the plurality of regions of the control region are overlapped with the location of the plural key subgroups of the virtual keyboard layout.

9. A system for receiving user input, the system comprising:

a display, for displaying a virtual keyboard layout and a control region, the virtual keyboard layout including a plurality of key subgroups each mapped to a respective one of a plurality of regions of the control region;

one or more sensors, for sensing movements of an object; and

a computing system, coupled to the one or more sensors and the display, for extracting a plurality of locations of the object from at least one captured image to identify a location of a feature point of the object, for determining a target region, among the plurality of regions of the control region, in which the feature point is located, for determining a plurality of keys mapped to the target region, and for translating the movements of the object as input data to the system.

11. The system as claimed in claim 9, wherein:

the computing system extracts the plurality of locations of the object, thereby analyzing at least one image of a plurality of user fingers, a plurality of user fingertips and/or the feature point of at least one user hand.

12. The system as claimed in claim 9, wherein

the feature point includes at least one point of a back of at least one user hand, and

the at least one point of the back of the at least one user hand includes a thumb-index web or a center of the back of the at least one user hand.

13. The system as claimed in claim 9, wherein:

the object includes a glove or gloves optionally having a feature point marker and a plurality of fingernail-type markers, or the object includes a bare hand with/without markers; and

the computing system extracts a plurality of locations of the plurality of fingernail-type markers and the feature point marker from the at least one captured image.

14. The system as claimed in claim 9, wherein the computing system controls the display to display an active identifier which surrounds the plurality of keys mapped to the target region.

15. The system as claimed in claim 9, wherein the computing system controls the display to display at least one of an input result, tracking information and the at least one capture image of the object.

16. The system as claimed in claim 9, wherein the computing system controls the display to display that the plurality of regions of the control region are combined into the plurality of key subgroups of the virtual keyboard layout, and a plurality of locations of the plurality of regions of the control region are overlapped with a plurality of locations of the plurality of key subgroups of the virtual keyboard layout.

10. The system according as claimed in 9, wherein the computing system controls the display to display a respective key subgroup of the virtual keyboard layout that is mapped to the respective one of the plurality of regions of the control region based on a position of the plurality of key subgroups in the virtual keyboard layout and a position of the respective one of the plurality of regions in the control region.

17. A program storage medium storing a computer program for causing an electronic device to perform the steps of:

displaying a virtual keyboard layout and a control region, the virtual keyboard layout including a plurality of key subgroups each mapped to a respective one of a plurality of regions of the control region;

extracting a plurality of locations of an object from at least one captured image to identify a location of a feature point of the object;

determining, among the plurality of regions of the control region, a target region in which the feature point is located;

determining a plurality of keys mapped to the target region; and

translating movements of the object as input data to a user interface system.

18. The program storage medium as claimed in claim 17, wherein a respective key subgroup of the virtual keyboard layout is mapped to the respective one of the plurality of regions of the control region based on a position of the plurality of key subgroups in the virtual keyboard layout and a position of the respective one of the plurality of regions in the control region.

19. The program storage medium as claimed in claim 17, wherein the step of extracting the plurality of locations of the object includes:

extracting at least one image of at least one of a plurality of user fingers, a plurality of user fingertips and/or the feature point of at least one user hand from the at least one captured image.

20. The program storage medium as claimed in claim 17, wherein

the feature point includes at least one point of a back of at least one user hand, and

the at least one point of the back of the at least one user hand includes a thumb-index web or a center of the back of the at least one user hand.

21. The program storage medium according to claim 17, wherein

the object includes a glove or gloves optionally having a feature point marker and a plurality of fingernail-type markers, or the object includes a bare hand with/without markers;

and

the steps of extracting the plurality of locations of the object includes;

extracting images of the plurality of fingernail-type markers and the feature point marker from the at least one captured image.

22. The program storage medium as claimed in claim 17, wherein the program storage medium causes the electronic device to further perform a step of:

displaying an active identifier surrounding the plurality of keys mapped to the target region.

23. The program storage medium as claimed in claim 17, wherein the program storage medium causes the electronic device to further perform a step of:

displaying at least one of an input result, tracking information and the at least one captured image of the object.

24. The program storage medium as claimed in claim 17, wherein the plurality of regions of the control region are combined into the plurality of key subgroups of the virtual keyboard layout, and a plurality of locations of the plurality of regions of the control region are overlapped with a plurality of locations of the plurality of key subgroups of the virtual keyboard layout.