DISPLAY METHOD OF ON-SCREEN KEYBOARD AND COMPUTER PROGRAM PRODUCT AND NON-TRANSITORY COMPUTER READABLE STORAGE MEDIUM THEREOF

• US 20170315627A1
• Filed: 04/28/2017
• Published: 11/02/2017
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: Mitake Information Corp.
• Original Assignee: Bing-Yang Yao
• Inventors: YAO, BING-YANG
• Granted Patent: US 10,452,155 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A display method of an on-screen keyboard, applied to a computing device connected to a physical keyboard, a screen and a sensor, the physical keyboard including a plurality of keys, and the sensor being configured to detect and obtain an operating position image, where the display method of the on-screen keyboard includes: displaying an on-screen keyboard at a topmost layer of a displayed image of the screen; reading the operating position image obtained by the sensor; converting the operating position image into an operating position data, where the operating position data includes relative position data of a key which is touched or is approached; and according to the operating position data, displaying a first key mark at a position corresponding to the key which is touched or is approached on the on-screen keyboard.

Claims

1. A display method of an on-screen keyboard, applied to a computing device connected to a physical keyboard, a screen and a sensor, the physical keyboard comprising a plurality of keys, and the sensor being configured to detect and obtain an operating position image, wherein the display method of the on-screen keyboard comprises:

displaying an on-screen keyboard at a topmost layer of a displayed image of the screen;

reading the operating position image obtained by the sensor;

converting the operating position image into an operating position data, wherein the operating position data comprises a relative position data of a key which is touched or approached; and

according to the operating position data, displaying a first key mark at a position corresponding to the key which is touched or approached on the on-screen keyboard.

2. The display method according to claim 1, wherein the sensor is a camera.

3. The display method according to claim 1, wherein the physical keyboard further responds to a keystroke of the key, so as to output a key scan code to the computing device, and the display method further comprises:

reading the key scan code outputted by the physical keyboard and corresponding to the keystroke of the key; and

according to the key scan code, displaying a second key mark at a position on the on-screen keyboard which corresponds to the keystroke of the key.

4. The display method according to claim 1, after the step of the displaying a first key mark at a position corresponding to the key which is touched or is approached on the on-screen keyboard, further comprising:

if the operating position data comprising the relative position data of the key which is touched or is approached is not received again within a preset time, canceling displaying of the corresponding first key mark.

5. The display method according to claim 3, after the step of the displaying a second key mark at a position corresponding to the keystroke of the key on the on-screen keyboard, further comprising:

identifying, according to the key scan code, whether the keystroke of the key is a locking-key type;

determining whether the key scan code corresponding to the key which is released is received; and

if the key scan code corresponding to the key which is released is received and the key is not a locking-key type, cancelling displaying of the corresponding second key mark.

6. The display method according to claim 1, further comprising:

hiding the on-screen keyboard when the operating position data comprising the relative position data of the key which is touched or is approached and the key scan code are not detected within an idle timeout.

7. The display method according to claim 3, wherein any one of shapes and colors of the first key mark and the second key mark are different.

8. The display method according to claim 3, wherein the first key mark or the second key mark has incomplete transparency.

9. The display method according to claim 3, further comprising:

an automatic correction mechanism which, when the second key mark is individually marked on the on-screen keyboard, calculates a difference between an erroneously determined position and an actual position, and feeds back the difference to the computing device, so as to correct a parameter for generating the operating position data at a next time.

10. The display method according to claim 1, wherein the on-screen keyboard has incomplete transparency.

11. A computer program product, comprising a plurality of computer executable instructions stored in a non-transitory computer readable medium, wherein the computer executable instructions are loaded and executed by a computing device connected to a physical keyboard, a screen and a sensor to cause the computing device to implement a display method of an on-screen keyboard, the physical keyboard comprising a plurality of keys, and the sensor being configured to detect and obtain an operating position image, wherein the display method of the on-screen keyboard comprises:

displaying an on-screen keyboard at a topmost layer of a displayed image of the screen;

reading the operating position image obtained by the sensor;

converting the operating position image into an operating position data, wherein the operating position data comprises a relative position data of a key which is touched or approached; and

according to the operating position data, displaying a first key mark at a position corresponding to the key which is touched or approached on the on-screen keyboard.

12. The computer program product for completing the display method of an on-screen keyboard according to claim 11, wherein the sensor is a camera.

13. The computer program product for completing the display method of an on-screen keyboard according to claim 11, wherein the physical keyboard further responds to a keystroke of the key, so as to output a key scan code to the computing device, and the display method further comprises:

reading the key scan code outputted by the physical keyboard and corresponding to the keystroke of the key; and

according to the key scan code, displaying a second key mark at a position on the on-screen keyboard which corresponds to the keystroke of the key.

14. The computer program product for completing the display method of an on-screen keyboard according to claim 11, after the step of the displaying a first key mark at a position corresponding to the key which is touched or is approached on the on-screen keyboard, further comprising:

if the operating position data comprising the relative position data of the key which is touched or is approached is not received again within a preset time, canceling displaying of the corresponding first key mark.

15. The computer program product for completing the display method of an on-screen keyboard according to claim 13, after the step of the displaying a second key mark at a position corresponding to the keystroke of the key on the on-screen keyboard, further comprising:

identifying, according to the key scan code, whether the keystroke of the key is a locking-key type;

determining whether the key scan code corresponding to the key which is released is received; and

if the key scan code corresponding to the key which is released is received and the key is not a locking-key type, cancelling displaying of the corresponding second key mark.

16. The computer program product for completing the display method of an on-screen keyboard according to claim 11, further comprising:

hiding the on-screen keyboard when the operating position data comprising the relative position data of the key which is touched or is approached and the key scan code are not detected within an idle timeout.

17. The computer program product for completing the display method of an on-screen keyboard according to claim 13, wherein any one of shapes and colors of the first key mark and the second key mark are different.

18. The computer program product for completing the display method of an on-screen keyboard according to claim 13, further comprising:

an automatic correction mechanism which, when the second key mark is individually marked on the on-screen keyboard, calculates a difference between an erroneously determined position and an actual position, and feeds back the difference to the computing device, so as to correct a parameter for generating the operating position data at a next time.

19. A non-transitory computer readable storage medium, storing a computer program comprising a plurality of computer executable instructions that, when executed by a computing device connected to a physical keyboard, a screen and a sensor, cause the computing device to implement a display method of an on-screen keyboard, the physical keyboard comprising a plurality of keys, and the sensor being configured to detect and obtain an operating position image, wherein the display method of the on-screen keyboard comprises:

displaying an on-screen keyboard at a topmost layer of a displayed image of the screen;

reading the operating position image obtained by the sensor;

converting the operating position image into an operating position data, wherein the operating position data comprises a relative position data of a key which is touched or approached; and

according to the operating position data, displaying a first key mark at a position corresponding to the key which is touched or approached on the on-screen keyboard.

20. The non-transitory computer readable storage medium according to claim 19, wherein the physical keyboard further responds to a keystroke of the key, so as to output a key scan code to the computing device, and the display method further comprises:

reading the key scan code outputted by the physical keyboard and corresponding to the keystroke of the key;

according to the key scan code, displaying a second key mark at a position on the on-screen keyboard which corresponds to the keystroke of the key; and

any one of shapes and colors of the first key mark and the second key mark are different.