System and method for semi-transparent display of hands over a keyboard in real-time

• US 8,140,970 B2
• Filed: 02/23/2009
• Issued: 03/20/2012
• Est. Priority Date: 02/23/2009
• Status: Active Grant

• Current Assignee: Terrace Licensing LLC (IP Edge LLC)
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Douglas Stuart, Kelley, John Falk, Rudd, James Robert, Torres, Robert J.
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US12/390,679
• Publication Number:
  US 20100214226A1
• Time in Patent Office: 1,121 Days
• Field of Search: 345/168, 715/709, 715/773
• US Class Current: 715/709
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G09B 13/04 Devices used in association with a real typewriter, teleprinter, or the like

Abstract

A method for displaying a virtual keyboard with a semi-transparent overlay of an end user'"'"'s hands. The method begins by receiving a video signal comprising an image of a keyboard and at least one hand belonging to the end user. Next the method creates a virtual keyboard comprising an outline for each key residing in the image of the keyboard. After creating the virtual keyboard, the method sends to a computer screen the virtual keyboard overlaying on top of the image of the keyboard. The overlaying giving an appearance that keys residing in the image of the keyboard obscured by the end user'"'"'s hand are now visible.

Claims

1. A method for displaying a virtual keyboard with a semi-transparent overlay of an end user'"'"'s hands, said method comprising:

receiving, by a processor of a computer system, a video signal comprising an image of a keyboard and at least one hand belonging to said end user;

said processor creating said virtual keyboard comprising an outline for each key residing in said image of said keyboard;

said processor sending to a computer screen said virtual keyboard overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible;

prior to said sending to said computer screen, said processor creating a virtual fingertip for each finger on each hand of said at least one hand;

subsequent to said sending to said computer screen;

said processor sending to said end user data to reproduce, said data comprising at least one character capable of being typed on a keyboard from which said image of said keyboard was created; and

prior to each character of said at least one character being typed by said end user, said processor highlighting a key residing in said virtual keyboard corresponding to a next character to be typed, and highlighting a specific virtual fingertip of a hand of said at least one hand corresponding to a specific finger which would be used to strike said highlighted key.

2. The method of claim 1, said method further comprising:

sending to said end user a list comprising at least one alternative keyboard arrangement which may be displayed on said hands belonging to said end user, said sending said list being in response to receiving from said end user a request for said list;

receiving from said end user identification of a specific alternative keyboard arrangement, said specific alternative keyboard arrangement being from said list; and

sending to said computer screen said specific alternative keyboard arrangement overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible.

3. The method of claim 2, where subsequent to said sending said specific alternative keyboard arrangement:

sending to said end user data to reproduce, said data comprising at least one character capable of being typed on said specific alternative keyboard arrangement; and

prior to each character of said at least one character being typed by said end user, highlighting a key residing in said specific alternative keyboard arrangement corresponding to a next character to be typed, and highlighting a virtual fingertip of a hand of said at least one hand corresponding to a finger which would be used to strike said highlighted key, said virtual fingertip being created prior to said sending to said end user said data.

4. A computer program product, comprising a computer readable physically tangible storage device having a computer-readable program code stored therein, said computer-readable program code containing instructions that when executed by a processor of a computer system implement a method for displaying a virtual keyboard with a semi-transparent overlay of an end user'"'"'s hands, said method comprising:

said processor receiving a video signal comprising an image of a keyboard and at least one hand belonging to said end user;

said processor creating said virtual keyboard comprising an outline for each key residing in said image of said keyboard;

said processor sending to a computer screen said virtual keyboard overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible;

prior to said sending to said computer screen, said processor creating a virtual fingertip for each finger on each hand of said at least one hand;

subsequent to said sending to said computer screen;

said processor sending to said end user data to reproduce, said data comprising at least one character capable of being typed on a keyboard from which said image of said keyboard was created; and

prior to each character of said at least one character being typed by said end user, said processor highlighting a key residing in said virtual keyboard corresponding to a next character to be typed, and highlighting a specific virtual fingertip of a hand of said at least one hand corresponding to a specific finger which would be used to strike said highlighted key.

5. The computer program product of claim 4, said method further comprising:

sending to said end user a list comprising at least one alternative keyboard arrangement which may be displayed on said hands belonging to said end user, said sending said list being in response to receiving from said end user a request for said list;

receiving from said end user identification of a specific alternative keyboard arrangement, said specific alternative keyboard arrangement being from said list; and

sending to said computer screen said specific alternative keyboard arrangement overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible.

6. The computer program product of claim 5, where subsequent to said sending said specific alternative keyboard arrangement:

sending to said end user data to reproduce, said data comprising at least one character capable of being typed on said specific alternative keyboard arrangement; and

prior to each character of said at least one character being typed by said end user, highlighting a key residing in said specific alternative keyboard arrangement corresponding to a next character to be typed, and highlighting a virtual fingertip of a hand of said at least one hand corresponding to a finger which would be used to strike said highlighted key, said virtual fingertip being created prior to said sending to said end user said data.

7. A computing system comprising a processor coupled to a computer-readable physically tangible storage device, said storage device comprising a software application, said software application comprising instructions that when executed by said processor, implement a method for displaying a virtual keyboard with a semi-transparent overlay of an end user'"'"'s hands, said method comprising:

said processor receiving a video signal comprising an image of a keyboard and at least one hand belonging to said end user;

said processor creating said virtual keyboard comprising an outline for each key residing in said image of said keyboard;

said processor sending to a computer screen said virtual keyboard overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible;

prior to said sending to said computer screen, said processor creating a virtual fingertip for each finger on each hand of said at least one hand;

subsequent to said sending to said computer screen;

said processor sending to said end user data to reproduce, said data comprising at least one character capable of being typed on a keyboard from which said image of said keyboard was created; and

prior to each character of said at least one character being typed by said end user, said processor highlighting a key residing in said virtual keyboard corresponding to a next character to be typed, and highlighting a specific virtual fingertip of a hand of said at least one hand corresponding to a specific finger which would be used to strike said highlighted key.

8. The computing system of claim 7, said method further comprising:

sending to said end user a list comprising at least one alternative keyboard arrangement which may be displayed on said hands belonging to said end user, said sending said list being in response to receiving from said end user a request for said list;

receiving from said end user identification of a specific alternative keyboard arrangement, said specific alternative keyboard arrangement being from said list; and

sending to said computer screen said specific alternative keyboard arrangement overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible.

9. The computing system of claim 8, where subsequent to said sending said specific alternative keyboard arrangement:

sending to said end user data to reproduce, said data comprising at least one character capable of being typed on said specific alternative keyboard arrangement; and

prior to each character of said at least one character being typed by said end user, highlighting a key residing in said specific alternative keyboard arrangement corresponding to a next character to be typed, and highlighting a virtual fingertip of a hand of said at least one hand corresponding to a finger which would be used to strike said highlighted key, said virtual fingertip being created prior to said sending to said end user said data.

10. A process for supporting computer infrastructure, said process comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a computing system, wherein said code in combination with the computing system is capable of implementing a method for displaying a virtual keyboard with a semi-transparent overlay of an end user'"'"'s hands, said method comprising:

receiving, by a processor of the computing system, a video signal comprising an image of a keyboard and at least one hand belonging to said end user;

said processor creating said virtual keyboard comprising an outline for each key residing in said image of said keyboard;

said processor sending to a computer screen said virtual keyboard overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible;

prior to said sending to said computer screen, said processor creating a virtual fingertip for each finger on each hand of said at least one hand;

subsequent to said sending to said computer screen;

said processor sending to said end user data to reproduce, said data comprising at least one character capable of being typed on a keyboard from which said image of said keyboard was created; and

prior to each character of said at least one character being typed by said end user, said processor highlighting a key residing in said virtual keyboard corresponding to a next character to be typed, and highlighting a specific virtual fingertip of a hand of said at least one hand corresponding to a specific finger which would be used to strike said highlighted key.

11. The process for supporting computer infrastructure of claim 10, said method further comprising:

sending to said end user a list comprising at least one alternative keyboard arrangement which may be displayed on said hands belonging to said end user, said sending said list being in response to receiving from said end user a request for said list;

receiving from said end user identification of a specific alternative keyboard arrangement, said specific alternative keyboard arrangement being from said list; and

sending to said computer screen said specific alternative keyboard arrangement overlaying on top of said image of said keyboard, said overlaying giving an appearance that keys residing in said image of said keyboard obscured by said at least one hand are visible.

12. The process for supporting computer infrastructure of claim 11, where subsequent to said sending said specific alternative keyboard arrangement:

sending to said end user data to reproduce, said data comprising at least one character capable of being typed on said specific alternative keyboard arrangement; and

prior to each character of said at least one character being typed by said end user, highlighting a key residing in said specific alternative keyboard arrangement corresponding to a next character to be typed, and highlighting a virtual fingertip of a hand of said at least one hand corresponding to a finger which would be used to strike said highlighted key, said virtual fingertip being created prior to said sending to said end user said data.