Finger position sensing and display

• US 9,557,825 B2
• Filed: 06/10/2014
• Issued: 01/31/2017
• Est. Priority Date: 06/10/2014
• Status: Active Grant

• Current Assignee: Maxwell Minoru Nakura-Fan
• Original Assignee: Maxwell Minoru Nakura-Fan
• Inventors: Nakura-Fan, Maxwell Minoru
• Primary Examiner(s): Tzeng, Fred
• Application Number:
  US14/300,587
• Publication Number:
  US 20150355723A1
• Time in Patent Office: 966 Days
• Field of Search: 345/168
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

An apparatus includes a first sensor associated with a key location on a keyboard. The first sensor provides a finger presence signal when a user'"'"'s finger is over the key location. A method includes generating a finger presence signal when a user'"'"'s finger is over a key of a keyboard. The finger presence signal is received after it is generated. The finger presence signal is associated with a key of the keyboard to produce an association of the user'"'"'s finger and the key.

Claims

1. A method for use with a data processing device to detect finger presence above a key, the method comprising:

providing a first fan shaped beam of light above a working area at a first elevation, the working area including indicia of a virtual keyboard having a plurality of keys, the first fan shaped beam of light can be used to detect a selected key;

providing a second fan shaped beam of light substantially parallel to and above the working area at a second elevation, the second elevation is larger than the first elevation, the second fan shaped beam of light is used to detect finger presence;

capturing images of the working area with an image capture device, the image capturing device is positioned above the second fan shaped beam of light and captures images of light from the second fan shaped beam of light;

detecting a presence of a user'"'"'s finger above a key of the plurality of keys from light measured during the capturing; and

outputting an identification of the key obtained during the detecting to the data processing device.

2. The method of claim 1, further comprising:

alternating in time a pulsed output in time for the first fan shaded beam of light and the second fan shaped beam of light.

3. The method of claim 2, wherein the indicia is in the form of a projection of light.

4. The method of claim 2, wherein the indicia is a tangible surface.

5. A system for use with a data processing device, the system comprising:

indicia of a virtual keyboard having a plurality of keys, the virtual keyboard defining a working area;

a first light source and a second light source, the second light source is configured to provide a second fan shaped beam of light substantially parallel to and above the working area at a second elevation, the second elevation is greater than a first elevation, the first light source and the second light source to output pulses alternating in time between the first light source and the second light source;

an image sensing system, the image sensing system to capture images of the working area responsive to light scattered from either the first fan shaped beam of light or the second fan shaded beam of light;

a processor, the processor is configured to analyze the captured images for variations in light intensity within the working area from the second fan shaped beam of light, the processor is further configured to;

generate a first histogram by summing the light intensity in each of a first number of columns;

determine a second number of columns corresponding to a portion of the first histogram that is above a first threshold light intensity value;

generate a second histogram by summing the light intensity in each of a first number of rows in the second number of columns,

determine a second number of rows corresponding to a portion of the second histogram that is above a second threshold light intensity value;

determine an identification of a key associated with finger presence based on the light intensity in the second number of rows and the second number of columns; and

output the identification of the key to the data processing device.

6. The apparatus system of claim 5, wherein the indicia is a projection of light onto a surface.

7. The system of claim 5, wherein the indicia is a tangible surface.

8. The system of claim 5, further comprising:

a second processor communicatively coupled to receive a first signal corresponding to the identification; and

an association component, upon receipt of the first signal the second processor executes the association component to associate the first signal with a key location to form a finger presence association.

9. The system of claim 8, wherein the association component utilizes stored data pertaining to the virtual keyboard to perform the association.

10. The system of claim 9, wherein the stored data in a lookup table is used to establish a keyboard layout and the keyboard layout is selected from the group consisting of QWERTY, AZERTY, Dvorak, ABCDE, XPeRT, and user defined.

11. The system of claim 10, wherein the communication component provides information to a text-to-speech processor to provide an audible message related to the association.

12. The system of claim 9, further comprising:

a communication component, to communicate the association to a user.

13. The system of claim 12, wherein the communication component provides information to a data display to display the association in a region of the data display.

14. A method for use with a data processing device to detect finger presence above a key, the apparatus method comprising:

providing a second fan shaped beam of light substantially parallel to and above a working area at a second elevation, the second elevation is larger than a first elevation, the working area including indicia of a virtual keyboard having a plurality of keys, the second fan shaded beam of light is used to detect finger presence above a key, the first elevation and a first fan shaped beam of light can be used to detect a selection of the key;

capturing images of the working area with an image capture device positioned above the second fan shaded beam of light, the image capture device captures images of light from the second fan shaded beam of light;

detecting a presence of a user'"'"'s finger above a key of the plurality of keys from light received during the capturing; and

outputting an identification of the key obtained during the detecting to the data processing device.

15. The method of claim 14, further comprising:

communicating the identification of the key to a user, wherein the communicating provides information to a data display to display the key on a soft keyboard, the soft keyboard is displayed in a region of the data display.

16. A method for use with a host device to detect finger presence above a key, the method comprising:

providing a second fan shaded beam of light substantially parallel to and above a working area at a second elevation, the second elevation is larger than a first elevation, the working area including indicia of a virtual keyboard having a plurality of keys, the second fan shaded beam of light is used to detect finger presence above a key and a first fan shaped beam of light projected substantially parallel to the working area at the first elevation can be used to detect a potential selection of the key;

capturing images of the working area with an image capture device positioned above the second fan shaped beam of light;

detecting a presence of a user'"'"'s finger above a key of the plurality of keys, the detecting further comprising;

analyzing each of the captured images for variations in light intensity, the analyzing further comprising;

generating a first histogram by summing the light intensity in each of a first number of columns;

determining a second number of columns corresponding to a portion of the first histogram that is above a first threshold light intensity value;

generating a second histogram by summing the light intensity in each of a first number of rows in the second number of columns,

determining a second number of rows corresponding to a portion of the second histogram that is above a second threshold light intensity value;

determining an identification of the key from the light intensity corresponding to the pixels in the second number of rows and the second number of columns; and

outputting an identification of the key obtained during the detecting to a host device.

17. The method of claim 16, further comprising:

communicating the key to a user.

18. The method of claim 17, wherein the communicating to display the key on a data display.

19. The method of claim 16, wherein a keyboard icon is displayed in a region of the data display and the key is displayed on the keyboard icon.

20. The method of claim 17, wherein the communicating is done with an audible communication.

21. A system for use with a data processing device to detect finger presence above a key, the system comprising:

indicia of a virtual keyboard having a plurality of keys, the virtual keyboard defining a working area;

a first light source configured to provide a first fan shaped beam of light above the working are at a first elevation, the first fan shaped beam of light can be used to detect a selected key from the plurality;

a second light source configured to provide a second fan shaped beam of light above the working area at a second elevation, the second elevation is greater than the first elevation, the first light source and the second light source to alternate in time a pulsed output, the second fan shaped beam of light is used to detect finger presence above a key;

an optical sensing system, the optical sensing system to capture images of the working area responsive to light scattered from either the first fan shaped beam of light or the second fan shaped beam of light;

a key selection module, the key selection module to analyze the captured images for variations in light intensity within the working area from the second fan shaped beam of light, the key selection module further configured to;

generate a first histogram by summing the light intensity in each of a first number of columns;

determine a second number of columns corresponding to a portion of the first histogram that is above a first threshold light intensity value;

generate a second histogram by summing the light intensity in each of a first number of rows in the second number of columns,

determine a second number of rows corresponding to a portion of the second histogram that is above a second threshold light intensity value;

determine an identification of the key associated with finger presence based on the light intensity in the second number of rows and the second number of columns; and

output the identification of the key as a finger presence signal to the data processing device.

22. The system of claim 21, further comprising:

an interface for receiving a finger presence signal responsive to the identification; and

an association component coupled to the interface, to receive the finger presence signal for associating a finger location with a key of the virtual keyboard to create a finger presence association; and

a display engine to receive the finger presence association and to display a soft keyboard on a data display, the soft keyboard contains the finger presence association as a hot-key on the soft keyboard.

23. A computer readable storage medium storing program code for causing a data processing device to perform the steps comprising:

generating a second fan shaped beam of light substantially parallel to and above a working area at a second elevation, the second elevation is larger than a first elevation, the working area including indicia of a virtual keyboard having a plurality of keys, the second fan shaped beam of light is used to detect finger presence above a key and a first fan shaped beam of light projected substantially parallel to the working area at the first elevation can be used to detect a potential selection of the key;

capturing images of the working area with an image capture device positioned above the second fan shaped beam of light;

detecting a presence of a user'"'"'s finger above a key of the plurality of keys, the detecting further comprising;

analyzing each of the captured images for variations in light intensity, the analyzing further comprising;

generating a first histogram by summing the light intensity in each of a first number of columns;

determining a second number of columns corresponding to a portion of the first histogram that is above a first threshold light intensity value;

generating a second histogram by summing the light intensity in each of a first number of rows in the second number of columns,

determining a second number of rows corresponding to a portion of the second histogram that is above a second threshold light intensity value;

determining an identification of the key from the light intensity corresponding to the pixels in the second number of rows and the second number of columns; and

outputting an identification of the key obtained during the detecting to the data processing device.

24. The computer readable storage medium of claim 23, further comprising:

associating the finger presence signal with the key of the virtual keyboard to produce an association of the user'"'"'s finger and the key; and

displaying the association on a data display.

25. The computer readable storage medium of claim 24, wherein the association is displayed on a soft keyboard, the soft keyboard is displayed on the data display.