Method and Apparatus for a Virtual Keyboard Plane

• US 20140375539A1
• Filed: 06/19/2013
• Published: 12/25/2014
• Est. Priority Date: 06/19/2013
• Status: Abandoned Application

• Current Assignee: TrackThings LLC
• Original Assignee: Thaddeus Gabara
• Inventors: Gabara, Thaddeus John
• Application Number:
  US13/922,165
• Publication Number:
  US 20140375539A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

Two cameras forming a 3-D camera system are used to project the key pattern of the plane from the displayed keyboard of a small display of a smartphone to that of a larger initialization plane of a virtual keyboard which is displaced in a parallel plane and increased in size from the displayed keyboard plane. The displayed keyboard is located on a screen of the display screen of the smartphone. The angular variation of the finger'"'"'s position from the displayed keyboard plane based on the camera image indicates the keyboard character being depressed. Plenoptic cameras can also be used. The displacement distance or baseline of the plenoptic cameras is advantageous to increase. Highlighting the keys on the displayed keyboard when the fingers are in the initialization plane or activation plane of a virtual keyboard either by color, shading, or any other visual means provides positive feedback to the user.

Claims

1. A keyboard apparatus comprising:

a plurality of cameras located on a same surface of a wireless potable unit as a display screen;

a displayed keyboard located on a screen of said display screen;

a virtual keyboard located parallel and above said displayed keyboard on said screen;

said virtual keyboard has dimensions proportionally larger than said displayed keyboard; and

a finger estimating system to identify locations of fingertips relative to said virtual keyboard using images obtained from said cameras.

2. The apparatus of claim 1, further comprising:

an initialization plane of said virtual keyboard having a first working distance; and

an activation plane of said virtual keyboard having a second working distance.

3. The apparatus of claim 2, further comprising:

an elevation angle and a tilt altitude angle determined from at least two camera images calculates said location of fingertips based on said finger estimating system to determine if said location of fingertips are in said initialization or said activation plane.

4. The apparatus of claim 1, further comprising:

a projection of said elevation angle and said azimuth angle of said location of fingertips onto said initialization or said activation plane determines points on an x-y Cartesian coordinate plane.

5. The apparatus of claim 4, further comprising:

a mapping system translating said points on said x-y Cartesian coordinate plane into corresponding keys of said virtual keyboard.

6. The apparatus of claim 5, further comprising:

keys highlighted on said displayed keyboard a first way if location of fingertips are in said initialization plane and keys highlighted on said displayed keyboard a different way if location of fingertips are in said activation plane.

7. The apparatus of claim 5, further comprising:

a text box in said screen of said display screen displaying a sequence of keys corresponding to a corresponding sequence of said fingertips entering said activation plane.

8. The apparatus of claim 1, further comprising:

an embedded algorithm to identify a Plane of depth (POD) of fingertips relative to said virtual keyboard using Light Field Photograph (LFP) images of said cameras, wherein

at least one camera has one or more lens.

9. A keyboard apparatus comprising:

a plurality of plenoptic cameras located on a same surface of a wireless potable unit as a display screen;

a displayed keyboard located on a screen of said display screen;

a virtual keyboard located parallel and above said displayed keyboard on said screen;

said virtual keyboard has dimensions proportionally larger than said displayed keyboard;

an embedded algorithm to identify a Plane of depth (POD) of location of fingertips relative to said virtual keyboard using a Light Field Photograph (LFP) image obtained from said cameras; and

a finger estimating system to identify said locations of fingertips in an x-y Cartesian coordinate plane.

10. The apparatus of claim 9, further comprising:

an initialization plane of said virtual keyboard having a first working distance; and

an activation plane of said virtual keyboard having a second working distance.

11. The apparatus of claim 10, further comprising:

said embedded algorithm determines if location of fingertips are located in said initialization or said activation plane.

12. The apparatus of claim 10, further comprising:

a projection of an elevation angle and an azimuth angle of said location of fingertips onto said initialization or said activation plane determines points on the x-y Cartesian coordinate plane.

13. The apparatus of claim 12, further comprising:

a mapping system translating said points on said x-y Cartesian coordinate plane into corresponding keys of said virtual keyboard.

14. The apparatus of claim 13, further comprising:

keys highlighted on said displayed keyboard a first way if location of fingertips are in said initialization plane and keys highlighted on said displayed keyboard a different way if location of fingertips are in said activation plane.

15. The apparatus of claim 13, further comprising:

a text box in said screen of said display screen displaying a sequence of keys corresponding to a corresponding sequence of said fingertips entering said activation plane.

16. A method of using a virtual keyboard comprising the steps of:

placing a plurality of cameras beside a display screen of a wireless portable unit;

calculating a number of baselines based on said plurality of cameras;

evaluating angles of elevation and altitude for each fingertip in an obtained image from each camera;

calculating a height of each fingertip from said display screen based on said angles using the finger estimation system;

determining a sequence of fingertips that are depressed;

mapping said sequence of fingertips to a sequence of keys; and

printing characters corresponding to said sequence of keys in a text box of said display screen.

17. The method of claim 16, further comprising the steps of:

projecting said elevation angle and said azimuth angle of fingertips onto an initialization or an activation plane to determine points on an x-y Cartesian coordinate plane.

18. The method of claim 16, wherein

said mapping sequence translates said points on said x-y Cartesian coordinate plane into corresponding keys of said virtual keyboard.

19. The method of claim 16, further comprising the steps of:

highlighting keys of said displayed keyboard based on if said location of fingertips are located in said initialization or said activation plane.

20. The method of claim 16, wherein

a text box in said screen of said display screen displaying a sequence of keys corresponding to a corresponding sequence of said fingertips entering said activation plane.