Electronic apparatus for simulating or interfacing a backward compatible human input device by means or control of a gesture recognition system

• US 20150138085A1
• Filed: 07/31/2014
• Published: 05/21/2015
• Est. Priority Date: 07/31/2013
• Status: Abandoned Application

• Current Assignee: Bradley Lloyd Wilk, Eric John Dluhos
• Original Assignee: Bradley Lloyd Wilk, Eric John Dluhos
• Inventors: Dluhos, Eric John, Wilk, Bradley Lloyd
• Application Number:
  US14/121,112
• Publication Number:
  US 20150138085A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 13/105 where the programme performs an input/output emulation function
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

Method and apparatus where human gestures are interpreted by means of software running on a host computer, into screen coordinates and low level commands—keyboard presses, clicks, double-clicks, drag-and-drop, wheel scroll etc.—which are sent to a hardware peripheral, instead of a software based Application Programming Interface, which hardware corrects and polishes the said screen coordinates and low level commands and translates said data by means of emulating, simulating or manipulating the protocol of an actual Human Input Device (HID)—such as a standard keyboard, mouse, joystick, touchpad, etc.—which actual HID-compliant device or simuloid is embedded into the invention, proper, and is in turn connected back into the host computer where it'"'"'s recognized by native drivers as a standard HID device so that it may interact with common end-user programs in the usual manner—but thus be controlled by means of human gestures.

Claims

1. A apparatus where human gestures are interpreted by means of software running on a host computer into screen coordinates and low level commands which are in turn are sent to an actual hardware peripheral, rather than to a software based Application Programming Interface (API), which hardware corrects and polishes the said screen coordinates and low level commands and finally translates said data by means of emulating, simulating or manipulating the protocol of an actual Human Input Device (HID) which actual HID-compliant devices embedded into the invention proper, and is in turn connected back into the host computer where it is recognized by native drivers as a standard HID device so that it may thereby interact with common end-user programs in the usual manner, thus controlling the computer by means of human gestures.

2. The apparatus of claim 1, wherein the low level commands are selected from the group consisting of:

a keyboard press, a click, a double-click, a drag-and-drop, and a wheel scroll.

3. The apparatus of claim 1, wherein the HID is selected from the group consisting of:

a standard keyboard, a mouse, a joystick, and a touchpad.

4. The apparatus of claim 1, further comprising that said gestures, which include analog movements and analog commands and other performances of a real or virtual subject in a real world or virtual environment, respectively, are captured by means of a camera or other sensor apparatus, real or virtual, and are presented by means of a suitable interface to a front end processing device, a computer employing a central processor, memory, secondary storage media and software, or some other form of electronic or electromechanical apparatus containing stored procedures, where they are first stored as raw sensory data and then processed by means of a plurality of filters and/or interpretation or translation algorithms, the intermediate output result of which is a plurality of two dimensional X, Y and or Z movement coordinates;

keyboard keystroke commands; and

/or joystick directional commands; and

/or mouse button click, doubleclick and wheel-scroll commands, in the form of digital codes, or other electronic signals presented as ongoing feed-forward output, or stored in an accessible table, array, queue, stack or other data structure.

5. The apparatus of claim 1, further comprising that said coordinates and commands are the processed interpretations of the said front end apparatus which is employed for its raw analysis purposes to decode or otherwise interpret the said input gestures so as to attain the said coordinates and commands which coordinates and commands being similar in terms of value to those normally gathered by an actual HID-compliant device such as a joystick, mouse, or keyboard.

6. The apparatus of claim 5, wherein the applied gestures are interpreted, filtered or otherwise translated or converted into said output coordinates and/or said output commands are indigenous to the particular gesture recognition system being employed at the time of conversion to produce said coordinates and command outputs, and the invention does not limit itself as to any specific gesture recognition method which may be employed by a person or persons skilled in the art for this purpose of producing these said intermediary outputs of coordinates and/or commands, with no claim being made here as to any specific algorithm of gesture recognition.

7. The apparatus of claim 1, further comprising that the analog movements and analog commands of said input gestures, therefore, having thus been translated into resulting digital interpretation as the resulting said digital codes, data or other electronic signals, represented as digital coordinates and commands, respectively, collected by the gesture recognition apparatus, may be stored temporarily into a buffer, a memory storage device such as RAM or other electronic or electromechanical device to hold the said coordinates and/or commands prior to utilization for some end-user purpose and end-user application making use of them.

8. The apparatus of claim 7, wherein said buffer or memory storage device is configured to contain one or more suitable data structures for this purpose of utilization.

9. The apparatus of claim 8, wherein said data structures include tables, arrays, and queues, stacks, and data structures invoking an Application Programming Interface (API).

10. The apparatus of claim 1, further comprising that the said coordinates and/or commands are applied for use by end user devices and end-user applications as follows:

where obtainment of the output coordinates and/or output commands is not by means of a straight feed-forward manner such that they are first stored into said buffer on the gesture recognition side of the system, they may be first accessed and collected from said storage buffer by means of a hardware and/or software interface solution which is algorithmically designed specifically to communicate with the indigenous access protocols of the said buffer so as to facilitate the collection of the said coordinates and commands.

11. The apparatus of claim 1, further comprising that the said apparatus having interpreted said coordinates and/or commands, transfers the said coordinates and/or commands into its own memory unit buffer, structures as an arrays, queues, stacks, amongst others;

the apparatus then processes said interpreted coordinates and/or commands, employing conventional filtering methods, to ultimately polish or otherwise refine in some manner the raw state of the coordinates to make them more presentable.

12. The apparatus of claim 11, wherein the filters include curve re-shaping and/or smoothing algorithms and techniques.

13. The apparatus of claim 1, further comprising that the said coordinates and/or commands, either polished as described, or raw as captured directly from the gesture recognition system by means of said capture interface, are then fed by the apparatus to an inclusive hardware based or software based emulator which invokes one or more Human Input Device (HID) compliant device interfaces so as to enter into communication with resident drivers of an end-user device running end-user application software.

14. The apparatus of claim 13, wherein the HID is selected from the group consisting of a joystick, a mouse, a keyboard.

15. The apparatus of claim 13, wherein the end-user device is selected from the group consisting of:

a personal computer and a mobile computing device.

16. The apparatus of claim 1, further comprising that the final result is that the said coordinates and/or commands are captured by the apparatus, processed by the apparatus, and presented by the apparatus to the end-user device and end-user application in a manner that said end-user device and end-user application receives the coordinates and/or commands in a form indistinguishable from those of HID-compliant devices.

17. The apparatus of claim 1, further comprising that the hardware employed by the invention also provides fittings to accommodate connecting conventional HID-compliant devices to the apparatus, thereby allowing the user to override the gesture-invoked coordinates and/or commands.

18. The apparatus of claim 17, further comprising that a plurality of HID-compliant protocol devices are used by the apparatus so that it may facilitate the connection of actual HID-compliant hardware with a practically seamless integration of said override.