Wrist-pendent wireless optical keyboard

• US 6,097,374 A
• Filed: 03/06/1998
• Issued: 08/01/2000
• Est. Priority Date: 03/06/1997
• Status: Expired due to Term

• Current Assignee: Harmonic Research, Inc.
• Original Assignee: Robert Bruce Howard
• Inventors: Howard, Robert Bruce
• Primary Examiner(s): WU, XIAO MIN
• Application Number:
  US09/035,983
• Time in Patent Office: 879 Days
• Field of Search: 345/156, 345/157, 345/158, 345/173, 345/174, 345/175, 345/168, 345/169, 340/20, 340/22
• US Class Current: 345/168
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/0231 Cordless keyboards
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0423 using sweeping light beams, e.g. using rotating or vibrating mirror
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2220/201 for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument
• CPC Class Codes: G10H 2220/415 Infrared beams
• CPC Class Codes: G10H 2240/211 Wireless transmission, e.g. of music parameters or control data by radio, infrared or ultrasound
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analogue or digital, e.g. DECT, GSM, UMTS

Abstract

A method and apparatus is provided for sensing, encoding and transmitting electronic keyboard data, pointing device control signals and voice input. The apparatus includes a wireless device appended, for example, to each wrist of the operator. The wireless device utilizes an optical reflectance matrix detecting key closures, an inertial motion sensor encoding pointing device commands and sound acquisition circuit, all of whose signals are optionally multiplexed and broadcast at, for example, radio frequency or wireless to a local base station connected to the computer-based host system. The optical reflectance matrix includes a plurality of infrared LED'"'"'s or other light sources sequentially illuminating columns in a plane below the palm of the user'"'"'s hand, from which reflections from the user'"'"'s fingers are detected along rows with a plurality of sensors such as phototransistors. The optical plane is penetrated by the user'"'"'s extended fingers (or other obstructions), in an "air typing" action. Hand motion is sensed with two dimensional accelerometers and encoded to generate a pointing device control signal. Acoustic signals are optionally acquired in an on-board audio system which also provides sound feedback transmitted from the base station for two-way voice communications and to confirm input.

Claims

1. A method of detecting at least one of a presence and absence of a human digit or prosthetic appendage of a wrist or other suitable appendage of a user within a specific temporal region defined in fixed relation to the wrist or other suitable appendage, comprising the steps of:

(a) defining a plane in which an array of optical sources affixed to the wrist direct narrow light beams intersectable angles to an array of substantially narrow beamwidth optical detectors positioned at in fixed relation to the optical sources producing a matrix;

(b) individually exciting each of the optical sources with a distinct waveform and respectively independently detecting by each of the optical detectors responsive thereto as detected waveforms;

(c) evaluating each of the detected waveforms for correlation with each of the distinct waveforms to determine at least one of a presence and absence of a reflection of the distinct waveforms, and producing correlation results responsive thereto;

(d) analyzing the correlation results for each of the optical detectors and the optical sources respectively, within the matrix to determine at least one valid key closure; and

(e) encoding the at least one valid key closure into at least one standard computer keyboard scancode.

2. The method of claim 1, wherein said encoding step (e) further comprises the step of encoding the at least one valid key closure into the at least one standard computer keyboard scancode comprising at least one standard computer musical instrument digital interface (MIDI) command code.

3. The method of claim 1, further comprising the steps of:

(f) detecting motion in three dimensional space of the wrist device by determining real-time acceleration vectors using an array of accelerometers in fixed position within the wrist device; and

(g) mathematically integrating the acceleration vectors producing three dimensional velocity vectors; and

(h) encoding the velocity vectors into pointing device commands comprising at least one of standard computer mouse and joystick formats.

4. The method of claim 3, wherein said encoding step (h) further comprises the step of encoding the velocity vectors into parametric control level changes.

5. The method of claim 1, further comprising the step of receiving local acoustic audio waveforms using at least one of a microphone, amplification and digital encoding circuitry.

6. The method of claim 1, further comprising the step of multiplexing the at least one standard computer keyboard scancodes into a multiplexed signal, and transmitting the multiplexed signal over a wireless link to a base station.

7. The method of claim 1, further comprising the steps of:

(f) multiplexing at least one of the standard computer keyboard scancode, a musical instrument digital interface (MIDI) command code, a parametric control command, and a digital audio waveform into a multiplexed signal;

(g) transmitting the multiplexed signal over a wireless link to a base station;

(h) receiving the multiplexed signal and demultiplexing the multiplexed signal for appropriate routing to a host system; and

(i) transmitting over another wireless link command and control codes, input confirmation, configuration and operating parameters, and optionally digital audio communications data to the wrist device.

8. The method of claim 1, wherein said defining step (a) further comprises the step of defining the plane in which the array of optical sources affixed to the wrist directs narrow light beams of less than 20 degree beam angles at the intersectable angles to the array of the substantially narrow beamwidth optical detectors.

9. The method of claim 1, further comprising the step of detecting at least one of hand motion and tilt in three dimensional space of the wrist device by determining real-time acceleration vectors using an array of acelerometers to free the fingers for typing and mouse key, alphanumeric key and function key closure while moving a cursor.

10. A method of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a specific temporal region defined in fixed relation to the wrist, comprising the steps of:

(a) illuminating a plurality of first light beams via a plurality of optical sources affixed to the wrist in a first position respectively in a first direction and remote from or not disposed on the at least one human digit;

(b) reflecting a plurality of second light beams via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, each of the plurality of second light beams having at least one component;

(c) detecting the at least one component via at least one of a plurality of optical detectors affixed to the wrist in a second position responsive thereto as at least one detected light beam and remote from or not disposed on the at least one human digit;

(d) evaluating the at least one detected light beam to determine at least one of a presence and absence of a reflection of the plurality of second light beams, and producing correlation results responsive thereto; and

(e) generating at least one activation signal responsive to the correlation results indicative of the at least one of the presence and absence of the at least one human digit on the wrist of the user within the specific temporal region.

11. The method of claim 10, further comprising the step of arranging at least one of the plurality of optical sources and the plurality of optical detectors in an array geometry in a form of a hand, the array geometry comprising a fan shape aligning with carpal bones in the hand.

12. The method of claim 10, wherein the plurality of optical sources and the plurality of optical detectors comprise narrow beamwidth optical emitters and detectors, respectively.

13. The method of claim 10, further comprising the step of detecting at least one of hand motion and tilt in three dimensional space of the wrist device by determining real-time acceleration vectors using an array of accelerometers to free the fingers for typing and mouse key, alphanumeric key or function key closure while moving a cursor.

14. A method of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a temporal region defined with respect to the wrist, comprising the steps of:

(a) illuminating a plurality of first light beams in a first direction from a first position remote from or not disposed on the at least one human digit;

(b) reflecting a plurality of second light beams via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, each of the plurality of second light beams having at least one component;

(c) detecting the at least one component of the at least one intersection responsive thereto as at least one detected light beam from a second position remote from or not disposed on the at least one human digit;

(d) evaluating the at least one detected light beam to determine at least one of a presence and absence of a reflection of the plurality of second light beams, and producing correlation results responsive thereto; and

(e) generating at least one activation signal responsive to the correlation results indicative of the at least one of the presence and absence of the at least one human digit within the temporal region.

15. The method of claim 14, further comprising the step of arranging at least one of the plurality of optical sources and the plurality of optical detectors in an array geometry in a form of a hand, the array geometry comprising a fan shape aligning with carpal bones in the hand.

16. The method of claim 14, wherein the plurality of optical sources and the plurality of optical detectors comprise narrow beamwidth optical emitters and detectors, respectively.

17. The method of claim 14, further comprising the step of detecting at least one of hand motion and tilt in three dimensional space of the wrist device by determining real-time acceleration vectors using an array of accelerometers to free the fingers for typing and mouse key, alphanumeric key or function key closure while moving a cursor.

18. A system of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a temporal region defined with respect to the wrist, comprising:

a wrist assembly removably connectable to the wrist of the user;

a plurality of optical sources disposed on a first location of said wrist assembly, and illuminating first light beams in a first direction, the first direction capable of intersecting with the at least one human digit, thereby reflecting at least one second light beam via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, the at least one second light beam having at least one component;

a plurality of optical detectors disposed on a second location of said wrist assembly, and detecting the at least one component of the at least one intersection responsive thereto as at least one detected light beam; and

a data processor adapted to be connected to said plurality of optical detectors, receiving and evaluating the at least one detected light beam to determine at least one of the presence and absence of a reflection of the at least one second light beam, and generating at least one activation signal responsive thereto indicative of the at least one of the presence and absence of the at least one human digit within the temporal region.

19. A system according to claim 18, wherein said data processor is connected to said plurality of optical detectors via at least one of hard wiring and wireless connection.

20. A system according to claim 18, wherein the at least one activation signal is used to control input of data into a computer system.

21. A system according to claim 18, wherein the at least one activation signal is used to control input of data for at least one of a character keyboard for input of characters, a joystick for input of joystick actions and a musical keyboard for input of musical actions.

22. A system of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a predetermined region defined with respect to the wrist, comprising:

a wrist assembly removably connectable to the wrist of the user;

a plurality of optical sources disposed on said wrist assembly, and illuminating first light beams in a first direction, the first direction capable of intersecting with the at least one human digit, thereby reflecting at least one second light beam via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, the at least one second light beam having at least one component;

a plurality of optical detectors disposed on said wrist assembly, and detecting the at least one component of the at least one intersection responsive thereto as at least one detected light beam; and

a data processor adapted to be connected to said plurality of optical detectors, receiving and evaluating the at least one detected light beam to determine at least one of a presence and absence of a reflection of the at least one second light beam, and generating at least one activation signal responsive thereto indicative of the at least one of the presence and absence of the at least one human digit within the predetermined region indicative of at least one predetermined input.

23. A system according to claim 22, further comprising another system removably connectable to another wrist of the user having at least another human digit,

wherein said system is used to access the at least one predetermined input accessible to the at least one human digit, and said another system is used to access at least another predetermined input accessible to the at least another human digit.

24. A system according to claim 22, further comprising another system removably connectable to another wrist of the user having at least another human digit,

wherein said system is used to access the at least one predetermined input accessible to the at least one human digit, and said another system is used to access at least another predetermined input accessible to the at least another human digit,

wherein said system and said another system are communicable with each other, and

wherein at least one of said system is optionally accessible to the at least another predetermined input, and said another system is optionally accessible to the at least one predetermined input, via a shift operation.

25. A method of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a specific temporal region defined in fixed relation to the wrist, comprising the steps of:

(a) illuminating a plurality of first light beams via a plurality of optical sources affixed to the wrist in a first position respectively in a first direction;

(b) reflecting a plurality of second light beams via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, each of the plurality of second light beams having at least one component;

(c) detecting the at least one component via at least one of a plurality of optical detectors affixed to the wrist in a second position responsive thereto as at least one detected light beam;

(d) evaluating the at least one detected light beam to determine at least one of a presence and absence of a reflection of the plurality of second light beams, and producing correlation results responsive thereto; and

(e) generating at least one activation signal responsive to the correlation results indicative of the at least one of the presence and absence of the at least one human digit on the wrist of the user within the specific temporal region,

wherein the plurality of optical sources and the plurality of optical detectors are at least one of retractable and collapsible.

26. A method of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a specific temporal region defined in fixed relation to the wrist, comprising the steps of:

(a) illuminating a plurality of first light beams via a plurality of optical sources affixed to the wrist in a first position respectively in a first direction;

(b) reflecting a plurality of second light beams via at least one intersection between at least one of the first light beams and the at least one human digit in a second direction, each of the plurality of second light beams having at least one component;

(c) detecting the at least one component via at least one of a plurality of optical detectors affixed to the wrist in a second position responsive thereto as at least one detected light beam;

(d) evaluating the at least one detected light beam to determine at least one of a presence and absence of a reflection of the plurality of second light beams, and producing correlation results responsive thereto; and

(e) generating at least one activation signal responsive to the correlation results indicative of the at least one of the presence and absence of the at least one human digit on the wrist of the user within the specific temporal region,

wherein the plurality of optical sources and the plurality of optical detectors are at least one of retractable and collapsible.

27. A system of detecting at least one of a presence and absence of at least one human digit on a wrist of a user within a predetermined region defined with respect to the wrist, comprising a wrist assembly having at least one emitter and at least one detector utilizing optical beam reflectance via reflectance of an emitted optical beam from the at least one emitter by the at least one human digit, wherein the at least one emitter and detector are remote from or not disposed on the at least one human digit and removably connectable to the wrist of the user, thereby defining a keyboard and providing the user with a logical keyboard portable with respect to the user and substantially fixed to said wrist via said wrist assembly.