Devices for controlling computers based on motions and positions of hands

• US 10,509,469 B2
• Filed: 04/20/2017
• Issued: 12/17/2019
• Est. Priority Date: 04/21/2016
• Status: Active Grant

• Current Assignee: Finchxr Limited
• Original Assignee: Finch Technologies Ltd.
• Inventors: Erivantcev, Viktor Vladimirovich, Kulchurin, Rustam Rafikovich, Lobanov, Alexander Sergeevich, Sergeev, Iakov Evgenevich, Gabdullina, Yana Olegovna, Kartashov, Alexey Ivanovich
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US15/492,915
• Publication Number:
  US 20170308165A1
• Time in Patent Office: 971 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

Disclosed herein is a data input device and method of operating the same. In one embodiment, a data input device comprises a plurality of inertial sensor units, one or more touch input devices, a microcontroller configured to collect sensor data from the inertial sensors and the one or more touch input devices and process the sensor data to generate processed sensor data, and a wireless transceiver configured to transmit the processed sensor data to a host computer. In another embodiment, a method comprises receiving sensor data from a handheld device; calculating hand movement characteristics in three dimensional space based on the sensor data; calculating the position and orientation of the components of the handheld device; identifying positions and movements of one or more fingers of a user manipulating the handheld device; identifying a gesture from the positions and movements of one or more fingers of a user manipulating the handheld device; identifying a recognized gesture corresponding to the identified gesture; and dispatching an event notifying the gesture to an application.

Claims

1. A data input device comprising:

a base unit adapted to be strapped to a backside of a hand of a user;

a plurality of inertial sensor units;

a plurality of rings configured to be worn on fingers of the hand of the user, wherein the plurality of inertial sensor units are respectively mounted in the plurality of rings, each of the plurality of rings includes a respective wireless transceiver to transmit sensor data to the base unit, each of the plurality of rings is communicatively coupled to the base unit via the respective wireless transceiver;

one or more touch input devices, wherein the one or more touch input devices includes a touchpad device mounted on one of the plurality of rings and adapted to receive touch inputs activated by a thumb of the hand of the user;

a microcontroller configured to collect sensor data from the inertial sensor units and the one or more touch input devices, and to process the sensor data to generate processed sensor data, wherein the base unit houses the microcontroller;

a plurality of vibration haptic actuators communicatively coupled to the microcontroller, wherein each actuator is at a respective one of the rings;

a first wireless transceiver configured to transmit the processed sensor data to a host computer, wherein the base unit further houses the first wireless transceiver; and

an arm module adapted to be secured to a shoulder or upper arm of the user, wherein the arm module houses an additional inertial sensor unit and a second wireless transceiver, wherein the arm module and base unit are independently moveable with respect to one another, and wherein the arm module is configured to record manual input data and transmit the recorded data to the base unit for further processing.

2. The data input device of claim 1 wherein the base unit further comprises a base inertial sensor unit communicatively coupled to the microcontroller.

3. The data input device of claim 1 further comprising a plurality of LEDs communicatively coupled to the microcontroller.

4. The data input device of claim 3 wherein a first set of the plurality of LEDs are mounted within the base unit and wherein a second set of the plurality of LEDs are mounted within the plurality of rings.

5. The data input device of claim 1 wherein the inertial sensor units each include an accelerometer and gyroscope.

6. The data input device of claim 5 wherein the inertial sensor units each include a magnetometer.

7. The data input device of claim 1 further comprising at least one of an ultrasound receiver or an optical marker, wherein each receiver or marker is communicatively coupled to the microcontroller.

8. The data input device of claim 1 wherein the processed sensor data comprises at least one of gyroscope vector data, acceleration vector data, quaternion rotation data, spatial coordinate data of a hand or wrist, or touch data from the one or more touch input devices.

9. A data input device comprising:

a plurality of first inertial sensor units;

a plurality of rings configured to be worn on fingers of the hand of a user, wherein a plurality of second inertial sensor units are respectively mounted in the plurality of rings;

one or more touch input devices, wherein the one or more touch input devices comprise one or more trigger devices located on a front portion of a handheld module adapted to be held in the hand of the user and further adapted to be manipulated by the fingers of the user while the user is holding the handheld module;

a microcontroller configured to collect sensor data from the first inertial sensor units, the second inertial sensor units, and the one or more touch input devices, and to process the sensor data to generate processed sensor data;

a first wireless transceiver configured to transmit the processed sensor data to a host computer;

an arm module adapted to be secured to a shoulder or upper arm of the user, wherein the arm module houses an additional inertial sensor unit and a second wireless transceiver, wherein the arm module and handheld module are independently moveable with respect to one another, and wherein the arm module is configured to record manual input data and transmit the recorded data to the handheld module for further processing;

an additional module adapted to be secured to a wrist, hand, back of a hand, or forearm, wherein the additional module comprises a set of inertial sensors; and

a plurality of vibration haptic actuators communicatively coupled to the microcontroller, wherein each actuator is at a respective one of the rings;

wherein the plurality of first inertial sensor units, the one or more touch input devices, the microcontroller and the first wireless transceiver are housed within the handheld module;

wherein each of the plurality of rings includes a respective wireless transceiver to transmit sensor data to the handheld module.

10. The data input device of claim 9 wherein the one or more touch input devices include touch input devices located on a top side of the handheld module selected from the group consisting of buttons, pointing sticks, trackballs, and touchpads.

11. The data input device of claim 9 wherein the handheld module further comprises a knob and a receptacle, wherein the knob of the handheld module is configured to be locked into a receptacle of a secondary handheld module and wherein the receptacle of the handheld module is configured to receive a knob of the secondary handheld module.

12. The data input device of claim 9 wherein the touch input devices comprise a joystick, power button, and action button arranged on a top side of the handheld module, and wherein the touch input devices are capable of being manipulated by a thumb of the user.

13. A method comprising:

receiving sensor data from a handheld device;

calculating hand movement characteristics in three-dimensional space based on the sensor data;

calculating the position and orientation of the components of the handheld device;

recording, by an arm module, manual input data, wherein the arm module is adapted to be secured to a shoulder or upper arm of a user, wherein the arm module houses an inertial sensor unit and a first wireless transceiver, and wherein the arm module and handheld device are independently moveable with respect to one another;

transmitting, by the arm module, the recorded data to the handheld device for further processing;

identifying positions and movements of fingers of the user manipulating the handheld device, wherein a plurality of rings are worn on the fingers of the user, wherein a plurality of inertial sensor units are respectively mounted in the plurality of rings, wherein each of the rings includes a second wireless transceiver to transmit sensor data, and wherein a vibration haptic actuator is located at each of the rings;

identifying a gesture from the positions and movements of the fingers of the user manipulating the handheld device;

identifying a recognized gesture corresponding to the identified gesture; and

dispatching an event notifying the gesture to an application.