Tracking torso leaning to generate inputs for computer systems

• US 10,509,464 B2
• Filed: 01/08/2018
• Issued: 12/17/2019
• Est. Priority Date: 01/08/2018
• Status: Active Grant

• Current Assignee: Finchxr Limited
• Original Assignee: Finch Technologies Ltd.
• Inventors: Erivantcev, Viktor Vladimirovich, Kulchurin, Rustam Rafikovich, Lobanov, Alexander Sergeevich, Sergeev, Iakov Evgenevich, Kartashov, Alexey Ivanovich, Goncharov, Daniil Olegovich
• Primary Examiner(s): Shah, Priyank J
• Application Number:
  US15/864,860
• Publication Number:
  US 20190212807A1
• Time in Patent Office: 708 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A system including a plurality sensor modules, each having an inertial measurement unit and attached to a portion of a user (e.g., upper arm, hand, and/or head) to measure the current orientation of the corresponding portion of the user. A computing device coupled to the sensor modules is configured to identify a type of the current pose of the user, compute a front facing direction of the torso of the user from the sensor measurements, select a torso leaning model based on the pose type, project onto a vertical plane the lengthwise directions of the arms of the user as tracked by the sensor modules attached to the upper arms of the user, and calculate a torso leaning angle from the projected lengthwise directions of the arms.

Claims

1. A system, comprising:

a plurality of sensor modules, each having an inertial measurement unit and attached to a portion of the user to generate motion data identifying an orientation of the portion of the user, wherein the plurality of sensor modules includes at least a first sensor module attached to a left upper arm of the user and a second sensor module attached to a right upper arm of the user; and

a computing device coupled to the plurality of sensor modules and configured to;

calculate a horizontal, front facing direction of a torso of the user;

identify a type of a current pose of the user based at least in part on the sensor modules;

select a torso leaning model according to the type;

project lengthwise directions of upper arms of the user in a vertical plane that contains the front facing direction or is perpendicular to the front facing direction; and

calculate a leaning angle of the torso based on the projected lengthwise directions according to the selected torso leaning model.

2. The system of claim 1, wherein the leaning angle of the torso is calculated without a sensor module configured on the torso of the user.

3. The system of claim 1, wherein the leaning angle is a forward leaning angle of the torso;

and the vertical plane contains the front facing direction of the torso.

4. The system of claim 3, wherein the forward leaning angle of the torso is based on a larger one of angles of the projected lengthwise directions of the upper arms of the user relative to a vertical direction in the vertical plane.

5. The system of claim 1, wherein the leaning angle is a backward leaning angle of the torso;

and the vertical plane contains the front facing direction of the torso.

6. The system of claim 5, wherein the backward leaning angle of the torso is based on a larger one of the angles of the projected lengthwise directions of the upper arms of the user above a predetermined threshold direction in the vertical plane.

7. The system of claim 6, wherein the predetermined threshold direction is forty five degrees above a horizontal direction in the vertical plane.

8. The system of claim 5, wherein the backward leaning angle of the torso is based on a smallest one of:

a projection of a front facing direction of a head of the user in the vertical plane above a horizontal direction; and

projections of the lengthwise directions of the upper arms of the user above a predetermined threshold direction in the vertical plane.

9. The system of claim 8, wherein the predetermined threshold direction is thirty degrees above the horizontal direction in the vertical plane.

10. The system of claim 1, wherein the leaning angle is a sideway leaning angle of the torso;

and the vertical plane is perpendicular to the front facing direction of the torso.

11. The system of claim 10, wherein the sideway leaning angle is based on a difference between the projected lengthwise directions of the upper arms of the user in the vertical plane.

12. The system of claim 11, wherein the computing device is further configured to calculate a rotation of a hand of the user along a lengthwise direction of the hand that is on one of the arms of the user that has a larger one of angles of the projected lengthwise directions of the upper arms relative to a vertical direction in the vertical plane.

13. The system of claim 12, wherein the sideway leaning angle is proportional to the difference between the angles of the projected lengthwise directions of the upper arms relative to the vertical direction in the vertical plane and is proportional to the rotation of the hand.

14. The system of claim 1, wherein the computing device is further configured to calculate, using data from sensor modules attached to hands of the user, rotations of the hands of the user along lengthwise directions of the hands respectively.

15. The system of claim 14, wherein the sideway leaning angle is based on a weighted difference between angles of the projected lengthwise directions of the upper arms of the user relative to a vertical direction in the vertical plane, weighted according to the rotations of the hands of the user.

16. The system of claim 1, wherein each of the plurality of sensor modules further includes a communication device for a communication link with the computing device.

17. The system of claim 1, wherein the inertial measurement unit includes a micro-electromechanical system (MEMS) gyroscope.

18. The system of claim 17, wherein the inertial measurement unit further includes a magnetometer and a MEMS accelerometer.

19. A method, comprising:

receiving, in a computing device, motion data from a plurality of sensor modules, each having an inertial measurement unit and attached to a portion of the user to identify an orientation of the portion of the user, wherein the plurality of sensor modules includes at least a first sensor module attached to a left upper arm of the user and a second sensor module attached to a right upper arm of the user;

calculating, by the computing device, a front facing direction of a torso of the user;

identifying, by the computing device, a type of a current pose of the user based at least in part on the sensor modules;

selecting, by the computing device, a torso leaning model according to the type;

projecting, by the computing device, lengthwise directions of upper arms of the user in a vertical plane that contains the front facing direction or is perpendicular to the front facing direction; and

calculating, by the computing device, a leaning angle of the torso based on the projected lengthwise directions according to the selected torso leaning model.

20. A non-transitory computer storage medium storing instructions which, when executed by a computing device, instructs the computing device to perform a method, the method comprising:

receiving, in the computing device, motion data from a plurality of sensor modules, each having an inertial measurement unit and attached to a portion of the user to identify an orientation of the portion of the user, wherein the plurality of sensor modules includes at least a first sensor module attached to a left upper arm of the user and a second sensor module attached to a right upper arm of the user;

calculating, by the computing device, a front facing direction of a torso of the user;

identifying, by the computing device, a type of a current pose of the user based at least in part on the sensor modules;

selecting, by the computing device, a torso leaning model according to the type;

projecting, by the computing device, lengthwise directions of upper arms of the user in a vertical plane that contains the front facing direction or is perpendicular to the front facing direction; and

calculating, by the computing device, a leaning angle of the torso based on the projected lengthwise directions according to the selected torso leaning model.