Method and Apparatus to Provide Haptic and Visual Feedback Of Skier Foot Motion and Forces Transmitted to the Ski Boot

• US 20160375346A1
• Filed: 06/23/2015
• Published: 12/29/2016
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Motion Metrics Ltd.
• Original Assignee: IPCOMM LLC
• Inventors: Czaja, Stanislaw, Afsar, Muhammad, Bachleda-Curus, Andrzej
• Granted Patent: US 9,968,840 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A43B 17/00 Insoles for insertion, e.g. footbeds or inlays, for attachment to the shoe after the upper has been joined
• CPC Class Codes: A43B 3/34 with electrical or electronic arrangements
• CPC Class Codes: A43B 5/04 Ski or like boots
• CPC Class Codes: A43B 5/0405 Linings, paddings or insertions; Inner boots
• CPC Class Codes: A43B 5/16 Skating boots
• CPC Class Codes: A43B 5/1616 Inner boots
• CPC Class Codes: A61B 2503/10 Athletes
• CPC Class Codes: A61B 2505/09 Rehabilitation or training
• CPC Class Codes: A61B 2560/0223 of calibration, e.g. protocols for calibrating sensors
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0247 Pressure sensors
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/1036 Measuring load distribution, e.g. podologic studies
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/1124 Determining motor skills
• CPC Class Codes: A61B 5/6807 Footwear
• CPC Class Codes: A63B 2071/0636 3D visualisation
• CPC Class Codes: A63B 2071/0655 Tactile feedback
• CPC Class Codes: A63B 2220/12 : Absolute positions, e.g. by using GPS
• CPC Class Codes: A63B 2220/16 : Angular positions
• CPC Class Codes: A63B 2220/36 : Speed measurement by electric or magnetic parameters
• CPC Class Codes: A63B 2220/40 : Acceleration
• CPC Class Codes: A63B 2220/44 : Angular acceleration
• CPC Class Codes: A63B 2220/56 : Pressure
• CPC Class Codes: A63B 2220/74 : Atmospheric pressure
• CPC Class Codes: A63B 2225/02 : Testing, calibrating or measuring of equipment
• CPC Class Codes: A63B 2225/20 : with means for remote communication, e.g. internet or the like
• CPC Class Codes: A63B 2225/50 : Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2225/54 : Transponders, e.g. RFID
• CPC Class Codes: A63B 69/18 : for skiing
• CPC Class Codes: A63C 11/003 : Signalling devices, e.g. acoustical or visual A63C11/225 takes precedence; means for indicating the location of accidentally snow-buried persons A63B29/021
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 : Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16Z 99/00 : Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04M 1/72412 : using two-way short-range wireless interfaces

Abstract

A system and method to provide haptic and visual feedback of motion of the skier foot and the forces transmitted to the skiboot insole and consequently to the ski/snow interface. This system comprises a motion and force sensors and a haptic actuator embedded in the skiboot insole in communication with a smart-phone based analysis application, configured to calculate insole motion and orientation and the distribution of pressure points inside the skiboot, then to provide haptic feedback to the skier foot instructing on timing and direction of change in the pressure points necessary to achieve optimal turn parameters. Furthermore, analyzed together with GPS coordinates are transmitted by the application to the cloud based server for further processing. Graphical and numerical representation of data is superimposed over a 3D map of a slope which may be viewed on the user smart-phone or on a remote computer terminal.

Claims

1. A system providing haptic and visual feedback of motion and forces transmitted to a ski by feet of a selected user comprising:

a motion and force processing elements embedded in a skiboot insoles of the selected user,

a haptic feedback actuators embedded in the skiboot insoles of the selected user; and

a smart-phone based analysis application in communication with the skiboot insoles using wireless radio interface,

and wherein the motion and force processing elements are configured to estimate 3D motion, and a location and value of force applied by feet of the selected user to the skiboot insoles, and wherein the smart-phone based analysis application is configured to analyze distribution, location and value of force in relation to motion of feet of the selected user and to;

provide haptic feedback to feet of the selected user instructing on timing,

location and value of force the selected user must apply to the skiboot insoles during turn;

obtain location GPS coordinates and transmit said location GPS coordinates together with motion and force data to a remote server for processing using a smart-phone cellular radio interface; and

to display data processed by the remote server on the smart-phone based analysis application or on a dedicated remote computer terminal.

2. The system of claim 1, wherein motion and force processing element comprises:

a three-axis accelerometer;

a three-axis gyroscope;

a three-axis magnetometer;

an atmospheric pressure sensor; and

a multiplicity of force pressure sensors;

and wherein said motion and force processing element is configured to measure motion of a selected user feet and a changes in distribution and location of force feet the selected user apply to skiboot insoles; and

communicate said change in distribution and location of force for analysis to a smart-phone based application.

3. The system of claim 1, wherein smart-phone based application is configured to retrieve and process 3D motion and force data from skiboot insoles and based on said processing, estimates phase of a ski turn, and wherein upon obtaining estimate of phase of the ski turn, communicates timing of change in distribution and location of force required for successful termination of the ski turn, to a user feet by applying control signal to a haptic actuator embedded in the skiboot insoles.

4. The system of claim 1, wherein motion and force distribution applied by a selected user feet to skiboot insoles, together with location GPS coordinates are processed by a remote server to provide a visual and numerical representation of each turn, and wherein said visual and numerical representation is overlaid on a 3D map obtained by using location GPS coordinates, and wherein after processing, said visual and numerical data and associated 3D map is stored in the remote server.

5. The system of claim 4, wherein visual and numerical data overlaid on 3D map can be retrieved by selected user from remote server for visualization and analysis of a ski run.

6. The system of claim 4, wherein visual and numerical data overlaid on 3D map can be retrieved from remote server by an instructor and used as reference to provide haptic instructions to feet of a selected user by controlling haptic actuators embedded in skiboot insoles of the selected user.

7. The system of claim 4, wherein timing change in distribution and location of force applied to skiboot insoles are collected independently for left and right turn and sorted into a velocity and a radius bins.

8. The system of claim 7, wherein results from velocity and radius bins for left turn are compared to the respective bins for right turn and difference in results is displayed as difference in symmetry of force distribution.

9. The system of claim 3, wherein change timing in distribution and location of force required for a successful ski turn, is derived from parameters obtained during calibration procedures and selected user feet motion vectors.

10. A method providing haptic and visual feedback of motion and forces transmitted to a ski by feet of a selected user comprised of:

obtaining system global and local coordinates;

obtaining a 3D motion vectors of feet of the selected user;

obtaining distribution, location and value of force applied to a skiboot insoles;

processing the distribution and location of force applied to the skiboot insoles in relation to motion vectors;

providing instructions about change in distribution and location of force required for successful termination of a turn by providing control signals to a haptic actuator embedded in the skiboot insoles;

obtaining location GPS coordinates;

transmitting motion, forces and location GPS coordinates data to a remote server using smart-phone cellular radio interface;

receiving a control instructions from a remote terminal and applying said control instructions to haptic actuators embedded in the skiboot insoles; and

providing visual and numerical results of motion and timing of change in the distribution and location of force applied by feet of the selected user to the skiboot insoles,

and wherein the system global and local coordinates are obtained by analyzing data from a magnetometer and an accelerometer; and

wherein the 3D motion vectors of feet of the selected user are obtained by analyzing data obtained from;

a 3-axis accelerometer sensor, a 3-axis gyroscope sensor, a 3-axis magnetometer sensor and an atmospheric pressure sensor; and

wherein the timing of change and the distribution and location of forces applied to the skiboot insoles are obtained by analyzing data from a multiplicity of force sensors embedded in the skiboot insoles.

11. The method of claim 10, wherein numerical and visual results of motion and distribution and location of force transmitted to skiboot insoles comprises:

a sample time;

a motion and rotational vectors and their angles;

an effective radius of the turn;

a distribution of force between inner and outer foot;

location of force inside the skiboot insoles; and

a difference in symmetry of force distribution between left and right turns, and wherein said numerical and visual results are overlaid on a skiing area 3D map obtained from GPS coordinates and wherein said numerical and visual results are displayed on a selected user smart-phone or on a dedicated remote terminal.

12. The method of claim 10, wherein timing change in the distribution and location of force required for a successful turn is obtained by analyzing 3D motion vectors in relation to calibration parameters obtained during calibration procedures.

13. The method of claim 10, wherein instructions related to the location of force necessary for successful termination of ski turn is obtained by analyzing orientation of feet of a selected user, ski velocity and trajectory, equipment parameters and physical parameters of the selected user.

14. The method of claim 13, wherein instructions related to timing of change in distribution and location of force to be applied to skiboot insoles by a selected user feet is provided in form of haptic vibrations to a toes of the selected user feet.

15. The method of claim 14, wherein instructions are provided in a form of varying frequencies and/or vibration amplitudes.

16. The method of claim 12, wherein calibration procedure comprises:

obtaining global and local coordinates from motion processing elements;

obtaining selected user weight and skiing proficiency level information;

obtaining equipment technical parameters information;

obtaining snow and ski slope condition information; and

performing measurements of distribution and location of force applied by feet of a selected user to skiboot insoles in standing position when the selected user is not in motion and the selected user;

rests in a natural position with body mass equally distributed between both feet;

leans on an inner edges of skis as in left and right turn; and

performing measurements of distribution and location of force applied to the skiboot insoles during a test run and storing results as a reference parameters.

17. The method of claim 16, wherein reference parameters obtained during the test run comprises:

ski velocity in relation to global and local coordinates;

feet motion in relation to global coordinates; and

distribution and location of force applied to skiboot insoles in relation to global and local coordinates.

18. The method of claim 17, wherein timing of change in distribution and location of force applied to skiboot insoles is collected independently for left turn and right turn and sorted into velocity and turn radius bins.

19. A non-transitory computer accessible memory medium for storing program instructions pertaining to a system providing haptic and visual feedback of motion and forces transmitted by feet of a selected user to a skiboot insoles, wherein the program instructions execute all of the following:

maintaining communication with a motion and force processing elements embedded in the skiboot insoles and with a remote server;

obtaining system'"'"'s global and local coordinates by analyzing data obtained from an accelerometer and magnetometer sensors;

obtaining 3D motion of feet of the selected user by analyzing data obtained from an accelerometer sensor, gyroscope sensor, magnetometer sensor and an atmospheric pressure sensor;

obtaining distribution and location of force applied by feet of the selected user to the skiboot insoles and processing said distribution and location of forces in relation to a 3D motion vectors of the skiboot insoles to derive timing of change in distribution and location of force required to perform successful turn;

providing instructions about timing of change in distribution and location of force required for successful termination of turn by applying control signals to a haptic actuators embedded in skiboot insoles;

obtaining location GPS coordinates;

transmitting motion, forces and location GPS coordinate data to remote server using smart-phone cellular radio interface;

receiving instruction required for successful termination of turn from a remote terminal and applying said instructions as control signals to the haptic actuators embedded in skiboot insoles;

performing calibration procedure to obtain reference values of force during static and dynamic test conditions; and

providing visual and numerical results of motion and timing of change in the distribution and location of force applied by feet of the selected user to the skiboot insoles.

20. The non-transitory computer accessible memory medium of claim 19, wherein results obtained during calibration, dynamic test or a regular runs are collected independently for a left turn and a right turn and sorted into a velocity and a radius bins, then said velocity and radius bins are compared and difference is displayed as difference in symmetry of force distribution.