Method and apparatus for producing output indicative of the content of speech or mouthed speech from movement of speech articulators

• US 10,283,120 B2
• Filed: 09/15/2015
• Issued: 05/07/2019
• Est. Priority Date: 09/16/2014
• Status: Active Grant

• Current Assignee: University of HULL
• Original Assignee: University of HULL
• Inventors: Gilbert, James Michael, Cheah, Lam Aun, Bai, Jie
• Primary Examiner(s): Shin, Seong-Ah A
• Application Number:
  US15/511,923
• Publication Number:
  US 20170249942A1
• Time in Patent Office: 1,330 Days
• Field of Search: 704261, 704243, 704255, 704275
• CPC Class Codes: A61B 2562/0223 Magnetic field sensors
• CPC Class Codes: A61B 5/05 Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves impedance plethysmography A61B5/0295; measuring movement of the entire body or parts thereof A61B5/11; detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof A61B5/24
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/682 Mouth, e.g., oral cavity; tongue; Lips; Teeth
• CPC Class Codes: A61B 5/741 using synthesised speech
• CPC Class Codes: G01R 33/0206 Three-component magnetometers
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis

Abstract

A method of producing output indicative of the content of speech or mouthed speech from movement of speech articulators is described. The method may including fixing a plurality of magnets respectively to a plurality of speech articulators of a human individual. Providing a support. Providing a plurality of signal magnetic field sensors. Providing at least three reference magnetic field sensors orientated differently from one another with respect to the Earth'"'"'s magnetic field. The signal and reference magnetic field sensors being fixed to the support which holds the sensors in fixed spatial relationships to one another. Producing, over a period of time, a respective signal from each signal magnetic field sensor and a respective signal from each reference magnetic field sensor. Obtaining, over the period of time, for each said signal magnetic field sensor signal, a respective correction value.

Claims

1. A method of producing output indicative of the content of speech or mouthed speech from movement of speech articulators, comprising:

fixing a plurality of magnets respectively to a plurality of speech articulators of a human individual;

providing a support;

providing a plurality of signal magnetic field sensors, each said signal magnetic field sensor being located so as to be able to sense a respective magnetic field intensity comprising a component produced by at least one of the magnets and a component produced by the Earth'"'"'s magnetic field;

providing at least three reference magnetic field sensors orientated differently from one another with respect to the Earth'"'"'s magnetic field, each said reference magnetic field sensor being located so as to be able to sense a respective magnetic field intensity produced at least partially by the Earth'"'"'s magnetic field;

the signal and reference magnetic field sensors being fixed to the support which holds the sensors in fixed spatial relationships to one another;

producing, over a period of time, a respective signal from each signal magnetic field sensor and a respective signal from each reference magnetic field sensor, each signal being indicative of a respective magnetic field intensity sensed by the corresponding magnetic field sensor, wherein over said period of time the signals from the signal and reference magnetic field sensors change in response to movement of the support relative to the Earth'"'"'s magnetic field, and the signals from the signal magnetic field sensors, and optionally also the signals from the reference magnetic field sensors, change in response to movement of the articulators; and

obtaining, over said period of time, for each said signal magnetic field sensor signal, a respective correction value, each correction value being indicative of the component that is produced by the Earth'"'"'s magnetic field in the magnetic field intensity sensed by the corresponding signal magnetic field sensor and each correction value changing over said time period as the component that is produced by the Earth'"'"'s magnetic field in the magnetic field intensity sensed by the corresponding signal magnetic field sensor changes as the support moves;

wherein each correction value is calculated using the signals from the at least three reference magnetic field sensors, and also using a respective spatial transformation, each spatial transformation relating the orientations of each of said at least three reference magnetic field sensors to the orientation of the corresponding signal magnetic field sensor.

2. The method as claimed in claim 1, and further including estimating said spatial transformations.

3. The method as claimed in claim 2, wherein each said spatial transformation is estimated using a least squares method using the signals from the at least three reference magnetic field sensors and the signal from the signal magnetic field sensor to which the transformation relates, said signals which are used for said estimation being obtained as the support is moved such that the changes in magnetic field intensities sensed by the magnetic field sensors are attributable substantially only to movement of the magnetic field sensors relative to the Earth'"'"'s magnetic field.

4. The method according to claim 1, further including using the correction values to correct the signals from the signal magnetic field sensors produced over said period of time to give corrected output signals in which variation is predominantly caused by movement of the articulators.

5. The method according to claim 4, wherein the corrected output signals are used to identify the content of speech or mouthed speech made by the individual during said period of time.

6. The method according to claim 5, further including using said identified content of speech or mouthed speech to generate artificial speech corresponding to the speech or mouthed speech made by the individual during the period of time.

7. The method according to claim 1, wherein the support is worn on the head of the human individual over said period of time, the support moving with the head of the human individual when the support is so worn.

8. The method according to claim 1, wherein, over said period of time, the distance from each reference magnetic field sensor to the one of the magnets which is closest to the reference magnetic field sensor is greater than the respective distance from each signal magnetic field sensor to the respective one of the magnets which is closest to said each signal magnetic field sensor, whereby the respective magnetic field intensity sensed by each reference magnetic field sensor either includes substantially no component produced by the magnets or includes a component produced by the magnets which is lower in magnitude than each respective magnetic field component produced by the magnets and sensed by each of the signal magnetic field sensors.

9. The method according to claim 1, wherein there are three reference magnetic field sensors and wherein the three reference magnetic field sensors sense magnetic field intensity in three mutually orthogonal axes, respectively.

10. The method according to claim 9, wherein the three reference magnetic field sensors are comprised within a tri-axial reference sensor unit.

11. The method according to claim 1, wherein, over said period of time, each said reference magnetic field sensor is sufficiently far from the magnets so that the magnetic field intensity sensed by the reference magnetic field sensor includes substantially no component from the magnets.

12. The system for performing the method of claim 1, comprising:

a plurality of magnets fixable respectively to a plurality of speech articulators of a human individual;

a plurality of signal magnetic field sensors for sensing respective magnetic field intensities;

at least three reference magnetic field sensors for sensing respective magnetic field intensities;

a support to which the signal and reference magnetic field sensors are fixed and which holds the sensors in fixed spatial relationships to one another.

13. The system according to claim 12, wherein the support is adapted to be worn on a head of a human.

14. The system according to claim 12, wherein there are three reference magnetic field sensors and wherein the three reference magnetic field sensors are comprised within a tri-axial reference sensor unit.

15. A system for performing the method of claim 3, comprising:

a plurality of magnets fixable respectively to a plurality of speech articulators of a human individual;

a plurality of signal magnetic field sensors for sensing respective magnetic field intensities;

at least three reference magnetic field sensors for sensing respective magnetic field intensities;

a support to which the signal and reference magnetic field sensors are fixed and which holds the sensors in fixed spatial relationships to one another.

16. The system according to claim 15, wherein the support is adapted to be worn on a head of a human.

17. The system according to claim 15, wherein there are three reference magnetic field sensors and wherein the three reference magnetic field sensors are comprised within a tri-axial reference sensor unit.

18. A system for performing the method of claim 4, comprising:

a plurality of magnets fixable respectively to a plurality of speech articulators of a human individual;

a plurality of signal magnetic field sensors for sensing respective magnetic field intensities;

at least three reference magnetic field sensors for sensing respective magnetic field intensities;

a support to which the signal and reference magnetic field sensors are fixed and which holds the sensors in fixed spatial relationships to one another.