SIGNAL PROCESSING METHOD AND APPARATUS BASED ON STRUCTURED SPARSITY OF PHONOLOGICAL FEATURES

• US 20170069306A1
• Filed: 09/04/2015
• Published: 03/09/2017
• Est. Priority Date: 09/04/2015
• Status: Abandoned Application

• Current Assignee: Foundation Of The Idiap Research Institute (IDIAP)
• Original Assignee: Foundation Of The Idiap Research Institute (IDIAP)
• Inventors: Cernak, Milos, Bourlard, Herve, ASAEI, Afsaneh
• Application Number:
  US14/846,036
• Publication Number:
  US 20170069306A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis
• CPC Class Codes: G10L 19/08 Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
• CPC Class Codes: G10L 2019/0001 Codebooks
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/24 the extracted parameters being the cepstrum

Abstract

A multimodal processing method comprising the steps of:

• • A) Retrieving a data set representing distinctive phonological features;
  • B) Identifying structured sparse patterns in said data set;
  • C) Processing said structured sparse patterns.

Claims

1. A signal processing method comprising the steps of:

A) Retrieving a data set representing phonological features;

B) Identifying structured sparse patterns in said data set;

C) Processing said structured sparse patterns.

2. The method of claim 1, said signal being a speech signal, said phonological features comprising major class features, laryngeal features, manner features, and place features.

3. The method of claim 1, wherein said features are specified by binary or univalent or multi-valued quantized values to signify whether a segment is described by the feature.

4. The method of claim 1, wherein the identification of structured sparse patterns uses a codebook of structured sparse patterns.

5. The method of claim 4, wherein the identification of structured sparse patterns uses a first said codebook of structured sparse patterns at physiology level.

6. The method of claim 4, wherein the identification of structured sparse patterns uses a second said codebook of structured sparse patterns at supra-segmental level.

7. The method of claim 1, wherein retrieving said data set includes extracting said data set from any combination of at least one among a speech signal, a video signal, a brain signal, an ultrasound signal representative of the tongue and/or lip movement, an optical camera signal representative of the tongue and/or lip movement, and/or an electromyography signal representative of speech articulator muscles and of the larynx.

8. The method of claim 7, wherein said signal processing includes phonological encoding.

9. The method of claim 8, wherein said signal processing comprises a structured compressive sampling of said data sets.

10. The method of claim 7, wherein said signal processing includes event analysis.

11. The method of claim 10, wherein said event analysis includes speech parametrization (such as formants, LPC, PLP, MFCC features) or visual clue extraction (such as a shape of mouths) or brain-computer interface feature extraction (such as electroencephalogram patterns) or extraction of feature from an ultrasound, optical or electromyography signal representative of the tongue and/or lip and/or speech articulator muscles and/or larynx movement or position.

12. A multimodal signal processing method comprising the steps of:

C) Retrieving phonological features;

D) Reconstructing uncompressed phonological features;

E) Synthesising speech parameters from said reconstructed uncompressed phonological features.

13. The method of claim 12, wherein said speech parameters include speech excitation and vocal tract, cepstral parameters.

14. The method of claim 12, wherein a deep neural network is used for mapping the uncompressed phonological features to speech parameters for re-synthesis.

15. The method of claim 12, comprising a step of creating said uncompressed phonological features from a text.

16. A multimodal signal processing apparatus comprising:

an event analysis module;

a feature identification module for retrieving a data set representing phonological features;

a processing module for identifying structured sparse patterns in said data set, and for processing said structured sparse patterns.

17. The multimodal signal processing apparatus of claim 16, said processing module being a structured compressive sampling module.

18. A multimodal signal processing apparatus comprising:

a sparse recovery module for receiving a digital signal and reconstructing a data set representing phonological features;

a phonological decoder for generating speech parameters;

a speech synthesis module for receiving said speech parameters and delivering estimated digital speech samples, or for converting text to canonical binary phonological features and delivering digital speech samples.

19. The apparatus of claim 18, said phonological decoder outputting line spectra and glottal signal parameters.

20. The apparatus of claim 18, said speech synthesis module comprising a deep neural network trained for mapping phonological features to speech parameters.

21. The apparatus of claim 18, said speech synthesis module being speaker dependant.