METHOD AND MOBILE DEVICE FOR AWARENESS OF LANGUAGE ABILITY

• US 20120053929A1
• Filed: 12/29/2010
• Published: 03/01/2012
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chiu, Yu-Hsien, HSIA, CHI-CHUN, Chuang, Wei-Che, Li, Kuo-Yuan
• Granted Patent: US 8,712,760 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

A method and mobile device for awareness of language ability are provided. “Repeated pattern index”-related properties, such as, a vocabulary usage amount, a vocabulary type, or a ratio, a time point, a time length or repeated contents of a repeated voice segment, and “community interaction index”-related properties, such as, a number of persons who speak with a user, a conversation time length, or whether the user talks alone during each time interval, are extracted according to voice data collected by a voice collection element worn on the user. Then, a language ability of the user is further calculated, so as to provide evaluation of the language ability of a dementia patient for reference.

Claims

1. A mobile method for awareness of language ability, comprising:

an audio processing step, wherein after a voice is received by a voice collection element, a voice activity detection module in a language ability evaluation unit extracts a voice segment with speech sound from the voice, and a feature extraction module in the language ability evaluation unit calculates a feature vector sequence of the voice segment, that is, extracts a voice segment feature vector for analysis;

a repeated pattern index estimating step, wherein a steady state voice segment detection and quantization module in the language ability evaluation unit directly obtains a codeword sequence, a repeated voice segment detection module in the language ability evaluation unit performs a repeated voice segment matching algorithm, so as to determine whether the codeword sequence contains one or at least one repeated voice segment, and not only a full-domain language model is established based on codewords of common daily expressions, but also a catching language model is established based on codewords that occur recently, which are used in repeated voice segment matching, so as to obtain a repeated pattern index; and

a community interaction index estimating step, wherein a speaker diarization module in the language ability evaluation unit detects a speaking time/times ratio of speakers, a conversation time length, and a speaker alternation times, and even detects whether a phenomenon of soliloquy exists, so as to obtain a community interaction index.

2. The mobile method for awareness of language ability according to claim 1, wherein a diarization method of the speaker diarization module comprises at least one of the following methods:

speaker grouping, speaker recognition and speaker identification, and a vowel triangle-based method.

3. The mobile method for awareness of language ability according to claim 2, wherein in the vowel triangle-based method, before speaker clustering is performed, it is necessary to find a feature value of the speaker voice in a time domain or a frequency domain, quantization and modeling are performed by using a probability model such as a Gaussian mixture model (GMM) according to the feature value, and then diarization is performed.

4. The mobile method for awareness of language ability according to claim 1, wherein the feature value is obtained by adopting formants estimation or a Mel-Frequency Cesptrum Coefficient.

5. The mobile method for awareness of language ability according to claim 1, wherein steps of the repeated voice segment matching algorithm comprises:

codeword encoding for homogeneous voice segments and a codeword language model, and in the step of codeword encoding for homogeneous voice segments, voice segment division and codeword encoding are directly performed for several homogeneous voice segments on a time axis, a state number of a Semi-Hidden Markov Model (Semi-HMM) is set as 1, length features of the homogeneous voice segments are described by using a duration model, and properties of the homogeneous voice segment length are maintained through the duration model.

6. A mobile device for awareness of language ability, comprising:

an analysis platform;

a voice collection element, electrically connected to the analysis platform, for collecting required voice data; and

a language ability evaluation unit, embedded in the analysis platform, or electrically connected to the analysis platform,

wherein the language ability evaluation unit receives the voice data collected by the voice collection element, evaluates and calculates a language ability, and outputs an evaluation result.

7. The mobile device for awareness of language ability according to claim 6, wherein the evaluation result comprises a repeated pattern index and a community interaction index.

8. The mobile device for awareness of language ability according to claim 7, wherein the repeated pattern index is evaluated and calculated according to one or more of the following properties:

a vocabulary usage amount, a vocabulary type, and a ratio, a time point, a time length and repeated contents of a repeated voice segment, and the community interaction index is evaluated and calculated according to one or more of the following properties;

a number of persons who speak with a user, a conversation time length, and whether the user talks alone during each time interval.

9. The mobile device for awareness of language ability according to claim 8, wherein the language ability evaluation unit at least comprises a feature extraction module, a repeated voice segment detection module, a speaker diarization module, and a vocabulary ability evaluation module.

10. The mobile device for awareness of language ability according to claim 9, wherein the feature extraction module receives the voice data input by the voice collection element, and estimates a speech parameter, comprising a frequency cesptrum coefficient, a line spectral pair coefficient, a pitch, a sound intensity, and a voice segment length;

the repeated voice segment detection module detects a repeated voice segment in the voice data, outputs a voice of the repeated voice segment and calculates an occurrence ratio of the repeated voice segment, an occurrence time point, a repetition time length, or literal contents of the repeated voice segment;

the speaker diarization module analyzes a speaker number, a speaking ratio of each speaker, a time length, or a speaker alternation sequence in speech data of a conversation through a speaker diarization method; and

the vocabulary ability evaluation module detects and outputs a vocabulary usage amount, a vocabulary type, or a ratio, a time point, a time length or repeated contents of a repeated voice segment through vocabulary recognition or continuous speech recognition.