Method of recognizing spoken language with recognition of language color

• US 7,280,964 B2
• Filed: 12/31/2002
• Issued: 10/09/2007
• Est. Priority Date: 04/21/2000
• Status: Expired due to Term

• Current Assignee: Lessac Technologies, Inc.
• Original Assignee: Lessac Technologies, Inc.
• Inventors: Wilson, H. Donald, Marple, Gary, Handal, Anthony H., Lessac, Michael
• Primary Examiner(s): Opsasnick; Michael N
• Application Number:
  US10/335,226
• Publication Number:
  US 20030229497A1
• Time in Patent Office: 1,743 Days
• Field of Search: 704/251, 704/276
• US Class Current: 704/251
• CPC Class Codes: G09B 19/04 Speaking with audible presentation of the material to be studied G09B5/04
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/0638 Interactive procedures

Abstract

In accordance with a present invention speech recognition is disclosed. It uses a microphone to receive audible sounds input by a user into a first computing device having a program with a database consisting of (i) digital representations of known audible sounds and associated alphanumeric representations of the known audible sounds and (ii) digital representations of known audible sounds corresponding to mispronunciations resulting from known classes of mispronounced words and phrases. The method is performed by receiving the audible sounds in the form of the electrical output of the microphone. A particular audible sound to be recognized is converted into a digital representation of the audible sound. The digital representation of the particular audible sound is then compared to the digital representations of the known audible sounds to determine which of those known audible sounds is most likely to be the particular audible sound being compared to the sounds in the database. A speech recognition output consisting of the alphanumeric representation associated with the audible sound most likely to be the particular audible sound is then produced. An error indication is then received from the user indicating that there is an error in recognition. The user also indicates the proper alphanumeric representation of the particular audible sound. This allows assistant to determine whether the error is a result of a known type or instance of mispronunciation. In response to a determination of error corresponding to a known type or instance of mispronunciation, the system presents an interactive training program from the computer to the user to enable the user to correct such mispronunciation.

Claims

1. A method of recognizing spoken language, comprising:

(a) collecting a plurality of spoken language samples for a word or phrase, each of said plurality of samples being associated with a language color, optionally a regional pronunciation, a mispronunciation, an emotion or Lessac energy;

(b) processing said spoken language samples into a database suitable for input into a speech recognition system, said database comprising a plurality of graphemes, each of said graphemes comprising a written representation of its associated spoken language sample or phoneme, said database comprising a color associated with at least one of said graphemes;

(c) receiving speech to be recognized and processing said speech to be recognized for input into a speech recognition system;

(d) inputting a first portion of said processed speech to be recognized into a speech recognition system, said speech recognition system selecting among processed spoken language samples of different color but corresponding to the same graphemes to obtain a best match to a particular processed language sample and an associated recognized grapheme;

(e) outputting said recognized graphemes from the speech recognition system;

(f) inputting a second portion of said processed speech into the speech recognition system, said speech recognition system, at least initially, limiting its selection among processed spoken language samples to processed spoken language samples of a single color, to efficiently obtain a best match to a particular processed language sample and an additional associated recognized grapheme; and

(g) outputting said additional recognized grapheme from the speech recognition system.

2. A method as in claim 1, wherein said spoken language samples are collected by random collection of speech samples and segregating them according to color.

3. A method as in claim 1, wherein said spoken language samples are collected by repeated generation of the same graphemes with different color using trained speakers, said trained speakers speaking the words with the desired colors.

4. A method of speech recognition using a microphone to receive audible sounds input by a user into a computing device coupled to said microphone, said computing device having a program with database information comprising (i) digital representations of known audible sounds corresponding to proper pronunciations of phonemes and associated alphanumeric representations of said known audible sounds corresponding to proper pronunciations of phonemes forming a first database and (ii) digital representations of known audible sounds corresponding to mispronunciations, forming a second database, the method comprising the steps of:

(a) generating said database information by (i) having a person, who normally speaks said known audible sounds properly, speak said properly pronounced known audible sounds, and digitizing said properly pronounced known audible sounds spoken by said person who properly speaks said known audible sounds to form a first database of digital representation of said properly pronounced known audible sounds and (ii) having a person who usually speaks said known audible sounds corresponding to mispronunciations and digitizing said known mispronounced audible sounds spoken by said person who usually speaks said known mispronounced audible sounds corresponding to mispronunciations to form a second database;

(b) receiving said audible sounds in the form of an electrical output of said microphone receiving speech to be recognized;

converting said electrical output corresponding to a particular audible sound into a digital representation of said particular audible sound to be recognized;

(d) comparing said digital representation of said particular audible sound to be recognized to said digital representations of said known audible sounds in said first and second databases to determine a match with the one of said known audible sounds most likely to be the particular audible sound to be recognized being compared to the sounds in said database; and

(e) outputting as a speech recognition output the alphanumeric representations associated with said audible sound most likely to be said particular audible sound.

5. A method as in claim 4, further comprising:

(f) outputting an error indication in response to a match with a known audible sound corresponding to a known mispronunciation; and

(g) in response to a determination of error corresponding to a known mispronunciation, presenting an interactive training program from said computing device to said user to enable said user to correct such mispronunciation.

6. A method as in claim 4 further comprising:

(e) outputting an error indication in response to a match with a known audible sound corresponding to a known mispronunciation; and

(f) in response to the detection of repeated instances or a reliable single instance of pronunciation error, presenting an interactive training program from said computer to said user to enable said user to correct such mispronunciation.

7. A method as in claim 4, further comprising:

(e) outputting an error indication in response to a match with a known audible sound corresponding to a known mispronunciation; and

(f) in response to a determination of error corresponding to a known mispronunciation, presenting an interactive training program from said computing device to said user to enable said user to correct such mispronunciation using Lessac System techniques.

8. A method of speech recognition as in claim 7, wherein said presenting an interactive training program from said computer to said user to enable said user to correct such mispronunciation is optional and is performed when elected by the user.

9. A method of speech recognition as in claim 7, wherein said user is presented with an interactive training program in response to the detection of repeated instances or a reliable single instance of pronunciation error.

10. A method of speech recognition as in claim 7, wherein said database information comprising (i) digital representations of known audible sounds corresponding to proper pronunciations of phonemes and associated alphanumeric representations of said known audible sounds corresponding to proper pronunciations of phonemes and (ii) digital representations of known audible sounds corresponding to mispronunciations is formed by (i) having a person, who normally speaks said known audible sounds properly, speak said known audible sounds, and digitizing said known audible sounds spoken by said person who properly speaks said known audible sounds;

and (ii) having a person who usually speaks said known audible sounds corresponding to mispronunciations and digitizing said known audible sounds spoken by said person who usually speaks said known audible sounds corresponding to mispronunciations.