Automatic speech recognition system for people with speech-affecting disabilities

• US 7,529,670 B1
• Filed: 05/16/2005
• Issued: 05/05/2009
• Est. Priority Date: 05/16/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Avaya Incorporated
• Inventors: Michaelis, Paul Roller
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): ORTIZ SANCHEZ, MICHAEL
• Application Number:
  US11/131,108
• Time in Patent Office: 1,450 Days
• Field of Search: 704/253, 704/231, 704/275, 381/54
• US Class Current: 704/253
• CPC Class Codes: A61B 5/091 Measuring volume of inspired or expired gases, e.g. to determine lung capacity
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence

Abstract

A speech recognition system is provided that, in one embodiment, includes an input 104 operable to receive voice utterances from a user, a speaker monitoring agent 132 operable to determine a pulmonary state of the user as a function of time, and a frame analyzer 120 operable to (i) determine a respective pulmonary state of the user at an approximate time when each of the voice utterances was made and (ii) process each of the voice utterance in a manner dependent upon the respective pulmonary state.

Claims

1. A speech recognition method, comprising:

(a) receiving, by an input port, a first voice utterance from a user, the first voice utterance comprising at least a first utterance component;

(b) determining, by speaker monitoring agent, a pulmonary state of the user at an approximate time when the first voice utterance was made, wherein the first and second pulmonary states indicate at least one of an amount of air in the user'"'"'s lungs, a degree of expansion of the user'"'"'s lungs, and a forcefulness with which air is exhaled;

(c) applying, by a frame analyzer, the following rules;

(C1) when the user had a first pulmonary state when the first voice utterance was made, using a first algorithm to identify a potential match for the voice utterance; and

(C2) when the user had a second pulmonary state when the first voice utterance was made, using a second different algorithm to identify a potential match for the first voice utterance.

2. The method of claim 1, wherein, when rule (C1) applies, further comprising the step:

comparing the first voice utterance against a first set of set of language components assuming that the first utterance component is of a first speech type and comparing the first utterance against a second set of language components assuming that the first utterance component is of a second speech type; and

wherein, when rule (C2) applies, further comprising the step;

comparing the first voice utterance against a first set of set of language components assuming that the first utterance component is of the first speech type without assuming that the first utterance component is of the second speech type.

3. The method of claim 2, wherein in the first pulmonary state the user'"'"'s lungs are less than half full of air and in the second pulmonary state the user'"'"'s lungs are more than half full of air.

4. The method of claim 3, wherein the first speech type is an unvoiced sound and the second speech type is a voiced sound.

5. The method of claim 4, wherein the determining step comprises the substep:

sensing at least one of a degree of chest expansion at a selected time, an amount of air in the user'"'"'s lungs at the selected time, and an air volume exhaled by the user over a selected time interval including the selected time.

6. The method of claim 4, wherein the determining step (b) comprises the substeps:

(B1) identifying the occurrence of an inhalation cycle by the user;

(B2) identifying a first timing of the first utterance by the user following the identified inhalation cycle;

(B3) determining a second timing of a second utterance by the user, the first and second utterances being spoken during exhalation of a common breath; and

(B4) determining a pulmonary state of the user at the second timing based on the time interval between the first and second timings.

7. A computer readable medium encoded with processor-executable instructions that, when executed, perform the steps of claim 1.

8. The method of claim 1, wherein the first and second algorithms provide differing levels of confidence for a potential match and wherein the first and second states indicate, at the approximate time, an amount of air remaining in the user'"'"'s lungs.

9. The method of claim 1, wherein the first and second states reflect a pulmonary state of the user and wherein step (b) comprises the substep:

(B1) detecting a pause of at least a predetermined length between temporally adjacent utterances of the user, the detected pause being assumed to be associated with a transition between temporally adjacent inhalation and exhalation cycles, wherein the approximate time is measured relative to a selected part of the pause based on the assumption that, at the selected part of the pause, the user'"'"'s lungs are at or near full capacity.

10. The method of claim 1, wherein the first and second pulmonary states indicate at least one of an amount of air in and a degree of expansion of the user'"'"'s lungs.

11. A speech recognition method, comprising:

(a) receiving, by an input, a voice utterance from a user, the voice utterance comprising at least a first utterance component, wherein the first utterance component is of a first speech type;

(b) determining, by a speaker monitoring agent, a state of the user at an approximate time when the voice utterance was made;

(c) applying, by a frame analyzer, the following rules;

(C1) when the user had a first state when the voice utterance was made, at least one of (i) determining a first level of confidence of a selected match for the voice utterance being correct and (ii) comparing the first level of confidence against a first determined confidence level to determine whether to query the user for input respecting the selected match; and

(C2) when the user had a second state when the voice utterance was made, at least one of (i) determining a second level of confidence of the selected match for the voice utterance being correct and (ii) comparing the first and/or second level of confidence against a second determined confidence level to determine whether to query the user for input respecting the selected match, wherein the first level of confidence is different from the second level of confidence and the first determined confidence level is different from the second determined confidence level;

wherein the first and second states indicate at least one of an amount of air in and a degree of expansion of the user'"'"'s lungs.

12. The method of claim 11, wherein the first level of confidence is lower than the second level of confidence and the first determined confidence level is lower than the second determined confidence level.

13. The method of claim 11, further comprising applying the following rules:

(C3) when the user had a first state when the voice utterance was made, using a first algorithm to identify a potential match for the voice utterance; and

(C4) when the user had a second state when the voice utterance was made, using a second different algorithm to identify a potential match for the voice utterance.

14. The method of claim 11, wherein when rule (C1) applies further comprising the step:

comparing the voice utterance against a first set of set of language components assuming that the first utterance component is of the first speech type and comparing the utterance against a second set of language components assuming that the first utterance component is of a second speech type; and

wherein when rule (C2) applies further comprising the step;

comparing the voice utterance against a first set of set of language components assuming that the first utterance component is of the first speech type without assuming that the first utterance component is of the second speech type.

15. The method of claim 14, wherein the first speech type is an unvoiced sound and the second speech type is a voiced sound.

16. The method of claim 11, wherein the determining step (b) comprises the substep:

sensing at least one of a degree of chest expansion at a selected time, an amount of air in the user'"'"'s lungs at the selected time, and an air volume exhaled by the user over a selected time interval including the selected time.

17. The method of claim 11, wherein the determining step (b) comprises the substeps:

(B1) identifying the occurrence of an inhalation cycle by the user;

(B2) identifying a first timing of a first utterance by the user following the identified inhalation cycle;

(B3) determining a second timing of a second utterance by the user, the first and second utterances being spoken during exhalation of a common breath; and

(B4) determining a pulmonary state of the user at the second timing based on the time interval between the first and second timings.

18. A computer readable medium encoded with processor-executable instructions that, when executed, perform the steps of claim 11.

19. A speech recognition system comprising:

(a) an input for receiving voice utterances from a user;

(b) a speaker monitoring agent for determining a pulmonary state of the user as a function of time, wherein the first and second pulmonary states indicate at least one of an amount of air in the user'"'"'s lungs, a degree of expansion of the user'"'"'s lungs, and a forcefulness with which air is exhaled; and

(c) a frame analyzer for (i) determining a respective pulmonary state of the user at an approximate time when each of the voice utterances was made and (ii) processing each voice utterance in a manner dependent upon the respective pulmonary state.

20. The speech recognition system of claim 19, wherein the frame analyzer is operable, when the user had a first state when a selected voice utterance was made, to use a first algorithm to identify a potential match for the voice utterance and, when the user had a second state when a selected voice utterance was made, to use a second different algorithm to identify a potential match for the voice utterance.

21. The speech recognition system of claim 20, wherein, when the user had a first state when a selected voice utterance was made, the frame analyzer is operable to at least one of (i) determine a first level of confidence of a selected match for the selected voice utterance being correct and (ii) compare the first level of confidence against a first determined confidence level to determine whether to query the user for input respecting the selected match and, when the user had a second state when the selected voice utterance was made, at least one of (i) determine a second level of confidence of the selected match for the selected voice utterance being correct and (ii) compare the first and/or second level of confidence against a second determined confidence level to determine whether to query the user for input respecting the selected match, wherein the first level of confidence is lower than the second level of confidence and the first determined confidence level is lower than the second determined confidence level.

22. The speech recognition system of claim 19, wherein a first voice utterance comprises at least a first utterance component, wherein the first utterance component is of a first speech type and wherein the speech analyzer is operable, when the user had a first state when the first voice utterance was made, to use a first algorithm to identify a potential match for the first voice utterance and, when the user had a second state when the first voice utterance was made, to use a second different algorithm to identify a potential match for the first voice utterance.

23. The speech recognition system of claim 22, wherein, depending on the pulmonary state, the frame analyzer operates in either a first or second mode and wherein the first mode comprises the suboperation of:

comparing the first voice utterance against a first set of set of language components assuming that the first utterance component is of the first speech type and comparing the first utterance against a second set of language components assuming that the first utterance component is of a second speech type; and

wherein the second mode comprises the suboperation of;

comparing the first voice utterance against a first set of set of language components assuming that the first utterance component is of the first speech type without assuming that the first utterance component is of the second speech type.

24. The speech recognition system of claim 23, wherein the first and second states indicate at least one of an amount of air in and a degree of expansion of the user'"'"'s lungs.

25. The speech recognition system of claim 23, wherein the first speech type is an unvoiced sound and the second speech type is a voiced sound.