Speech recognition system and method

• US 20030120486A1
• Filed: 12/19/2002
• Published: 06/26/2003
• Est. Priority Date: 12/20/2001
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett Packard Company (HP Inc.)
• Inventors: Tucker, Roger Cecil Ferry, Brittan, Paul St. John
• Application Number:
  US10/322,623
• Publication Number:
  US 20030120486A1
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A speech input stream is fed to a first speech recogniser. A confidence measure is formed for each recognition hypothesis produced in output by the first speech recogniser and this confidence measure is compared against an acceptability threshold. Where the confidence measure of a recognition hypothesis is below the threshold, the corresponding portion of the speech input is passed to a second speech recogniser and the recognition hypothesis produced is used instead of, or as a supplement to, that output by the first speech recogniser. In a preferred embodiment, the first speech recogniser is a recogniser trained to a particular user whilst the second recogniser is one associated with a particular speech application currently being accessed by the user.

Claims

1. A speech recognition method comprising the steps of:

(a) carrying out recognition of a speech input stream using a first speech recognizer to derive respective first recognition hypotheses for successive portions of the input stream;

(b) in carrying out step (a), determining a confidence measure for each first recognition hypothesis;

(c) at least in respect of those portions of the speech input stream for which the confidence measure is below an acceptability threshold, passing the speech input stream to a second speech recognizer to produce corresponding second recognition hypotheses; and

(d) forming an output recognition-hypothesis stream using recognition hypotheses from the first recognition hypotheses and only those second recognition hypotheses corresponding to the first recognition hypotheses that have a confidence measure below said threshold.

2. A method according to claim 1, wherein the output recognition-hypothesis stream comprises the first recognition hypotheses but with at least some of the hypotheses that have a confidence measure below said threshold replaced by the corresponding second hypotheses.

3. A method according to claim 1, wherein the output recognition-hypothesis stream comprises all said first recognition hypotheses and at least some of the second recognition hypotheses corresponding to the first recognition hypotheses that have a confidence measure below said threshold.

4. A method according to claim 1, wherein the first speech recognizer is local to a user and the second speech recognizer is remote from the user, step (c) involving passing speech input portions to the second speech recognizer over a communications infrastructure.

5. A method according to claim 4, wherein the second speech recognizer is part of a remote resource further including a speech application to which said output recognition-hypothesis stream is supplied, step (d) being carried out at the remote resource with at least the first recognition hypotheses that have corresponding confidence measures which reach said acceptability threshold, being passed to the remote resource.

6. A method according to claim 1, wherein the first and second speech recognizers are included in respective items of mobile personal equipment, step (c) involving passing speech input portions to the second speech recognizer over a short-range communications link.

7. A method according to claim 6, wherein the item of equipment including the first speech recognizer further includes a speech application to which said output recognition-hypothesis stream is supplied.

8. A method according to claim 1, comprising the further steps of:

(i) determining a confidence measure for each second recognition hypothesis;

(ii) at least in respect of those portions of the speech input stream for which the confidence measures of the corresponding second recognition hypotheses are below a second acceptability threshold, passing the speech input stream to a third speech recognizer to produce corresponding third recognition hypotheses;

the forming of the output recognition-hypothesis stream in step (d) using at least some of the third recognition hypotheses for which the corresponding first and second recognition hypotheses have associated confidence measures below their respective acceptability thresholds.

9. A method according to claim 1, wherein the first speech recognizer is trained to a user'"'"'s voice and the second speech recognizer is intended to recognize a specific domain or application vocabulary spoken by different users without being training to their voices.

10. A method according to claim 1, wherein in step (c) only those portions of the speech input stream for which the corresponding first recognition hypotheses have confidence measures below the acceptability threshold are passed to the second speech recognizer.

11. A method according to claim 1, wherein in step (c) only those portions of the speech input stream for which the corresponding first recognition hypotheses have confidence measures below the acceptability threshold are passed to the second speech recognizer, and confidence measures are produced for the resultant second recognition hypotheses;

step (d) involving including all the first and second recognition hypotheses in the output recognition-hypothesis stream together with confidence measures at least for the second recognition hypotheses and the corresponding first recognition hypotheses.

12. A method according to claim 1, wherein in step (c) only those portions of the speech input stream for which the corresponding first recognition hypotheses have confidence measures below the acceptability threshold are passed to the second speech recognizer, and confidence measures are produced for the resultant second recognition hypotheses;

step (d) involving replacing a first recognition hypothesis with the corresponding second recognition hypothesis only when the confidence measures associated with the two hypotheses indicate at least a degree more confidence in the second recognition hypothesis as compared to the corresponding first recognition hypothesis.

13. A method according to claim 1, wherein in step (c) all portions of the speech input stream are passed to the second speech recognizer, and in step (d) all those first recognition hypotheses that have confidence measures below said acceptability threshold are replaced by the corresponding second hypotheses in the output recognition-hypothesis stream.

14. A method according to claim 1, wherein in step (c) all portions of the speech input stream are passed to the second speech recognizer and confidence measures are produced for the second recognition hypotheses, step (d) involving replacing a first recognition hypothesis with a corresponding second recognition hypothesis in the output recognition-hypothesis stream only when the confidence measures associated with the two hypotheses indicate at least a degree more confidence in the second recognition hypothesis as compared to the corresponding first recognition hypothesis.

15. A method according to claim 1, wherein in step (c) all portions of the speech input stream are passed to the second speech recognizer and confidence measures are produced for the second recognition hypotheses, step (d) involving including in the output recognition-hypothesis stream:

all the first recognition hypotheses,

the second recognition hypotheses for which the confidence measures of the corresponding first recognition hypotheses are below their acceptability threshold, and

the confidence measures at least for the included second recognition hypotheses and the corresponding first recognition hypotheses.

16. A speech recognition system comprising:

a first speech recognizer for carrying out recognition of a speech input stream to derive respective first recognition hypotheses for successive portions of the input stream;

an acceptability-determination subsystem for deriving a confidence measure for each first recognition hypothesis and comparing this measure with an acceptability threshold to determine the acceptability of the recognition hypothesis;

a second speech recognizer for producing second recognition hypotheses for portions of the input stream;

a transfer arrangement for passing to the second speech recognizer at least those portions of the speech input stream for which the confidence measure is below said acceptability threshold; and

a control arrangement for forming an output recognition-hypothesis stream using recognition hypotheses from the first recognition hypotheses and only those second recognition hypotheses corresponding to the first recognition hypotheses that have a confidence measure below said threshold.

17. A system according to claim 16, wherein the control arrangement is operative to form the output recognition-hypothesis stream by using the first recognition hypotheses but with at least some of the hypotheses that have a confidence measure below said threshold replaced by the corresponding second hypotheses.

18. A system according to claim 16, wherein the control arrangement is operative to form the output recognition-hypothesis stream by including all said first recognition hypotheses and at least some of the second recognition hypotheses corresponding to the first recognition hypotheses that have a confidence measure below said threshold.

19. A system according to claim 16, wherein the first speech recognizer is local to a user and the second speech recognizer is remote from the user, the transfer arrangement being operative to pass speech input portions to the second speech recognizer over a communications infrastructure.

20. A system according to claim 19, further comprising a remote resource comprising said second speech recognizer and a speech application to which said output recognition-hypothesis stream is supplied, the transfer arrangement being operative to pass to the remote speech-based resource at least the first recognition hypotheses that have corresponding confidence scores which reach said acceptability threshold, and the control arrangement comprising means for forming the output recognition-hypothesis stream at the remote resource.

21. A system according to claim 16, further comprising first and second items of personal mobile equipment respectively including said first and second recognizers, the said first and second items of equipment each further including a short-range communication subsystem by which speech input portions can be passed from the first to the second item of equipment.

22. A system according to claim 21, wherein the first item of equipment further includes a speech application, the control arrangement being operative to pass said output recognition-hypothesis stream to the speech application.

23. A system according to claim 16, wherein the first speech recognizer is trainable to a user'"'"'s voice and the second speech recognizer is intended to recognize a specific domain or application vocabulary spoken by different users without being training to their voices.

24. A system according to claim 16, wherein the transfer arrangement is operative to pass to the second speech recognizer only those portions of the speech input stream for which the confidence measure is below the acceptability threshold.

25. A system according to claim 16, wherein the transfer arrangement is operative to pass to the second speech recognizer only those portions of the speech input stream for which the confidence measure is below the acceptability threshold, the system further comprising a further acceptability-determination subsystem for determining a confidence measure for each second recognition hypothesis, and the control arrangement being operative to form the output recognition-hypothesis stream by including all the first and second recognition hypotheses together with confidence measures at least for the second recognition hypotheses and the corresponding first recognition hypotheses.

26. A system according to claim 16, wherein the transfer arrangement is operative to pass to the second speech recognizer only those portions of the speech input stream for which the confidence measure is below the acceptability threshold, the system further comprising a further acceptability-determination subsystem for determining a confidence measure for each second recognition hypothesis, and the control arrangement being operative to form the output recognition-hypothesis stream by taking the first recognition hypotheses and replacing a first recognition hypothesis with the corresponding second recognition hypothesis only when the confidence measures associated with the two hypotheses indicate at least a degree more confidence in the second recognition hypothesis as compared to the corresponding first recognition hypothesis.

27. A system according to claim 16, wherein the transfer arrangement is operative to pass all portions of the speech input stream to the second speech recognizer, the control arrangement being operative to form the output recognition-hypothesis stream by replacing all those first recognition hypotheses that have confidence measures below said acceptability threshold by the corresponding second hypotheses.

28. A system according to claim 16, further comprising a further acceptability-determination subsystem for determining a confidence measure for each second recognition hypothesis, the transfer arrangement being operative to pass all portions of the speech input stream to the second speech recognizer, and the control arrangement being operative to form the output recognition-hypothesis stream by replacing a first recognition hypothesis with a corresponding second recognition hypothesis only when the confidence measures associated with the two hypotheses indicate at least a degree more confidence in the second recognition hypothesis as compared to the corresponding first recognition hypothesis.

29. A system according to claim 16, further comprising a further acceptability-determination subsystem for determining a confidence measure for each second recognition hypothesis, the transfer arrangement being operative to pass all portions of the speech input stream to the second speech recognizer, and the control arrangement being operative to form the output recognition-hypothesis stream by including:

all the first recognition hypotheses,

the second recognition hypotheses for which the confidence measures of the corresponding first recognition hypotheses are below their acceptability threshold, and

the confidence measures at least for the included second recognition hypotheses and the corresponding first recognition hypotheses.