Device for Modifying and Improving the Behaviour of Speech Recognition Systems

• US 20090086934A1
• Filed: 08/18/2008
• Published: 04/02/2009
• Est. Priority Date: 08/17/2007
• Status: Abandoned Application

• Current Assignee: Envox International Limited
• Original Assignee: Fluency Voice Limited
• Inventors: Thomas, Trevor
• Application Number:
  US12/193,373
• Publication Number:
  US 20090086934A1
• Time in Patent Office: Days
• US Class Current: 379/88.10
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques

Abstract

A speech recognition system comprises a controller application, a speech recogniser, and a modification apparatus disposed between them. The controller application has an output arranged to produce a speech signal, an input arranged to receive a recognised speech result signal and a control command output arranged to produce a command signal. The speech recogniser has a pattern matcher, a speech input arranged to receive the speech signal from the controller application, an output arranged to produce the recognised speech recognised signal and a control command input arranged to receive the command signal. The modification apparatus is disposed between the output of the controller application and the input of the speech recogniser; between the output of the speech recogniser and the input of the controller application; and between the control command output of the controller application and the control command input of the speech recogniser. The modification apparatus is arranged to modify at least one of the signals between the controller application and the speech recogniser. In this way speech recognition accuracy can be significantly improved.

Claims

1. A speech recognition system comprising

a controller application having an output arranged to produce a speech signal, an input arranged to receive a recognized speech result signal, and a control command output arranged to produce a command signal;

a speech recognizer having a pattern matcher, a speech input arranged to receive the speech signal from the controller application, an output arranged to produce the recognized speech result signal and a control command input arranged to receive the command signal,

characterised by a modification apparatus disposed between the output of the controller application and the input of the speech recognizer;

between the output of the speech recognizer and the input of the controller application; and

between the control command output of the controller application and the control command input of the speech recognizer, the modification apparatus arranged to modify at least one of the signals between the controller application and the speech recognizer.

2. A speech recognition system according to claim 1, further comprising a database linked to the modification apparatus.

3. A speech recognition system according to claim 2, wherein the database is structured to store data concerning individuals including characteristics of that individual'"'"'s speech or of system settings or grammars which are more appropriate for recognition of that individual'"'"'s speech.

4. A speech recognition system according to claim 3, wherein the database is further structured to store data concerning individuals for use by the speech recogniser.

5. A speech recognition system according to claim 3, wherein the database is further structured to store data concerning individuals for use by the modification apparatus.

6. A speech recognition system according to claim 3, wherein the database is further structured to store data concerning individuals for use by the controlling application.

7. A speech recognition system according to claim 1, further comprising memory arranged to store data.

8. A speech recognition system according to claim 1, further comprising a secondary speech recogniser connected to the modification apparatus.

9. A speech recognition system according to claim 1, further comprising a second pattern matcher connected to the modification apparatus.

10. A speech recognition system according to claim 1, further comprising a speaker verification apparatus connected to at least one of the modification apparatus and the speech recogniser.

11. A speech recognition system according to any claim 1, further comprising a speech signal measuring device.

12. A speech recognition system according to claim 1, wherein the modification apparatus is disposed between the control command output of the controller application and the control command input of the speech recognizer, the modification apparatus arranged to modify the command signal between the controller application and the speech recognizer.

13. A system according to claim 12, wherein the modification of the command signal causes change of operation of speech recognizer.

14. A system according to claim 12, wherein the modification apparatus adjusts variables for operation of the speech recognizer.

15. A system according to claim 14, wherein a variable is the length of time the speech recognizer waits after the caller has finished talking before returning the results back to the controller application.

16. A system according to claim 12, further including a caller characteristic assessment unit arranged to determine characteristics of a caller'"'"'s speech and to determine what changes can be made to the command signal.

17. A system according to claim 15, wherein the caller characteristic assessment unit is part of the speech recognizer.

18. A system according to claim 16, wherein the caller characteristic assessment unit is a speech signal power level detector arranged to detect whether a person is speaking or not.

19. A system according to claim 18, wherein the speech signal power detector is a signal processor.

20. A system according to claim 19, wherein the signal processor is arranged to detect when the power of the speech signal crosses a threshold.

21. A system according to claim 18, wherein the detector measures the length of the pause between spoken words.

22. A system according to claim 16, wherein the caller characteristic assessment unit is arranged to determine whether a default length of time the speech recognizer waits after the caller has finished talking before returning the results back to the controller application is appropriate, and if it is not, determines how long it should be.

23. A system according to claim 22, wherein the assessment unit is arranged to pass the length of time to the modification apparatus.

24. A system according to claim 23, wherein the modification apparatus is arranged to store the length of time in the database.

25. A system according to claim 15, wherein the modification apparatus is arranged to modify the length of time the speech recognizer waits after the caller has finished talking before returning the results back to the controller application based on the characteristics of the speech.

26. A system according to claim 14, wherein a variable is the use of a particular grammar by the speech recogniser.

27. A system according to claim 16, wherein the assessment unit is arranged to determine what grammar is appropriate to the characteristics of the speech signal of a speaker.

28. A system according to claim 27, wherein the assessment unit is arranged to identify whether or not a speaker'"'"'s speech signal includes the characteristic of start-up and/or close-down phrases.

29. A system according to claim 27, wherein the assessment unit is arranged to identify whether or not a speaker'"'"'s speech signal includes the characteristic of a dialect.

30. A system according to claim 27, wherein the assessment unit is arranged to pass the identity of the most appropriate grammar to the modification apparatus.

31. A system according to claim 30, wherein the modification apparatus is arranged to store the identity of the most appropriate grammar in the database.

32. A system according to claim 26, wherein the modification apparatus is arranged to modify the grammar before passing the speech signal to the speech recogniser.

33. A speech recognition system according to claim 1, wherein the modification apparatus is disposed between the speech signal output of the controller application and the speech input of the speech recognizer, the modification apparatus arranged to modify the speech signal between the controller application and the speech recognizer.

34. A speech recognition system according to claim 33, wherein the modification apparatus is arranged to store the speech signal.

35. A speech recognition system according to claim 34, wherein the modification apparatus stores the speech signal in a database.

36. A speech recognition system according to claim 34, wherein the modification apparatus stores the speech signal in memory.

37. A speech recognition system according to claim 34, wherein the modification apparatus is arranged to direct the speech signal to the speech recogniser for recognition.

38. A speech recognition system according to claim 37, wherein the modification apparatus is arranged to return at least a part of the speech result signal to the controller application.

39. A speech recognition system according to claim 38, wherein the controller application is arranged to confirm the part of the speech result with the speaker.

40. A speech recognition system according to claim 39, wherein, if the speech result is wrong, the controller application is arranged to ask for the part of the speech again.

41. A speech recognition system according to claim 40, wherein the modification apparatus is arranged to send the new speech signal and the stored speech signal to the speech recogniser for recognition.

42. A speech recognition system according to claim 41, wherein the modification apparatus is arranged to send a speech signal to the speech recogniser, part of which is from the new speech signal, and part of which is from the stored speech signal.

43. A speech recognition system according to claim 38, wherein the part of the signal is the postcode, and the rest of the speech signal is other address information.

44. A speech recognition system according to claim 1, wherein the modification apparatus is disposed between the speech result signal output of the speech recogniser and the speech result input of the controller application, the modification apparatus arranged to modify the speech result signal between the speech recognizer and the controller application.

45. A speech recognition system according to claim 44, wherein the modification apparatus is arranged to store the speech result signal.

46. A speech recognition system according to claim 45, wherein the modification apparatus stores the speech result signal in a database.

47. A speech recognition system according to claim 45, wherein the modification apparatus stores the speech result signal in memory.

48. A speech recognition system according to claim 45, wherein the modification apparatus is arranged to direct the speech signal to the speech recogniser for recognition.

49. A speech recognition system according to claim 48, wherein the modification apparatus is arranged to return at least a part of the speech result signal to the controller application.

50. A speech recognition system according to claim 49, wherein the controller application is arranged to confirm the part of the speech result with the speaker.

51. A speech recognition system according to claim 50, wherein, if the speech result is wrong, the controller application is arranged to ask for the part of the speech again.

52. A speech recognition system according to claim 51, wherein the modification apparatus is arranged to send the new speech signal and the stored speech signal to the speech recogniser for recognition.

53. A speech recognition system according to claim 52, wherein the modification apparatus is arranged to send the new speech signal to the speech recogniser.

54. A speech recognition system according to claim 53, wherein the modification apparatus is arranged to combine the speech result signal from the speech recogniser with the a part of the stored speech signal.

55. A speech recognition system according to claim 54, wherein the speech result signal is the postcode, and the part of the stored speech signal is other address information.

56. A speech recognition system according to claim 44, wherein the modification device is arranged to identify defective parts of a speech signal and to modify the results signal to add an indication identifying the location corresponding to the defective part of the speech signal.

57. A speech recognition system according to claim 56, wherein the modification is arranged to identify at least one of the following as defects:

missing packets;

errors in a packet; and

the presence of noise exceeding a noise threshold.

58. A speech recognition system according to claim 57, wherein the modification apparatus includes a counter for counting packets so as to be able to identify missing packets.

59. A speech recognition system according to claim 57, wherein the modification apparatus is arranged to read error codes in packets to identify packets containing errors.

60. A speech recognition system according to claim 57, wherein the modification apparatus includes a noise detection device arranged to measure the noise level in the speech signal.

61. A speech recognition system according to claim 56, wherein the modification apparatus adds a marker to the result signal in a position corresponding to the defect.

62. A speech recognition system according to claim 56, wherein the modification device includes a word confidence score generator arranged to generate a word confidence score corresponding to words in the results signal where a defect is present.

63. A speech recognition system according to claim 56, wherein the controller application is arranged to identify words in the results signal where a defect is present.

64. A speech recognition system according to claim 63, wherein the controller application is arranged to identify important words from the result signal.

65. A speech recognition system according to claim 64, wherein the controller application is arranged to identify important words which include a defect.

66. A speech recognition system according to claim 65, wherein the controller application is arranged to ask a person a confirmatory question where an important word includes a defect.

67. A speech recognition system according to claim 1 wherein the modification apparatus is disposed between the speech signal output of the controller application and the speech input of the speech recogniser, and between the control command output of the controller application and the control command input of the speech recogniser, the modification apparatus arranged to modify the speech signal between the controller application and the speech recogniser and to modify the command signal between the controller application and the speech recogniser.

68. A system according to claim 67 wherein the modification of the command signal causes change of operation of the speech recogniser.

69. A system according to claim 67 wherein the modification apparatus is arranged to store the speech signal.

70. A system according to claim 69, wherein the modification apparatus stores the speech signal in a database.

71. A system according to claim 69, wherein the modification apparatus stores the speech signal in memory.

72. A system according to claim 69 wherein the modification apparatus is arranged to direct the speech signal to the speech recogniser for recognition.

73. A system according to claim 69 wherein the modification apparatus is arranged to identify a misrecognised word.

74. A system according to claim 75 wherein the modification apparatus sends the stored speech signal to the recogniser again and the modification apparatus modifies the command signal to the speech recogniser to cause it to carry out a recognition using a different grammar.

75. A system according to claim 67 wherein the speech signal is sent to the speech recogniser with a first processed command signal directing the speech recogniser to recognise a first part of the speech signal, and the speech signal is sent to the speech recogniser again with a second processed command signal directing the speech recogniser to recognise a second part of the speech signal.

76. A system according to claim 75 wherein, during the recognition of the first part of the speech signal a first grammar is used, and during recognition of the second part of the speech signal, a second grammar is used.

77. A system according to claim 67 further comprising a speech signal measuring device, wherein it can be determined whether or not a person is speaking.

78. A system according to claim 77, wherein, after a period of time has passed, the speech signal is passed to the speech recogniser for recognition.

79. A system according to claim 78, wherein, if a person starts speaking again before the results of recognition have been returned, the modification apparatus cancels the recognition which has already taken place.

80. A system according to claim 1, wherein the modification apparatus is disposed between the speech signal output of the controller application and the speech signal input of the speech recogniser and between the speech result signal output of the speech recogniser and the speech result of the input of the controller application, the modification apparatus arranged to modify the speech signal between the controller application and the speech recogniser and to modify the speech result signal between the speech recogniser and the controller application.

81. A system according to claim 1, wherein the modification apparatus is disposed between the speech result signal output of the speech recogniser and the speech result input of the controller application, and between the control command output of the controller application and the control command input of the speech recogniser, the modification apparatus arranged to modify the speech result signal between the speech recogniser and the controller application and arranged to modify the command signal between the controller application and the speech recogniser.

82. A system according to claim 1, wherein the modification apparatus is disposed between the speech signal output of the controller application and the speech signal input of the speech recogniser, between the speech result signal output of the speech recogniser and the speech result of the input of the controller application, and between the control command output of the controller application and the control command input of the speech recogniser, the modification apparatus arranged to modify the speech signal between the controller application and the speech recogniser, to modify the speech result signal between the speech recogniser and the controller application, and to modify the command signal between the controller application and the speech recogniser.

83. A method of recognising speech in a system comprising a controller application having an output arranged to produce a speech signal, an input arranged to receive a recognised speech result signal and a control command output arranged to produce a command signal;

and

a speech recogniser having a pattern matcher, a speech input arranged to receive the speech signal, an output arranged to produce the recognised speech result signal and a control command input arranged to receive the command signal,

characterised by the modification of at least one of the signals between the controller application and the speech recogniser.

84. A method according to claim 83 wherein the command signal is modified between the controller application and the speech recogniser.

85. A method according to claim 84 wherein the modification of the command signal causes a change in operation of the speech recogniser.

86. A method according to claim 84 wherein the modification adjusts variables for operation of the speech recogniser.

87. A method according to claim 86 wherein the variable is the length of time the speech recogniser waits after the caller has finished talking before returning the results to the controller application.

88. A method according to claim 84 further comprising determining the characteristics of a caller'"'"'s speech and determining what changes can be made to the command signal.

89. A method according to claim 88 wherein determining the characteristics of a caller'"'"'s speech is carried out by detecting the speech signal power level.

90. A method according to claim 89 further comprising detecting when the power of the speech signal crosses a threshold.

91. A method according to claim 89 further comprising measuring the length of the pause between spoken words.

92. A method according to claim 88 further comprising determining whether a default length of time the speech recogniser waits after the caller has finished talking before returning the results back to the controller application is appropriate, and if it is not, determining how long it should be.

93. A method according to claim 92 further comprising storing the length of time.

94. A method according to claim 93 in which the length of time is stored in a database.

96. A method according to claim 86 wherein a variable is the use of a particular grammar by the speech recogniser.

97. A method according to claim 88 further comprising determining what grammar is appropriate to the characteristics of the speech signal of a speaker.

98. A method according to claim 97 further comprising identifying whether or not a speaker'"'"'s speech signal includes the characteristic of start and/or close down phrases.

99. A method according to claim 97 further comprising identifying whether or not a speaker'"'"'s speech signal includes the characteristic of a dialect.

100. A method according to claim 97 further comprising identifying the most appropriate grammar to be used by a modification apparatus.

101. A method according to claim 100 further comprising the storing of the identity of the most appropriate grammar in a database.

102. A method according to claim 96, wherein the grammar is modified before passing the speech signal to the speech recognizer.

103. A method according to 83, wherein the speech signal between the controller application and the speech recogniser is modified.

104. A method according to claim 103 further comprising the storing of the speech signal.

105. A method according to claim 104 wherein the speech signal is stored in a database.

106. A method according to claim 104 wherein the speech signal is stored in memory.

107. A method according to claim 104 further comprising directing the speech signal to the speech recogniser for recognition.

108. A method according to claim 107 further comprising returning at least a part of the speech result signal to the controller application.

109. A method according to claim 108 further comprising confirming part of the speech result with the speaker.

110. A method according to claim 109 further comprising asking the speaker for the part of the speech again if the speech result is wrong.

111. A method according to claim 110 further comprising sending the new speech signal and the stored speech signal to the speech recogniser for recognition.

112. A method according to claim 111 further comprising sending a speech signal to the speech recogniser, part of which is from the new speech signal, and part of which is from the stored speech signal.

113. A method according to claim 108 wherein part of the signal is the post code, and the rest of the speech signal is other address information.

114. A method according to claim 83, further comprising modifying the speech result signal between the speech recogniser and the controller application.

115. A method according to claim 114 further comprising storing the speech result signal.

116. A method according to claim 115 further comprising storing the speech result signal in a database.

117. A method according to claim 115 further comprising storing the speech result signal in memory.

118. A method according to claim 115 further comprising directing the speech signal to the speech recogniser for recognition.

119. A method according to claim 118 further comprising returning at least a part of the speech result signal to the controller application.

120. A method according to claim 119 comprising confirming the part of the speech result with the speaker.

121. A method according to claim 120 further comprising asking the speaker for the part of the speech again if the speech result is wrong.

122. A method according to claim 121 further comprising sending the speech signal and the stored speech signal to the speech recogniser for recognition.

123. A method according to claim 122 further comprising sending the new speech signal to the new speech recognizer.

124. A method according to claim 123 further comprising combining the speech result signal from the speech recogniser with a part of the stored speech signal.

125. A method according to claim 124 wherein the speech result signal is the post code and the part of the stored speech signal is the other address information.

126. A method according to claim 114 further comprising identifying defective parts of a speech signal and modifying the results signal to add an indication identifying the location corresponding to the defective part of the speech signal.

127. A method according to claim 126 further comprising identifying at least one of the following as defects:

missing packets;

errors in a packet; and

the presence of noise exceeding a noise threshold.

128. A method according to claim 127 further comprising the identification of missing packets by use of a packet counter.

129. A method according to claim 127 further comprising reading error codes in packets to identify packets containing errors.

130. A method according to claim 127 further comprising measuring the noise level in the speech signal.

131. A method according to claim 126 further comprising adding a marker to the result signal in a position corresponding to the defect.

132. A method according to claim 83, wherein the speech signal and the command signal are modified.

133. A method according to claim 132, wherein the modification of the command signal causes a change in operation of the speech recogniser.

134. A method according to claim 132, further comprising storing the speech signal.

135. A method according to claim 134, further comprising storing the speech signal in a database.

136. A method according to claim 134, further comprising storing the speech signal in memory.

137. A method according to claim 134, further comprising directing the speech signal to the speech recogniser for recognition.

138. A method according to claim 134, further comprising identifying a misrecognised word.

139. A method according to claim 138, further comprising sending the stored speech signal to the recogniser again and modifying the command signal to the speech recogniser to cause it to carry out a recognition using a different grammar.

140. A method according to claim 132, further comprising sending the speech signal to the speech recogniser with a first processed command signal directing the speech recogniser to recognise a first part of the speech signal and sending the speech signal to the speech recogniser again with a second processed command signal directing the speech recogniser to recognise a second part of the speech signal.

141. A method according to claim 140, comprising using a first grammar during recognition of the first part of the speech signal, and using a second grammar during recognition of the second part of the speech signal.

95. A method according to claim 95 further comprising modifying the length of time the speech recogniser waits after the caller has finished speaking before returning the results to the controller application based on the characteristics of the speech.