Method and apparatus for improved speech recognition with supplementary information

• US 20050049860A1
• Filed: 08/29/2003
• Published: 03/03/2005
• Est. Priority Date: 08/29/2003
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Chengalvarayan, Rathinavelu, Junqua, Jean-Claude, Contolini, Matteo, Kuhn, Roland
• Granted Patent: US 6,983,244 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 1/271 controlled by voice recognition

Abstract

A method for improving recognition results of a speech recognizer uses supplementary information to confirm recognition results. A user inputs speech to a speech recognizer. The speech recognizer resides on a mobile device or on a server at a remote location. The speech recognizer determines a recognition result based on the input speech. A confidence measure is calculated for the recognition result. If the confidence measure is below a threshold, the user is prompted for supplementary data. The supplementary data is determined dynamically based on ambiguities between the input speech and the recognition result, wherein the supplementary data will distinguish the input speech over potential incorrect results. The supplementary data may be a subset of alphanumeric characters that comprise the input speech, or other data associated with a desired result, such as an area code or location. The user may provide the supplementary data verbally, or manually using a keypad, touchpad, touchscreen, or stylus pen.

Claims

1. A method for improving recognition results of a speech recognizer at a remote location, comprising:

receiving input speech at the remote location from a user;

determining one or more candidate matches based on the input speech;

prompting the user for supplementary data associated with the input speech;

receiving the supplementary data from the user; and

selecting one of the one or more candidate matches based on the input speech and the supplementary data.

2. The method of claim 1 further comprising interpreting the input speech at a speech recognizer.

3. The method of claim 1 further comprising generating a confidence measure for the one or more candidate matches based on the input speech.

4. The method of claim 3 wherein the user is prompted for the supplementary data if the confidence measure is below a threshold.

5. The method of claim 1 wherein the supplementary data is a subset of alphanumeric characters comprising the input speech.

6. The method of claim 1 wherein the supplementary data is at least one of an area code, a location, and initials of a name.

7. The method of claim 1 wherein receiving the supplementary data includes receiving the supplementary data through at least one of verbal input and manual input.

8. The method of claim 7 wherein manual input includes input from at least one of a keypad, a touchpad, a touchscreen, and a stylus pen.

9. The method of claim 1 further comprising prompting the user to confirm the selected candidate match.

10. The method of claim 1 further comprising establishing a communication between the user and an entity associated with the one of the one or more candidate matches.

11. The method of claim 10 wherein establishing the communication includes initiating at least one of a telephone call, an electronic mail message, a voicemail message, and a text message.

12. The method of claim 10 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

13. The method of claim 1 wherein the supplementary data is related to ambiguities of the input speech.

14. The method of claim 3 further comprising modifying the confidence measure according to a speech model that is indicative of the user.

15. The method of claim 1 wherein determining one or more candidate matches includes determining one or more candidate matches based on a recognition history of the one or more candidate matches, wherein the recognition history indicates if one of the one or more candidate matches is difficult for a speech recognizer to interpret.

16. The method of claim 15 further comprising generating a confidence measure for the one or more candidate matches based on the input speech and the recognition history and prompting the user for the supplementary data if the confidence measure is below a threshold.

17. The method of claim 16 further comprising adjusting at least one of the confidence measure and the threshold based on the recognition history.

18. The method of claim 1 wherein the remote location is a directory assistance service.

19. The method of claim 1 further comprising determining an approximate match between the supplementary data and the one or more candidate matches based on at least one of typographical errors and misspellings in the supplementary data received from the user.

20. The method of claim 1 further comprising determining an approximate match between the input speech and the one or more candidate matches based on at least one of the supplementary data and mispronunciations in the input speech.

21. The method of claim 2 wherein the speech recognizer interprets the input speech based in part on the supplementary data.

22. The method of claim 1 further comprising providing information about an entity associated with the one of the one or more candidate matches to the user.

23. The method of claim 22 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

24. The method of claim 22 wherein the information includes at least one of navigation information, directory assistance information, company information, and contact information relating to the entity.

25. The method of claim 1 further comprising prompting the user to input information to be transmitted to an entity associated with the one of the one or more candidate matches.

26. The method of claim 25 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

27. The method of claim 1 wherein the input speech is a subset of the one or more candidate matches.

45. The method of claim 4.4 wherein manual input includes input from at least one of a keypad, a touchpad, a touchscreen, and a stylus pen.

28. A method for improving recognition results of a speech recognizer in an electronic device, comprising:

receiving supplementary data from a user at the device;

determining one or more candidate matches based on the supplementary data;

prompting the user for input speech associated with the supplementary data;

receiving the input speech from the user; and

selecting one of the one or more candidate matches based on the input speech and the supplementary data.

29. The method of claim 28 wherein the supplementary data is a subset of alphanumeric characters comprising the input speech.

30. The method of claim 28 wherein the supplementary data is at least one of an area code, a location, and initials of a name.

31. The method of claim 28 wherein receiving the supplementary data includes receiving the supplementary data through at least one of verbal input and manual input.

32. The method of claim 31 wherein manual input includes input from at least one of a keypad, a touchpad, a touchscreen, and a stylus pen.

33. The method of claim 28 further comprising interpreting the input speech at a speech recognizer, wherein the speech recognizer interprets the input speech according to the supplementary data.

34. The method of claim 33 wherein the speech recognizer interprets the input speech based in part on the supplementary data.

35. A method for improving recognition results of a speech recognizer in an electronic device, comprising:

receiving input speech from a user at the device;

determining one or more candidate entries from a plurality of candidate entries based on the input speech;

receiving the supplementary data from the user at the device; and

selecting one of the one or more candidate entries based on the input speech and the supplementary data.

36. The method of claim 35 further comprising interpreting the input speech at a speech recognizer.

37. The method of claim 35 further comprising generating a confidence measure for the one or more candidate entries based on the input speech.

38. The method of claim 37 wherein the user is prompted for the supplementary data if the confidence measure is below a threshold.

39. The method of claim 35 wherein the plurality of candidate entries comprise a database on the device.

40. The method of claim 35 wherein the supplementary data is a subset of alphanumeric characters comprising the input speech.

41. The method of claim 35 wherein the supplementary data is at least one of an area code, a location, and initials of a name.

42. The method of claim 35 wherein the device is at least one of a mobile telephone and a personal data assistant.

43. The method of claim 35 further comprising prompting the user to confirm the selected candidate entry.

44. The method of claim 35 wherein receiving the supplementary data includes receiving the supplementary data through at least one of verbal input and manual input.

46. The method of claim 35 further comprising prompting the user to input information to be transmitted to an entity associated with the one of the one or more candidate matches.

47. The method of claim 35 further comprising establishing a communication between the user and an entity associated with the one of the one or more candidate matches.

48. The method of claim 47 wherein establishing the communication includes initiating at least one of a telephone call, an electronic mail message, a voicemail message, and a text message.

49. The method of claim 35 further comprising providing information about an entity associated with the one of the one or more candidate matches to the user.

50. The method of claim 49 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

51. The method of claim 49 wherein the information includes at least one of navigation information, directory assistance information, company information, and contact information relating to the entity.

52. The method of claim 47 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

53. The method of claim 46 wherein the entity is at least one of a company, a government agency, one or more persons, a location, and a service.

54. A system for directing telephone calls based on input speech comprising:

a speech recognizer that receives input speech from a remote user;

a database that includes a plurality of entries;

a controller that communicates with the speech recognizer and the database to select one or more candidate entries from the plurality of entries based on the input speech, wherein the controller determines supplementary data based on ambiguities between the input speech and the one or more candidate entries and prompts the user for the supplementary data.

55. The system of claim 54 wherein the controller generates a confidence measure, and wherein the controller prompts the user if the confidence measure is below a threshold.

56. The system of claim 54 further comprising one or more speech models, wherein the controller selects the one or more candidate entries based on the one or more speech models, the input speech, and the supplementary data.

57. The system of claim 54 wherein the supplementary data is a subset of alphanumeric characters comprising the input speech.

58. The system of claim 54 wherein the controller dials one of a telephone number and a telephone extension based on at least one of the input speech and the supplementary data.