Method for efficient, safe and reliable data entry by voice under adverse conditions

• US 6,996,528 B2
• Filed: 08/03/2001
• Issued: 02/07/2006
• Est. Priority Date: 08/03/2001
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Rigazio, Luca, Junqua, Jean-Claude, Veprek, Peter, Boman, Robert C., Morin, Philippe R.
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US09/921,766
• Publication Number:
  US 20030033146A1
• Time in Patent Office: 1,649 Days
• Field of Search: 704/244, 704/246, 704/249, 704/251, 704/254, 704/270, 704/273, 704/275, 340/539.11, 340/539.13, 235/384
• US Class Current: 704/254
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A method and apparatus for data entry by voice under adverse conditions is disclosed. More specifically it provides a way for efficient and robust form filling by voice. A form can typically contain one or several fields that must be filled in. The user communicates to a speech recognition system and word spotting is performed upon the utterance. The spotted words of an utterance form a phrase that can contain field-specific values and/or commands. Recognized values are echoed back to the speaker via a text-to-speech system. Unreliable or unsafe inputs for which the confidence measure is found to be low (e.g. ill-pronounced speech or noises) are rejected by the spotter. Speaker adaptation is furthermore performed transparently to improve speech recognition accuracy. Other input modalities can be additionally supported (e.g. keyboard and touch-screen). The system maintains a dialogue history to enable editing and correction operations on all active fields.

Claims

1. A method of data entry by voice under adverse conditions for efficient and robust form filling, the method comprising:

communicating an input utterance from a speaker to a speech recognition means;

spotting a plurality of spotted words of at least two recognized spoken words within the input utterance, wherein the spotted words form a phrase containing at least one of field-specific values or commands;

obtaining blocks of text from input utterances separated from other input utterances by natural speech pauses, exactly one block of text per input utterance;

determining whether a block of text contains recognized values as opposed to commands;

populating a license plate number field of a form by concatenating, based on a sequence of input utterances from which the blocks of text are obtained, blocks of text determined to contain recognized values;

echoing blocks of text back to the speaker via a text-to-speech system, wherein audio feedback echoing the blocks of text is performed upon interpretation of each input utterance, and a sequence of recognized values echoed in the audio feedback reflects a sequence of spotted words within the input utterance from which the recognized values are obtained;

rejecting unreliable or unsafe inputs for which a confidence measure is found to be low; and

maintaining a dialogue history enabling editing operations and correction operations on all active fields.

2. The method of claim 1, further comprising the step of determining a focus field based on word semantic, including distinguishing between the license plate number field and a state field based on recognition of a state name value as opposed to recognition of letter and number values.

3. The method of claim 1, wherein automatic adaptation is performed once a complete form has been filled and sent for search in a database.

4. The method of claim 1, wherein a backup input system is accommodated for additional safety and flexibility.

5. The method of claim 1, wherein commands include at least one of a correction command for deletion of a last data entry, a deletion command for clearing of an entire output form buffer with restoration of all default values, a repeat command for echoing of at least one of the contents of an entire form or the contents of an entire form field as output speech or a send command for flushing of an entire output form buffer to a communication module.

6. The method of claim 1, wherein field-specific values include at least one of letters or numbers for a license plate number field, numbers for a license plate year field, at least one of state names or state name abbreviations for a license plate state field, or at least one of vehicle make names or vehicle model names for a license plate vehicle type field.

7. The method of claim 1, wherein editing operations include at least one of replacement of the contents of a field with a field-specific value or concatenation with contents of a field of a field-specific value.

8. The method of claim 1, wherein correction operations include at least one of deleting a last data entry or clearing an entire output form buffer, wherein clearing of an entire output form buffer results in restoration of default values.

9. The method of claim 1, further comprising providing a full duplex dialogue interaction including speech recognition and passive, auditory feedback.

10. The method of claim 1, further comprising using a dialogue model that provides feedback to the speaker of each recognized block of text, affording the speaker an opportunity to correct recognition errors by:

(a) speaking a command operable to designate a particular one of plural recognized blocks of text in the license plate field for replacement; and

(b) providing a subsequent input utterance containing field specific values to replace the particular recognized block of text in the license plate field without replacing at least one other recognized block of text in the license plate field.

11. The method of claim 10, further comprising treating output from a recognizer as entries to a dialogue system.

12. An article of manufacture for data entry by voice under adverse conditions enabling efficient and robust form filling, the article of manufacture comprising:

an operating system;

a memory in communication with said operating system;

a speech recognition means in communication with said operating system;

a speech generation means in communication with said operating system; and

a dialogue history maintenance means in communication with said operating system,

wherein said operating system manages said memory, said speech recognition means, said speech generation means, and said dialogue history maintenance means in a manner permitting the user to monitor speech recognition of an input utterance by means of a generated speech corresponding to at least one of field-specific values or commands contained within the phrase formed by spotted words within the input utterance, and to perform editing operations and correction operations on all active fields, wherein audio feedback echoing at least one of recognized values or recognized commands is performed upon interpretation of each input utterance, and a sequence of recognized values echoed in the audio feedback reflects a sequence of spotted words within the input utterance from which the recognized values are obtained,

said article of manufacture further comprising dialogue management means for using a dialogue model that provides feedback to the speaker of each recognized block of text obtained from an input utterance separated from other input utterances by natural speech pauses, exactly one block of text per input utterance, thereby affording the speaker an opportunity to correct recognition errors by;

(a) speaking a command operable to designate a particular one of plural recognized blocks of text in a license plate field for replacement; and

(b) providing a subsequent input utterance containing field specific values to replace the particular recognized block of text in the license plate field without replacing at least one other recognized block of text in the license plate field.

13. The article of manufacture of claim 12, further comprising a user interface, wherein said user interface provides a backup input system for additional safety and flexibility.

14. The article of manufacture of claim 13, wherein said user interface includes at least one of a keyboard, an active display or a touch screen.

15. The article of manufacture of claim 12, wherein the speech generation means includes at least one of a speech synthesizer or reproduction of a previously recorded voice.

16. The article of manufacture of claim 12, wherein a focus field is determined based on word semantic.

17. The article of manufacture of claim 12, wherein automatic adaptation is performed once a complete form has been filled and sent for search in a database.

18. The article of manufacture of claim 12, wherein commands include at least one of a correction command for deletion of a last data entry, a deletion command for clearing of an entire output form buffer with restoration of all default values, a repeat command for echoing of at least one of the contents of an entire form and the contents of an entire form field as output speech, or a send command for flushing of an entire output form buffer to a communication module.

19. The article of manufacture of claim 12, wherein field-specific values include at least one of letters or numbers for a license plate number field, numbers for a license plate year field, at least one of state names or state name abbreviations for a license plate state field, or at least one of vehicle make names or vehicle model names for a license plate vehicle type field.

20. The article of manufacture of claim 12, wherein editing operations include at least one of replacement of the contents of a field with a field-specific value or concatenation with contents of a field of a field-specific value.

21. The article of manufacture of claim 12, wherein correction operations include at least one of deleting a last data entry or clearing an entire output form buffer, wherein clearing of an entire output form buffer results in restoration of default values.

22. The article of manufacture of claim 12, further comprising a dialogue management means for providing a full duplex dialogue interaction including speech recognition and passive, auditory feedback.

23. The article of manufacture of claim 12, wherein said dialogue model treats output from said speech recognition means as entries to said dialogue management means.