Vehicle control

• US 8,311,827 B2
• Filed: 09/19/2008
• Issued: 11/13/2012
• Est. Priority Date: 09/21/2007
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Scarlatti, David, Molina, Roberto, Hernandez, Ruben San-Segundo, Lopez, Javier Ferreiros, Villar, Victor Perez
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US12/233,890
• Publication Number:
  US 20090083034A1
• Time in Patent Office: 1,516 Days
• Field of Search: 704/200, 704/207, 704/211, 704/251, 704/270, 704/275
• US Class Current: 704/251
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

The present invention relates to voice-activated vehicle control, and to the control of UAVs (unmanned air vehicles) using speech in particular. A method of controlling a vehicle is provided that includes receiving one or more instructions issued as speech and analyzing the speech using speech recognition software to provide a sequence of words and a word confidence measure for each word so recognized. The sequence of words is analyzed to identify a semantic concept corresponding to an instruction based on the analysis, and a semantic confidence level for the semantic concept identified derived at least in part with reference to the word confidence measures of the words associated with the semantic concept. A spoken confirmation of the semantic concept so identified based on the semantic confidence level is provided, and the semantic concept is used to provide a control input for the vehicle.

Claims

1. A computer-implemented method of controlling a vehicle, comprising:

receiving one or more instructions issued as speech;

analyzing the speech using speech recognition software to provide a sequence of words and a word confidence measure for each word so recognized;

analyzing the sequence of words to identify a semantic concept corresponding to an instruction based on the analysis, and a semantic confidence level for the identified semantic concept derived at least in part with reference to the word confidence measures of the words associated with the semantic concept;

providing a spoken confirmation of the semantic concept so identified based on the semantic confidence level and an indicated verbosity level, the spoken confirmation being provided with one of a speaking rate or a pitch that is increased as the indicated verbosity level decreases; and

using the semantic concept so identified to provide a control input for the vehicle;

wherein the step of providing the spoken confirmation of the semantic concept comprises indicating that the instruction was not understood when the semantic confidence level is below a threshold, or the step of using the semantic concept comprises providing the control input for the vehicle when the semantic confidence level exceeds the threshold.

2. The method of claim 1, wherein the step of analyzing the speech to provide a sequence of words comprises using continuous hidden Markov models.

3. The method of claim 1, wherein the step of analyzing the speech to provide a sequence of words comprises using a language model that provides information about the most likely sequences of words expected to be found.

4. The method of claim 3, wherein the step of analyzing the speech to provide a sequence of words comprises using the language 30 model to provide a 2-gram statistical model.

5. The method of claim 1, wherein the step of analyzing the sequence of words comprises analyzing the sequence of words and the associated word confidence measures to identify the semantic concept.

6. The method of claim 5, wherein the semantic concept comprises an identifier and a value.

7. The method of claim 5, wherein the step of analyzing the sequence of words comprises:

identifying the semantic concept using a bottom-up approach starting with an analysis of each word identified individually and then extending the analysis to neighboring words.

8. The method of claim 5, wherein the step of analyzing the sequence of words to identify the semantic concept comprises:

tagging each word according to its type, and looking for known patterns from the sequence of tags, thereby allowing deduction of the semantic concept.

9. The method of claim 1, further comprising:

providing the spoken confirmation of the identified semantic concept based on the semantic confidence level and an urgency level, the verbosity of the spoken confirmation decreasing with increasing urgency level.

10. The method of claim 9, wherein the providing the spoken confirmation step comprises providing the spoken confirmation of the identified semantic concept with at least one of the speaking rate or the pitch that is increased as the urgency level increases.

11. An apparatus for controlling a vehicle, comprising:

an input arranged to receive one or more instructions issued as speech;

a memory arranged to store the received speech;

a speech recognition module arranged to analyze the speech to provide a sequence of words and a word confidence measure for each word recognized;

a natural language understanding module arranged to receive the sequence of words and the word confidence measures, and analyze the sequence of words to identify a semantic concept corresponding to an instruction based on the analysis and a semantic confidence level for the identified semantic concept derived at least in part with reference to the word confidence measures of the words associated with the semantic concept;

a response generation module arranged to provide a spoken confirmation of the identified semantic concept based on the semantic confidence level and an indicated verbosity level; and

a command generation module arranged to use the identified semantic concept to provide a control input for the vehicle;

wherein the response generation module is arranged to provide the spoken confirmation of the identified semantic concept to indicate that the instruction was not understood when the semantic confidence level is below a threshold, or the command generation module is arranged to use the identified semantic concept to provide a control input for the vehicle when the semantic confidence level exceeds the threshold, and

the response generation module is arranged to provide the spoken confirmation of the identified semantic concept with at least one of a speaking rate or a pitch that is increased as the indicated verbosity level decreases.

12. The apparatus of claim 11, further comprising an acoustic model database operably coupled to the speech recognition module, wherein the acoustic model database has hidden Markov models stored therein, and additionally has optionally continuous hidden Markov models stored therein.

13. The apparatus of claim 11, further comprising a language model database operably coupled to the speech recognition module, wherein the language model database has data stored therein that provides information about the most likely sequences of words expected to be found.

14. The apparatus of claim 13, wherein the language model database has a 2-gram statistical model stored therein.

15. The apparatus of claim 11, wherein the natural language understanding module is arranged to analyze the sequence of words and the associated word confidence measures to identify the semantic concept.

16. The apparatus of claim 11, wherein the natural language understanding module is arranged to provide the semantic concept as an identifier and a value.

17. The apparatus of claim 11, wherein the natural language understanding module is arranged to analyze the sequence of words to identify the semantic concept using a bottom-up approach starting with an analysis of each word identified individually and then extending the analysis to neighboring words.

18. The apparatus of claim 11, wherein the natural language understanding module is arranged to analyze the sequence of words to identify the semantic concept by tagging each word according to at least one of its type, command, digit, or letter, and to look for known patterns from the sequence of tags, thereby allowing deduction of the semantic concept.

19. The apparatus of claim 11, wherein the response generation module is arranged to provide the spoken confirmation of the identified semantic concept based on the semantic confidence level and an urgency level, the verbosity of the spoken confirmation decreasing with increasing urgency level.

20. The apparatus of claim 19, wherein the response generation module is arranged to provide the spoken confirmation of the identified semantic concept with at least one of a speaking rate or a pitch that is increased as the urgency level increases.

21. The apparatus of claim 11, wherein the response generation module has associated with it a database containing response templates for use in constructing the spoken confirmation.

22. A computer-implemented method of controlling a vehicle, comprising:

receiving one or more instructions issued as speech;

analyzing the speech using speech recognition software to provide a sequence of words and a word confidence measure for each word so recognized;

analyzing the sequence of words to identify a semantic concept corresponding to an instruction based on the analysis, and a semantic confidence level for the identified semantic concept derived at least in part with reference to the word confidence measures of the words associated with the semantic concept;

providing a spoken confirmation of the semantic concept so identified based on the semantic confidence level, the spoken confirmation having one of a speaking rate or a pitch adjusted according to an indicated response length; and

using the semantic concept so identified to provide a control input for the vehicle;

wherein the step of providing the spoken confirmation of the semantic concept comprises indicating that the instruction was not understood when the semantic confidence level is below a threshold, or the step of using the semantic concept comprises providing the control input for the vehicle when the semantic confidence level exceeds the threshold.

23. The method of claim 22, wherein the step of analyzing the speech to provide a sequence of words comprises using continuous hidden Markov models.

24. The method of claim 22, wherein the step of analyzing the speech to provide a sequence of words comprises using a language model that provides information about the most likely sequences of words expected to be found.

25. The method of claim 22, wherein the step of analyzing the speech to provide a sequence of words comprises using the language 30 model to provide a 2-gram statistical model.